Technical SEO evidence before conclusions

Use AnalyseSpider for the small evidence step before a larger crawl or platform investigation: isolate the requests, responses, directives, and unknowns that justify the next check.

Use this when

The output is a diagnostic handoff, not an automated verdict about indexing or ranking.

Useful starting jobs

Parse a bounded log sample, classify redirect and error patterns, inspect pasted response headers and HTML, or check whether an IP value is public, private, or reserved.

  • Crawler candidate inventory
  • Status and path distribution
  • Redirect and directive capture
  • Observed, inferred, and unknown handoff

Where this stops

AnalyseSpider can fetch one public HTTP or HTTPS URL through a tightly bounded, SSRF-protected gateway. It does not crawl a site, enter authenticated areas, render JavaScript, query Search Console, or decide canonical selection. Its separate browser-local verifier can compare one logged IP and user-agent claim with dated operator-published ranges; that is request-origin evidence, not proof of indexing, training, or citation.

Client logs can contain personal or confidential data. Minimise the sample before pasting it into any tool, including a local one, and never submit a private or signed URL to the live checker.

A practical sequence

  1. Reduce the sample to the affected time and route set.
  2. Inspect local observations and rejected rows.
  3. Write the strongest supported hypothesis.
  4. Verify it in the relevant server, crawler, search-platform, or application surface.