Current processing boundary
The Log File Inspector, IP Address Check and Bot Request Verifier run in your browser. Pasted response headers, redirect blocks and robots.txt rules also stay local. The main crawler check and the optional public-URL modes in the Response Inspector, Redirect Chain tool and Robots Rule Tester send one public URL to the protected gateway described below. The Website Index Check is a separate bounded operation: it reads the submitted site's robots.txt and sitemap or homepage links, then checks at most 50 public URLs on that same site.
The crawler check keeps at most five successful reports in this browser for 24 hours so a later result can be compared with an earlier one. That local history contains the checked URL but is not uploaded as a history record. Copy and JSON export also happen in the browser.
Do not submit private, signed, password-reset, preview, or secret-bearing URLs. The full checked path and query can appear in the generated report, the short-lived gateway cache, browser-local history, clipboard content, or a downloaded JSON file.
Browser-local processing reduces exposure, but it does not make unsafe input harmless. Do not paste confidential logs into any device or browser you do not control. Results remain an interpretation of the supplied material—not proof of live server behavior.
Hosting boundary
The public site is delivered by Vercel. Loading a page necessarily sends connection and request metadata—such as the IP address, requested path, timestamp, user agent, and response status—to that hosting layer for delivery, reliability, and security. This is separate from the browser-local tool input, which the current application does not transmit.
Cookie-free Umami measurement records page use, performance and allowlisted product-event categories. It never receives submitted URLs, IPs, log lines, pasted technical evidence, report contents or local history. There is no advertising tag, account system, or contact form. The complete controller, hosting, event, retention, and data-subject information is published in the privacy notice.
Result vocabulary
Observed
A literal value found in the supplied evidence: a status line, header, canonical tag, address syntax, log request, user agent, or response code.
Inferred
A bounded interpretation based on those values. For example, a 301 plus a Location header suggests a redirect response. It does not prove that the destination succeeds.
Unknown
A question the current evidence cannot answer: live DNS, the final redirect hop, rendered DOM, current index state, precise location, identity, or future behavior.
Live crawler check and safety gates
AnalyseSpider exposes tightly bounded server-side checks through dedicated gateway operations and does not enqueue the Crawl Foundry Site Audit worker. The live production container is limited to 0.5 CPU and 256 MB RAM. Single-URL checks allow one concurrent run, six runs per ten minutes, and 30 per 24 hours for one pseudonymous client. The optional crawler-token comparison uses one additional run and the same network limits. A website scan allows one concurrent scan, two scans per ten minutes, and ten per 24 hours; each scan checks at most 50 same-site URLs with bounded internal concurrency and a one-hour result cache.
The public fetch path enforces all of the following:
- private, loopback, link-local, metadata, and internal DNS blocking;
- DNS-rebinding checks before and after every redirect;
- strict scheme, port, redirect-depth, response-size, and timeout caps;
- a proof-of-work challenge bound to the exact tool scope and normalized URL;
- exact allowed origins, rate limits, replay protection, a circuit breaker, and a scope kill switch;
- tests for redirect-to-private targets and changing DNS answers.
The result reports only what this HTTP fetch observed. It does not render JavaScript, enter logged-in areas, guarantee indexation, or measure actual AI mentions. The separate browser-local Bot Request Verifier can match a log-line IP and exact product token against dated ranges published by Google, OpenAI, Perplexity or Anthropic. No range match proves indexing, training or citation. No DataForSEO or paid LLM provider is used.
The Website Indexability Check can open a Google site: query for the whole domain or one exact URL. AnalyseSpider does not scrape or interpret Google results. After returning, the visitor can record “result seen” or “no result seen” in the current browser tab. The timestamped observation is included in a downloaded JSON report but is not uploaded. A visible result is supporting public evidence; an absent result remains inconclusive because Google documents that site: results are not exhaustive. The checked URL is sent to Google only after the visitor clicks the link.
Sources and change control
Historical archives are used only to establish broad former topic and URL intent. A paid target-level backlink export was used to prioritize which old jobs deserved a new successor. Neither source grants rights to old content or proves current search performance.
The repository publishes versioned rights and legacy-action manifests. Material method changes are dated. Corrections can be submitted through the current contact route.