Technical reference
Definitions and checklists for labels that are often over-interpreted in audits. Protocol meaning, platform guidance, and diagnostic inference stay separate.
Reference pages cite current primary documentation and show their review date. They clarify a result; they do not replace the source standard.
HTTP status codes for web diagnostics
A compact diagnostic reference for common 2xx, 3xx, 4xx, and 5xx responses, with protocol meaning separated from SEO inference.
Crawler user agents: claim, token, and verification
A reference for classifying crawler strings without treating an easily spoofed header as verified identity.
Robots directives in headers and HTML
Read robots meta elements and X-Robots-Tag response headers as page-level controls, with crawler access and conflicts kept visible.
Crawler verification methods compared
Compare user-agent matching, published IP ranges, forward-confirmed reverse DNS, operator tools, and request-log evidence without overstating identity.