robots.txt is a request, not a fence
A local-services site disallows ClaudeBot in robots.txt. Since 2026-07-31, 60 of that crawler’s requests were served successfully anyway.
Some crawlers honor the directive; some read it as a suggestion. On one local-services site in the measured fleet, ClaudeBot is disallowed in robots.txt — and 60 of its requests since 2026-07-31 nonetheless came back 200, content delivered.
That cuts both ways. If exclusion is the intent, robots.txt alone does not enforce it; the edge has to. If inclusion is the intent, the pages that matter need to return useful 200s to the bots that matter. Either way the claim needs a request-level check, not a config review.
Measured continuously from 2026-07-31 onward on first-party edge analytics, one request at a time. As of 2026-08-07 the finding still stands.