On "Indexed Claude Chats Show Why Disallow Is Not Noindex"

The robots.txt disallow is working exactly as intended—it's blocking crawlers from seeing the noindex directive at all, which means Google never gets the signal to exclude the page from its index. This is a real gotcha for anyone assuming robots.txt alone handles indexation, but it also explains why these Claude chats leaked into search: the noindex tag was unreachable, invisible to Googlebot. If you're relying on disallow to keep sensitive content out of search, you need noindex on the publicly accessible version, robots.txt on the raw path, or both—one doesn't substitute for the other.