Google Explains Why URLs Blocked By Robots.txt Can Still Be Indexed
Search Console reported 51,000 URLs as "Indexed, though blocked by robots.txt," and Google says that's not necessarily a problem.
TopicsTechnical SEO
Related
The AI Convergence Problem
Search Engine Journal
Shared training data plus shared incentives equals everyone sounding like everyone else. Is your brand the one being averaged out?
Daily Search Forum Recap: June 9, 2026
Search Engine Roundtable
Here is a recap of what happened in the search forums today, through the eyes of the Search Engine Roundtable and other search forums on the web. Apple announced the new Apple Intelligence and Siri AI...
Google May 2026 Core Update Hits Hard Saturday - May 30th
Search Engine Roundtable
The Google May 2026 core update seemed to have hit some sites in a very big way yesterday, Saturday, May 30, 2026. As a reminder, the May 2026 core update was announced on May 21, 2026, and then that following weekend, we saw some significant ranking volatility, and then again, t
Google Is Adding Business Profile Tools To The Gemini App
Search Engine Journal
Google is adding a direct Google Business Profile connection and Business notebooks to the Gemini app, rolling out this month.
Written For Readers Who Don’t Read
Search Engine Journal
Most of the web's reading is now done by bots. For anyone publishing online, the rules on access, quality, and cheating no longer hold.
![Why LLMs Cite Reddit Instead Of Your Brand: A Practical AI Visibility Audit [Webinar]](https://cdn.searchenginejournal.com/wp-content/uploads/2026/05/featured-4-958.png)
Why LLMs Cite Reddit Instead Of Your Brand: A Practical AI Visibility Audit [Webinar]
Search Engine Journal
Understand the role Reddit plays in AI answers and how brands can capitalize on community signals to enhance visibility.