Welcome to the TechSEO Vitals Newsletter, brought to you by Martin Stepanek, an experienced technical SEO and web performance consultant with over 10 years of expertise in web development, website architecture design, and performance optimization.
Share
π£ Cloudflare Is About to Block Googlebot for Sites That Blocked AI Training - TechSEO Vitals #41
Published 29 days agoΒ β’Β 6 min read
π Hey Reader, Welcome back to another episode of the TechSEO Vitals Newsletter! Last year, blocking AI bots felt like the obvious move. Cloudflare made it one click. A lot of people pressed it. In a few weeks, that click starts blocking Googlebot. Not a warning. Not a preference in a text file. Cloudflare blocks the request before it can read your robots.txt file. β‘οΈ TL;DR Cloudflare now judges crawlers on all their behaviors, not one β and the most restrictive rule wins Googlebot crawls for...
Subscribe to keep reading
This post is free to read but only available to subscribers.