Cloudflare has issued a deadline for AI web crawlers to differentiate between bots used for search purposes and those used for AI training. These crawlers must comply by September or face blocking on pages with advertisements.
The firm, responsible for managing significant internet traffic, is taking a firm stance against AI crawlers that indiscriminately gather data from online sources.
“The deadline aims to ensure that AI data collection is specified and not bundled with search activities,” a spokesperson said.
Cloudflare’s decision underscores the importance of distinguishing between search-related and AI training data to maintain a balanced and regulated internet environment.