One robots.txt rule lets websites opt out of AI training at four labs

Anthropic, Google, Microsoft and OpenAI have agreed to honor a shared signal that lets a website opt out of having its content used to train AI models, separately from search indexing, per Thurrott.com on Oct. 30; Anthropic has already implemented it, while Google, Microsoft and OpenAI will do so in 2027. The signal was created by a working group at the Internet Engineering Task Force (IETF), the body that sets internet standards, and will be proposed as an international standard once complete, Thurrott reported. Microsoft’s Bing blog says a site opts out by adding a robots.txt group addressed to __usage_ai-training with a Disallow rule, that a site publishing no training preference is treated as allowing training, that Bing will begin honoring the preferences on Feb. 1, 2027, and that they do not change a site’s eligibility for Bing search results.