Security & Compliance
November 2, 2026The shared AI training opt-out: your Azure robots.txt plan
On 30 October 2026 Google, Microsoft and OpenAI said they will honour one robots.txt group, User-Agent: __usage_ai-training, as a site-wide AI training preference, defined in an IETF draft written at Anthropic. The token covers training only, ignores wildcard groups, and leaves grounding to vendor tags, so Azure and Next.js teams need a posture decision, an explicit Google-Extended choice and Bing page tags before Microsoft retires NoArchive on 1 March 2027.
robots.txt
AI training opt-out
IETF AIPREF
Google-Extended
Bing
GPTBot
ClaudeBot
EU AI Act
Code of Practice
Next.js
Azure Front Door
Upphovsrättslagen
By Falak Mahmood