Signal

Google and OpenAI back a shared signal for website AI-training preferences

The companies plan to recognize a common robots.txt preference alongside their existing crawler controls. The proposal could simplify publisher choices, but implementation is pending and the referenced draft is not an adopted IETF standard.

2 min read
Google Search Central article describing a shared robots.txt AI training control
Google's plan to support a shared AI training control in robots.txt · Credit: Google View source

Google and OpenAI announced plans on October 30 to support a shared way for websites to express preferences about foundational generative-AI training in robots.txt. Instead of maintaining only separate instructions for individual providers, website operators could communicate a broad preference to participating companies through a common policy token.

The companies describe support that is still to come. OpenAI aims to implement the signal in the coming months, while Google says it aims to support the control in 2027. Neither announcement establishes that the shared mechanism is already operating across the web.

Google says it plans to interpret a shared training opt-out as a Google-Extended disallow when the site has not explicitly specified Google-Extended. Existing company-specific controls would continue alongside the new signal, allowing publishers to combine a broad preference with more specific choices.

The referenced training-preference document is an individual Internet-Draft. Its Datatracker page states that it has no formal standing as an endorsed IETF standard. The announcements therefore concern voluntary provider commitments and continuing technical work, rather than a completed global standard or a new legal right.

For publishers, the potential benefit is lower operational complexity when communicating the same preference to multiple providers. The limits are equally practical: support depends on implementation and participation, and a machine-readable preference does not by itself demonstrate enforcement. Operators should distinguish today's existing controls from the promised shared mechanism.

Sources