Skip to content

Back to the catalog

openai.comarticle

Making it easier for websites to express AI training preferences

OpenAI announces it will honor a common robots.txt signal, defined in the IETF draft draft-kelley-aipref-training-preference-signal, that lets site operators state whether content may be used for foundational generative AI training. The signal lets operators set one preference for several participating providers. OpenAI plans to implement it in the coming months, and its existing crawler controls stay in force.

TL;DR OpenAI commits to honoring a shared IETF-backed robots.txt signal for AI training preferences, alongside its existing crawler rules.
Open original ↗

Organisations named here

  • OpenAI
  • IETF
  • IETF AI Preferences working group
  • Internet Engineering Task Force (IETF)

Technologies named here

  • OpenAI crawler controls
  • robots.txt

Standards and laws named here

  • draft-ietf-aipref-vocab
  • draft-kelley-aipref-training-preference-signal
  • robots.txt

Concepts named here

  • AI training preference signal
  • robots.txt

Related in ai-policy