OpenAI's search crawler. It is used to surface websites in the search features of ChatGPT.
Source: developers.openai.comOperator
OpenAI
Job
Search index
Builds the index an AI answer searches. Block it and you drop out of that product's results.
robots.txt
Follows robots.txt
OpenAI says sites that opt out of OAI-SearchBot are not shown in ChatGPT search answers, though they can still appear as navigational links.
Source: developers.openai.comMozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
Add one of these groups to the robots.txt at the root of your domain.
A crawler obeys only the most specific group that names it (RFC 9309), so a named group does not inherit the rules under User-agent: *. Repeat any paths you keep private, as the example does with /admin/.
User-agent: OAI-SearchBot Allow: / Disallow: /admin/
User-agent: OAI-SearchBot Disallow: /
Read from the robots.txt we serve. We want AI engines to read and cite this site, so every AI crawler is allowed everywhere except the app, the API and the sign-in pages (/dashboard/, /api/, /login, /signup, /sign-in, /sign-up). The file also declares Content-Signal: ai-train=yes, search=yes, ai-input=yes.
User-agent: OAI-SearchBot Allow: / Allow: /api/markdown/ Disallow: /dashboard/ Disallow: /api/ Disallow: /login Disallow: /signup Disallow: /sign-in Disallow: /sign-up
See the whole file. The same rules repeat in every group, because a crawler that matches a named group ignores the wildcard one.
OpenAI says each of its robots.txt settings is independent: you can allow OAI-SearchBot to appear in search while disallowing GPTBot. A robots.txt change can take about 24 hours to take effect.
Sources, checked 2026-09-24