GPTBot

AI training crawler operated by OpenAI

54.9%

Not from OpenAI

112

Verified source IPs

41

Impostor source IPs

Yes

Published IP ranges

Measured across WebDecoy's production detection corpus, 2025-11-14 to 2026-08-19.

What GPTBot is

Collects public web content used to train OpenAI models.

It identifies itself with a User-Agent containing GPTBot/1.2; +https://openai.com/gptbot, and honours the robots.txt token GPTBot. Operator documentation: OpenAI.

How much of this traffic is actually GPTBot

OpenAI publishes the IP ranges its crawler operates from, so this has a real answer rather than an opinion. We took every request in our corpus declaring this User-Agent and checked its source address against that published list.

ResultSource addressesShare of traffic
Verified. Inside a published OpenAI range11245.1%
Failed. Outside every published range4154.9%

54.9% of traffic claiming to be GPTBot could not be traced back to OpenAI. It is something else wearing the name. Most sites wave this User-Agent straight through without checking, which is exactly why the name is worth wearing.

Verifying it yourself

Fetch the published ranges and test the source address against them:

# 1. fetch the published ranges
curl -s -o ranges.json \
  https://openai.com/gptbot.json

# 2. check an address against them
python3 - 203.0.113.42 <<'PY'
import json, sys, ipaddress
ip = ipaddress.ip_address(sys.argv[1])
data = json.load(open('ranges.json'))
hit = False
for p in data['prefixes']:
    net = p.get('ipv4Prefix') or p['ipv6Prefix']
    if ip in ipaddress.ip_network(net):
        hit = True
print('verified' if hit else 'NOT VERIFIED')
PY

Swap in the address you are checking. Never trust the User-Agent on its own. It is a header the client picks for itself, and as the numbers above show, plenty of clients pick a lie.

Blocking it

If you want the real crawler to stay away, robots.txt is the polite way to ask:

User-agent: GPTBot
Disallow: /

That only works on a crawler willing to obey it, which rules out every request that was already lying about who it is. Stopping those takes enforcement at the network or application layer, keyed on something the client cannot simply retype. That is the problem WebDecoy exists to solve.

For the wider picture, see how to detect AI scrapers across GPTBot, ClaudeBot and Perplexity, or the mechanics of cryptographically verifying a trusted bot rather than taking its word for it.

Frequently Asked Questions

Is GPTBot a real crawler or is it being faked? +

Both. GPTBot is a genuine crawler operated by OpenAI, but 54.9% of the traffic we observed carrying its User-Agent did not come from an IP address in OpenAI's published range list. Only 112 of the source addresses using this name verified against that list; 41 did not. A User-Agent string is a claim, not proof.

How do I verify GPTBot myself? +

OpenAI publishes its crawler IP ranges at https://openai.com/gptbot.json. Take the source IP of the request, check whether it falls inside one of the published prefixes, and reject it if it does not. Never match on the User-Agent string alone, because that is the part attackers control.

Should I block GPTBot? +

That depends on what it does for you. Collects public web content used to train OpenAI models. Blocking the verified traffic is a business decision, and only you can make it. Blocking the unverified traffic is not much of a decision at all, because requests that only claim to be GPTBot bring none of the upside.

Does GPTBot respect robots.txt? +

OpenAI says it does, and you can disallow it with the token GPTBot. We are not claiming to have checked. Proving compliance needs a disallowed path to watch, and the sites in this corpus disallow nothing, so no crawler had anything to violate. Treat published compliance as a vendor statement until someone measures it. What the data here answers is the question that comes first: whether a request came from OpenAI at all.

Methodology

Every request in WebDecoy's production detection corpus whose User-Agent declares one of these crawlers, checked IP-by-IP against the operator's own published IP range list. A request is counted as verified when its source IP falls inside a published prefix, and forged when it does not. Operators that publish no machine-readable range list cannot be checked this way and are reported without a forgery figure. Corpus: 81,034 detections from 17,187 distinct addresses, 2025-11-14 to 2026-08-19. Published ranges retrieved 2026-08-19. Operators revise their ranges over time, so a request logged months ago is checked against today's list; we re-ran the comparison over a recent 30-day window and the rates held.

Other crawlers

← All crawlers

See which GPTBot requests are real

WebDecoy verifies declared crawlers against published ranges and identifies the ones that fail, as persistent actors rather than disposable IPs.

Start Free Trial