Amazonbot
AI training crawler operated by Amazon
Amazon
Operator
Reverse DNS
Verification
Amazonbot
robots.txt token
Yes
Honours robots.txt
Measured across detections recorded by WebDecoy, mostly on WebDecoy's own properties, 2025-11-14 to 2026-08-19. A small sample, not the whole web.
What Amazonbot is
Crawls pages to support Alexa and Amazon search features.
It identifies itself with a User-Agent containing Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot. Its robots.txt token is Amazonbot , and it is listed as honouring robots.txt. Operator documentation: Amazon .
Verifying it yourself
Use a forward-confirmed reverse DNS lookup. The hostname must end in .crawl.amazonbot.amazon , and must resolve back to the address you started with:
# 1. reverse lookup: hostname should end in .crawl.amazonbot.amazon
dig +short -x 203.0.113.42
# 2. forward lookup: must return the address you started with
dig +short that-hostname.crawl.amazonbot.amazonSwap in the address you are checking. Never trust the User-Agent on its own: it is a header the client picks for itself.
Blocking it
If you want the real crawler to stay away, robots.txt is the polite way to ask:
User-agent: Amazonbot
Disallow: /robots.txt only works on a client willing to obey it, which rules out every request that was already lying about who it is. Stopping those takes enforcement at the network or application layer, keyed on something the client cannot simply retype. That is the problem WebDecoy exists to solve.
Frequently Asked Questions
What is Amazonbot? +
Amazonbot is an ai training crawler operated by Amazon. Crawls pages to support Alexa and Amazon search features.
How do I verify Amazonbot? +
Use a forward-confirmed reverse DNS lookup. Resolve the source address to a hostname, check that the hostname ends in .crawl.amazonbot.amazon, then resolve that hostname and confirm it returns the address you started with. A reverse record on its own proves nothing.
How do I block Amazonbot? +
Add a robots.txt rule for the token Amazonbot. That stops the real crawler, but not clients that only borrow its name, so pair it with verification and enforcement at your edge.
Methodology
Every request recorded by WebDecoy's sensors, mostly on WebDecoy's own properties, whose User-Agent declares one of these crawlers, checked IP-by-IP against the operator's own published IP range list. A request is counted as verified when its source IP falls inside a published prefix, and forged when it does not. Operators that publish no machine-readable range list cannot be checked this way and are reported without a forgery figure. Corpus: 81,034 detections from 17,187 distinct addresses, 2025-11-14 to 2026-08-19. Published ranges retrieved 2026-08-19. Operators revise their ranges over time, so a request logged months ago is checked against today's list; we re-ran the comparison over a recent 30-day window and the rates held.
This crawler dataset is licensed under CC BY 4.0; reuse is permitted with attribution to WebDecoy.
Related crawlers
- AmazonBuyForMe Amazon
- AbliterantBot Abliterant
- AI2Bot Allen Institute for AI
- Anthropic Anthropic
- Applebot-Extended Apple
- ByteSpider ByteDance
- CCBot Common Crawl
- ChatGLM-Spider Zhipu AI
- Claude-Web Anthropic
- ClaudeBot Anthropic
- Cohere Cohere
- DeepSeek DeepSeek
See which Amazonbot requests are real
WebDecoy verifies declared crawlers against what their operators publish and identifies the ones that fail, as persistent actors rather than disposable IPs.