Amazon runs two other user agents, Amzn-SearchBot for search experiences such as Alexa and Amzn-User for user requests, and states that neither crawls content for generative AI model training, a split that matters for AI training data licensing and acquisition.
Every statement here comes from Amazon's crawler page, updated 2026-10-26 and re-read on 2026-11-16, and the page recommends no rule. A site can opt out per bot in robots.txt or per page with the noarchive meta tag, which Amazon defines as "do not use the page for model training".
Every figure and statement carries its stating party and check date, and how we source, date and check every figure sets out the method.
What Amazonbot collects, and how Amzn-SearchBot and Amzn-User differ
Amazon states that Amazonbot is used to improve its products and services and may be used to train Amazon AI models, and it describes Amzn-SearchBot and Amzn-User as separate agents that do not crawl content for generative AI model training. Its words are that Amazonbot "helps us provide more accurate information to customers" (checked 2026-11-16).
The list below gives each agent's stated purpose, and the section hub on AI crawlers and how to allow, block or charge them places Amazon's agents among those of other operators.
- Amazonbot: improves Amazon's products and services and "may be used to train Amazon AI models".
- Amzn-SearchBot: improves search experiences, and permitted content is "eligible to appear in search experiences such as Alexa"; Amazon says it does not crawl for generative AI model training.
- Amzn-User: "supports user actions, such as responding to Alexa queries that require up-to-date information"; Amazon says it does not crawl for generative AI model training either.
Amazon does not say which Amazon AI models Amazonbot feeds, so none is named here.
Amazon's three user agents as Amazon documents them
Amazon documents exactly three user agents on its crawler page, each with an example string and its own published IP address list. The table below compares them in the order Amazon lists them, as of 2026-11-16, and Amazon's page title is "Amazon Web Search crawlers".
| User agent | Purpose (Amazon's words) | Example user-agent string | IP list | robots.txt (Amazon's statement) | Training (Amazon's statement) | Checked | Source |
|---|---|---|---|---|---|---|---|
| Amazonbot | "used to improve our products and services" | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36 |
developer.amazon.com/amazonbot/ip-addresses/ | allow and disallow honored; no crawl-delay; cache up to 30 days | "may be used to train Amazon AI models" | 2026-10-26, re-checked 2026-11-16 | developer.amazon.com/amazonbot |
| Amzn-SearchBot | "used to improve search experiences in Amazon products and services"; content "eligible to appear in search experiences such as Alexa" | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-SearchBot/0.1) Chrome/W.X.Y.Z Safari/537.36 |
developer.amazon.com/amazonbot/searchbot-ip-addresses/ | follows rules for other search bots when not named | "does not crawl content for generative AI model training" | 2026-11-16 | developer.amazon.com/amazonbot |
| Amzn-User | "supports user actions, such as responding to Alexa queries that require up-to-date information" | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-User/0.1) Chrome/W.X.Y.Z Safari/537.36 |
developer.amazon.com/amazonbot/live-ip-addresses/ | "may not follow all robots.txt directives" | "does not crawl content for generative AI model training" | 2026-10-26, re-checked 2026-11-16 | developer.amazon.com/amazonbot |
Reading key:
- All three strings are labeled "Example User Agent String" by Amazon, and "/0.1" is Amazon's own version.
- "W.X.Y.Z" is a placeholder, so a real log line carries an actual Chrome version where the example shows it.
- The page names no other Amazon agent, and this is not proof that none exists.
robots.txt lines for Amazonbot, Amzn-SearchBot and Amzn-User
Each Amazon user agent is its own robots.txt setting, so a site can disallow Amazonbot while leaving Amzn-SearchBot allowed for search experiences such as Alexa. Amazon writes that "each user agent setting is independent of the others", and Amzn-SearchBot follows the directives given to other search bots when robots.txt does not mention it (checked 2026-11-16).
The table below compares five whole-site goals as of 2026-11-16, each token in its own User-agent group, with what Amazon's statements say the lines do and what keeps running. Amazon names no robots.txt group for training alone, so an Amazonbot disallow stops all Amazonbot crawling, not only its training use.
| Goal | Lines | What it does (Amazon's statement) | What keeps running | Source |
|---|---|---|---|---|
| Stop Amazonbot, stay eligible for Alexa search | User-agent: Amazonbot then Disallow: / |
Amazonbot stops crawling the site, including the crawling Amazon says may train its AI models | Amzn-SearchBot crawling; Amzn-User fetches | developer.amazon.com/amazonbot |
| Same, stated explicitly | User-agent: Amzn-SearchBot then Allow: /, then User-agent: Amazonbot then Disallow: / |
independent settings per user agent | same as above | developer.amazon.com/amazonbot |
| Allow crawling, opt pages out of training | no Amazon group; noarchive robots meta tag on chosen pages |
pages stay crawlable; tagged pages are "do not use the page for model training" | all three agents crawl or fetch | developer.amazon.com/amazonbot |
| Leave Alexa search too | the first row plus User-agent: Amzn-SearchBot then Disallow: / |
Amzn-SearchBot stops crawling; without its own group it follows other search bots' rules | Amzn-User fetches | developer.amazon.com/amazonbot |
| Ask user-initiated fetches to stop | User-agent: Amzn-User then Disallow: / |
a request Amazon says it "may not follow" in full | depends on Amazon | developer.amazon.com/amazonbot |
User-agent: Amazonbot
Disallow: /
These are whole-site lines only. Syntax, precedence and other operators' tokens are covered on the page about robots.txt for AI crawlers.
How Amazon's crawlers read robots.txt: hosts, a 30-day cache and unreachable files
Amazon states that its crawlers respect the Robots Exclusion Protocol, honoring the user-agent line and the allow and disallow directives, read robots.txt per host, and may act on a cached copy from the last 30 days (checked 2026-11-16). Amazon applies this to all three listed user agents, and RFC 9309 (2022) is the protocol's specification.
Amazon's page makes two timing statements. The first says each setting "may take ~24 hours for our systems to reflect changes", and the second says the crawlers fetch host-level files "or use a cached copy from the last 30 days".
RFC 9309 says a cached copy should not be used for more than 24 hours unless the file is unreachable. Amazon does not reconcile its two statements on the page, and none is drawn here.
For an unreachable file, Amazon writes that the crawlers "will behave as if it does not exist". RFC 9309 says that on a server error (5xx) a crawler must assume complete disallow, and Amazon's wording does not separate error classes, so no conclusion follows.
- Host-level file: example.com/robots.txt covers example.com only, and site.example.com needs its own file.
- Cache: a copy from the last 30 days may be used.
- Unreachable file: treated as absent, in Amazon's words.
- Crawl-delay: not supported, by Amazon's statement.
noarchive as Amazon's page-level training opt-out
Amazon states that its crawlers respect the page-level robots meta tag noarchive and defines it as "do not use the page for model training", which lets a site keep a page crawlable while opting that page out of Amazon model training.
The sentence on Amazon's page reads that "when these user agents access web pages they respect the link-level rel=nofollow directive, and page level robots meta tags of noarchive (do not use the page for model training), noindex (do not index the page) and none (do not index the page)" (checked 2026-11-16).
The table below compares the four directives Amazon names, as of 2026-11-16, and "these user agents" refers to all three agents.
| Directive | Where it sits | Amazon's meaning | Applies to | Checked | Source |
|---|---|---|---|---|---|
| noarchive | page-level robots meta tag | "do not use the page for model training" | all three user agents | 2026-10-26, re-checked 2026-11-16 | developer.amazon.com/amazonbot |
| noindex | page-level robots meta tag | "do not index the page" | all three user agents | 2026-11-16 | developer.amazon.com/amazonbot |
| none | page-level robots meta tag | "do not index the page" | all three user agents | 2026-11-16 | developer.amazon.com/amazonbot |
| rel=nofollow | link level | respected, no meaning spelled out | all three user agents | 2026-11-16 | developer.amazon.com/amazonbot |
The generic form is <meta name="robots" content="noarchive">. Amazon's page shows no Amazon-specific meta name or X-Robots-Tag header, gives no timing for a tag to take effect and says nothing about pages collected before a tag was added, so none of these is described here.
Other operators define page-level tags differently or not at all, and the page on machine-readable TDM reservations and meta tags compares how other developers read them.
Checking that a visitor really is Amazonbot: example strings and published IP addresses
A request is from Amazonbot only if its user agent carries the Amazonbot token and its source IP address appears on the list Amazon publishes at developer.amazon.com/amazonbot/ip-addresses/, and Amzn-SearchBot and Amzn-User have their own lists. All three list pages answered on 2026-11-16, at the addresses in the user-agent table above.
A string alone proves nothing, because any client can send it, and Amazon publishes the lists so that site owners can compare addresses themselves. A check takes three steps:
- Match the token in the user-agent string (Amazonbot, Amzn-SearchBot or Amzn-User) against the example Amazon prints, allowing for a real Chrome version where the example shows W.X.Y.Z.
- Match the source IP address against the list for that agent, since the three lists are separate.
- When in doubt, write to amazonbot@amazon.com and include the relevant domain names, as Amazon asks of content owners and publishers.
Amazon publishes no reverse-DNS method on the page, so none is described. General spoofing checks across operators stay with the section hub.
What an Amazonbot disallow or a noarchive tag does and does not change
An Amazonbot disallow stops future Amazonbot crawling of the site by Amazon's statement, and a noarchive tag asks that a page not be used for model training, while neither stops Amzn-SearchBot crawling for search or Amzn-User fetches on a user's request.
That is the meaning Amazon gives each signal (checked 2026-11-16), and robots.txt is not legally binding by itself under RFC 9309.
An Amazonbot disallow leaves four things running:
- Amzn-SearchBot crawling for search experiences, a separate line that follows other search bots' rules when unnamed.
- Amzn-User fetches on a user's request, which Amazon says "may not follow all robots.txt directives".
- A cached robots.txt, usable for up to 30 days, and the roughly 24 hours Amazon gives for settings to take effect.
- Content collected before the rule, since no Amazon statement about deleting it is recorded here.
A noarchive tag works at page level and leaves indexing untouched unless noindex or none is added, the two tags Amazon reads as "do not index the page". Amazon states that Amzn-SearchBot and Amzn-User do not crawl for generative AI model training, so the tag's stated purpose concerns Amazonbot's use of a page.
Licensing agreements a site has signed with anyone are untouched by either signal.
What Amazon says a site may get for allowing Amazonbot
Amazon's crawler page states that a site that allows Amazonbot in robots.txt "may be eligible for benefits with Amazon Content Partners", a program it describes with an affiliate commission boost, hosting credits and AI traffic management tools.
The wording is Amazon's claim, checked 2026-11-16: "+1% affiliate commission boost on eligible sales, free hosting credits, and AI traffic management tools" for "independent content creators".
The program's own terms could not be read on 2026-11-16, because contentpartners.amazon.com loads only with JavaScript, so nothing beyond the crawler-page sentence is stated. It is a benefit Amazon describes, not a license fee or a payment for training data, and no recommendation follows.
Amazon's EU commitments behind an Amazonbot disallow
In the EU, an Amazonbot disallow can act as the machine-readable reservation of DSM Directive Article 4(3), and Amazon is a listed signatory of the GPAI Code of Practice, which commits signatories to crawlers that follow robots.txt as specified in RFC 9309.
Article 4(3) of Directive (EU) 2019/790 (in force 2019-06-06) allows the text and data mining exception only where use has not been reserved "in an appropriate manner, such as machine-readable means in the case of content made publicly available online", as set out on the page about the EU text and data mining exception and its opt-out.
Amazon is among the 22 signatories on the European Commission's Code page, last updated 2026-10-07 (checked 2026-11-09). The Code is voluntary and was published on 2025-07-10, and its Copyright chapter, Measure 1.3, says signatories employ crawlers that read and follow robots.txt as specified in RFC 9309.
Regulation (EU) 2024/1689, Article 53(1)(c), applies from 2025-08-02 and requires general-purpose AI providers to keep a policy that identifies and complies with Article 4(3) reservations.
No ruling on whether a page-level tag such as noarchive counts as an Article 4(3) reservation is recorded here, and no US rule that makes robots.txt binding is recorded here.
Blocking Amazonbot or tagging pages noarchive: benefits, risks, downsides and alternatives
Disallowing Amazonbot or tagging pages noarchive costs nothing and Amazon states what each means, but neither earns anything, and neither reaches Amzn-User fetches or content collected earlier. Licensing content to Amazon is a separate decision from crawler rules, and the profile of Amazon as a data buyer covers that side.
Benefits:
- The rule is free, and a plain text file or a single tag.
- Amazon states in its own words what an Amazonbot disallow and a noarchive tag mean.
- Alexa search eligibility stays if Amzn-SearchBot stays allowed, since each setting is independent.
- A tag allows a page-level choice instead of a whole-site one.
- In the EU, the disallow can act as a machine-readable reservation under DSM Directive Article 4(3).
Risks:
- Amzn-User may still fetch pages, because Amazon says it may not follow all robots.txt directives.
- Spoofed strings pass as Amazon's bots without an IP check.
- A cached file can delay a robots.txt change for up to 30 days by Amazon's statement.
- An unreachable robots.txt is treated by Amazon as absent.
Downsides:
- An Amazonbot disallow also ends Amazonbot's other uses, since Amazon offers no training-only token.
- Disallowing Amazonbot may forgo the Content Partners eligibility Amazon describes.
- A disallow or tag brings no payment, and it is prospective only.
Alternatives:
- Allow everything.
- Disallow Amazonbot and keep Amzn-SearchBot allowed.
- Allow crawling and tag selected pages noarchive.
- Charge or license crawling through intermediaries, license content directly, or decide on whether to sell or license your data to AI companies at all, where not licensing is a valid outcome.
Blocking, tagging, allowing and licensing can be combined, and none is recommended here.
Other AI crawlers, opt-out signals and the law around Amazon's crawling
Amazonbot is one crawler among many, and the pages below cover the others, the other ways to signal an opt-out and the law around crawling.
- AI crawlers and how to allow, block or charge them: every AI crawler and control method in one place.
- GPTBot and OpenAI's other crawlers: OpenAI's GPTBot and its search and user bots.
- Google-Extended: opting out of Gemini training.
- Bytespider: ByteDance's undocumented crawler, by contrast with Amazon's documented one.
- ClaudeBot: Anthropic's ClaudeBot, Claude-User and Claude-SearchBot.
- Applebot-Extended: Apple's training opt-out token.
- Pay-per-crawl and crawler licensing: charging crawlers instead of blocking them.
- The shared AI training opt-out: the draft shared AI training opt-out.
- Opting out of AI training through platform settings: opt-outs for content hosted on platforms, where robots.txt is not the owner's to edit.
- EU AI Act Article 53 copyright policy: the copyright policy GPAI providers such as Amazon must keep under Article 53.
- Is web scraping for AI training legal: copyright, contract and computer-misuse law on scraping.
This page is information, not legal advice.
Frequently asked questions
Does blocking Amazonbot remove a site from Alexa answers?
Not by Amazon's account: Amzn-SearchBot, a separate robots.txt setting, decides whether content is eligible for search experiences such as Alexa. Amazon says each user agent setting is independent, and Amzn-User may also fetch live pages for Alexa queries (checked 2026-11-16).
Does noarchive stop Amazonbot from crawling a page?
No: Amazon defines noarchive as "do not use the page for model training", a rule on use, while crawling is governed by robots.txt allow and disallow lines. Noindex and none are the tags Amazon reads as "do not index the page" (checked 2026-11-16).
What is the Amazonbot user agent string?
Amazon gives Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36 as an example string. W.X.Y.Z stands for a Chrome version (checked 2026-11-16).
Does Amazonbot support crawl-delay?
No: Amazon states that its crawlers do not support the crawl-delay directive. They do honor the user-agent line and allow and disallow directives (checked 2026-10-26 and 2026-11-16).
How long does a robots.txt change take to reach Amazonbot?
Amazon's page gives two statements: settings "may take ~24 hours for our systems to reflect changes", and its crawlers use a cached copy of robots.txt "from the last 30 days". RFC 9309 says a cached copy should not be used for more than 24 hours.