DataPacts

Applebot-Extended: Opting Out of Apple Intelligence Training

Checked

At a glance

Applebot-Extended is a robots.txt token through which a site tells Apple not to use its content, collected by the Applebot crawler, to train Apple's generative foundation models, and Apple states that Applebot-Extended itself does not crawl webpages.

Independent reference. Every buyer is covered on the same terms, every figure carries its source and check date, and no buyer pays to be listed or placed higher.

Request an independent introduction

Tell us what data your company holds. We reply with the buyers that fit and what their terms leave out. Your details reach a buyer only after you agree to a named introduction.

IndependentYour details stay with us until you agreeNot selling is a valid outcome

The token is one of the crawler controls within AI training data licensing and acquisition, and it governs training only, not Apple's search use of the same pages.

Every statement about Apple's controls here is Apple's, from its support page "About Applebot" published 2026-09-04 and re-read on 2026-11-17, and this page recommends no rule. Apple names three uses of Applebot's data and gives each its own control, and a disallow of the token leaves Applebot crawling and Search inclusion running.

Every figure and statement carries its stating party and check date, and how we source, date and check every figure sets out the method.

What Applebot-Extended is: Apple's training control over Applebot's data, not a crawler

Apple describes Applebot-Extended as a secondary user agent that gives web publishers additional controls over how their content can be used by Apple, and states that it "does not crawl webpages" and "is only used to determine how to use the data crawled by the Applebot user agent" (page published 2026-09-04, checked 2026-11-17).

The two names play different roles.

  • Applebot: the crawler that fetches pages, whose data powers the search technology in Spotlight, Siri and Safari and, Apple says, "may also be used to help train" Apple foundation models.
  • Applebot-Extended: the secondary user agent that only decides how that crawled data is used for training.

The scope Apple states is wider than the H1 product name. A disallow opts content out of training Apple's "general purpose foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools".

Because the token never fetches anything, no server log shows an Applebot-Extended visit; Apple reads the token when Applebot's data is used.

Two errors recur in search results, a description of the token as a training crawler and a claim that it does not exist, and Apple's page supports neither. Applebot-Extended is one entry in the list of AI crawlers and how to allow, block or charge them.

Three uses of Applebot-crawled content and the control Apple gives for each

Apple names three uses of Applebot-crawled content, search, foundation model training and context for AI answers, and gives each its own control. The table below compares them in Apple's order, as of 2026-11-17, and ranks none.

Use (Apple's words) Products Apple names Control What the control does (Apple) Source
search technology Spotlight, Siri, Safari robots.txt rules for Applebot; noindex meta tag Applebot does not crawl the paths; noindex keeps the page out of Spotlight and Siri Suggestions support.apple.com/en-us/119829
"help train Apple foundation models" Apple Intelligence, Services, Developer Tools robots.txt group for Applebot-Extended opts content out of training Apple's general purpose foundation models support.apple.com/en-us/119829
"additional context and up-to-date content" for AI output, for example broad world knowledge answers in Siri and Search Siri, Search nosnippet (meta tag or X-Robots-Tag: applebot: nosnippet); isAccessibleForFree: false for paywalled pages content not used as that context; paywalled pages stay eligible for search results support.apple.com/en-us/119829

The controls stack and do not replace one another. Apple says that even with Applebot-Extended disallowed and nosnippet set, a site's instructions may still allow Applebot to crawl, and the content remains discoverable through Spotlight, Siri and Safari.

The isAccessibleForFree signal applies at the page level only, and Apple adds that "Section-level markup using hasPart is not supported".

Writing the Applebot-Extended rule next to Applebot rules

The rule is a separate User-agent group naming Applebot-Extended; Apple's own example is User-agent: Applebot-Extended followed by Disallow: /private/, which withholds one path from training, and Disallow: / withholds the whole site. The table below shows whole-site and path rules as of 2026-11-17, with Apple's statement of what each does.

Goal Lines What it does (Apple's statement) What keeps running Source
Keep one path out of training User-agent: Applebot-ExtendedDisallow: /private/ (Apple's example) content under the path not used to train foundation models Applebot crawling, search inclusion support.apple.com/en-us/119829
Keep the whole site out of training User-agent: Applebot-ExtendedDisallow: / same, for every path Applebot crawling, search inclusion, ranking unaffected support.apple.com/en-us/119829
Also keep pages out of AI answers the row above plus <meta name="applebot" content="nosnippet"> per page no description or web answer; not used as AI-output context search inclusion with title only support.apple.com/en-us/119829
Leave Apple search too User-agent: ApplebotDisallow: / Applebot does not crawl iTMS for registered Podcasts URLs support.apple.com/en-us/119829
Rules written only for Googlebot no Applebot group Applebot follows the Googlebot instructions Applebot-Extended handling not stated support.apple.com/en-us/119829

Apple states that Applebot "respects standard robots.txt directives in general search crawls that are targeted at Applebot". It also states that when robots instructions do not mention Applebot but mention Googlebot, Applebot follows the Googlebot instructions.

Apple does not say whether that fallback, a User-agent: * group or a Google-Extended rule is read as an Applebot-Extended rule, so no advice on those cases is given here. Wildcard groups, fallbacks and other operators' tokens are set out under robots.txt for AI crawlers.

Rendering is a separate matter. Apple writes that if JavaScript, CSS and other resources are blocked in robots.txt, Applebot may not render the content properly, so a training opt-out belongs in the Applebot-Extended group and not in resource blocks for Applebot.

What an Applebot-Extended disallow changes and what keeps running

A disallow tells Apple not to use the content to train its foundation models; Apple states that the pages "can still be included in search results" and that "Site rules for Applebot-Extended are not considered in ranking for Search" (page published 2026-09-04, checked 2026-11-17).

Five things keep running after a disallow, by Apple's statements.

  • Applebot crawling, which the token does not control.
  • Inclusion in search results.
  • Search ranking, which ignores Applebot-Extended rules.
  • Use as AI-answer context, unless nosnippet or isAccessibleForFree false is set on the page.
  • The iTMS agent for Apple Podcasts, which "does not follow robots.txt, as it is not a general search crawler" and "only crawls URLs associated with registered content on Apple Podcasts".

Apple's page says nothing about content used before a disallow was added or about how fast a change applies. Robots.txt is a convention defined in RFC 9309 and is not legally binding by itself.

Apple gives its own argument for the opposite choice, "Allowing Applebot-Extended will help improve the capabilities and quality of Apple's generative AI models over time", which is Apple's claim and is quoted without comment.

Checking that a visitor really is Applebot: reverse DNS, the IP file and the user-agent format

Apple's own tests for Applebot traffic are a reverse DNS lookup in the *.applebot.apple.com domain or a match of the IP address against the CIDR prefixes in Apple's JSON file; the user-agent string alone proves nothing, because any client can send it.

Apple writes that traffic is "generally identified by using reverse DNS in the *.applebot.apple.com domain".

A check takes three steps.

  1. Run a reverse lookup on the source IP and confirm the host sits in *.applebot.apple.com, then confirm by a forward lookup that the host points back to the same address; Apple's example is 17-58-101-179.applebot.apple.com with address 17.58.101.179.
  2. Match the IP against the CIDR prefixes in search.developer.apple.com/applebot.json.
  3. Read the user-agent for "Applebot" and the link +http://www.apple.com/go/applebot, which is only a hint.

The file carries a creation time of 2026-09-15 and 24 IPv4 prefixes at the 2026-11-17 reading, with no IPv6 prefixes.

Apple gives the general user-agent format as ending in "(Applebot/Applebot_version; +http://www.apple.com/go/applebot)" and notes that "Occasionally, Applebot will update the browser version that it advertises". No Applebot-Extended string exists, because the token does not crawl. Apple asks for questions or concerns at applebot@apple.com.

In the EU, DSM Directive Article 4(3) lets rightsholders reserve text and data mining rights by machine-readable means, and the AI Act requires providers of general-purpose AI models to keep a policy that identifies and complies with such reservations; no court ruling on record here decides whether an Applebot-Extended disallow is such a reservation.

Article 4(3) of Directive (EU) 2019/790, in force since 2019-06-06, lets rightsholders reserve their rights "in an appropriate manner, such as machine-readable means in the case of content made publicly available online", and the EU text and data mining exception and its opt-out are set out on their own page.

Article 53(1)(c) of Regulation (EU) 2024/1689 sets that policy duty, and the obligations for general-purpose AI model providers apply from 2025-08-02. The voluntary General-Purpose AI Code of Practice, published 2025-07-10, says in Measure 1.3 of its copyright chapter that signatories' crawlers follow robots.txt as specified in RFC 9309.

Apple's own EU status is covered on its buyer profile and is not repeated here.

Outside such a reservation, robots.txt is a technical convention. RFC 9309 defines the protocol and does not make it binding, and this page is information, not a statement of any court's view.

Disallowing Applebot-Extended: benefits, risks, downsides and alternatives

Disallowing Applebot-Extended costs nothing and, by Apple's statement, leaves Search inclusion and ranking untouched, but it earns nothing, covers training only and relies on Apple honoring its own page.

Licensing content to Apple is a separate question, and Apple publishes no program, terms or payouts for data holders as of 2026-11-09, as the profile of Apple as a data buyer records.

Benefits:

  • The rule is a two-line edit to robots.txt.
  • Search inclusion and ranking stay as they are, by Apple's statement.
  • In the EU the disallow is a machine-readable robots.txt signal of the kind Article 4(3) and the Code of Practice refer to.

Risks:

  • Spoofed user agents need a reverse-DNS or IP check before any log line is trusted.
  • A rule that sits only in a User-agent: * group or under Googlebot may not reach Applebot-Extended, and Apple is silent on it.
  • Blocking Applebot itself to stop training also removes search visibility in Spotlight, Siri and Safari.

Downsides:

  • No payment: the support page mentions none.
  • Training only: use as AI-answer context needs nosnippet or isAccessibleForFree.
  • Earlier use of the content is not addressed by Apple.

Alternatives:

  1. Allow everything, which is the course Apple's page gives its reason for.
  2. Disallow Applebot-Extended only for chosen paths, as in Apple's example.
  3. Add nosnippet on pages as well.
  4. License the content directly, or decide on whether to sell or license data to AI companies at all.

Other crawler controls, Apple's acquisition record and the law around Applebot's data

Applebot-Extended is one training control among many, and the pages below cover the others, the other ways to signal an opt-out and the law around crawling.

This page is information, not legal advice.

Frequently asked questions

Does blocking Applebot-Extended remove a site from Siri, Spotlight or Safari?

No, by Apple's statement: Applebot-Extended does not crawl, pages that disallow it can still be included in search results, and its rules are not considered in Search ranking. The page was published 2026-09-04 and checked 2026-11-17.

Is there an Applebot-Extended user-agent string in server logs?

No: Apple states Applebot-Extended does not crawl webpages, so it sends no requests. The visits come from Applebot, whose user-agent contains "Applebot" and Apple's link apple.com/go/applebot (checked 2026-11-17).

Can Applebot be slowed with Crawl-delay?

No. Apple states that "Applebot does not follow crawl-delay" and that its crawl rate adjusts automatically when a site slows down or returns errors. The page was published 2026-09-04 and checked 2026-11-17.

Does an Applebot-Extended disallow remove content Apple already used?

Apple does not say. Its page describes how Applebot-Extended decides the use of crawled data and says nothing about content used for training before a disallow was added (checked 2026-11-17).