DataPacts

Google-Extended: Opting Out of Gemini Training

Checked

At a glance

Google-Extended is the name a site owner writes into robots.txt to tell Google that content Google crawls from the site should not be used to train Gemini models or to ground Gemini answers.

Independent reference. Every buyer is covered on the same terms, every figure carries its source and check date, and no buyer pays to be listed or placed higher.

Request an independent introduction

Tell us what data your company holds. We reply with the buyers that fit and what their terms leave out. Your details reach a buyer only after you agree to a named introduction.

IndependentYour details stay with us until you agreeNot selling is a valid outcome

One Googlebot crawl feeds Google Search and Gemini; a Google-Extended disallow cuts only Gemini training and grounding, leaves Search inclusion and ranking alone, and never reaches licensed, account or past content.

It is one crawler opt-out within AI training data licensing and acquisition, and it works only forward and only for the uses Google ties to the token.

Google states that the token leaves Google Search inclusion and ranking alone, and the stated scope does not name AI Overviews (checked 2026-11-10). Google's crawler documentation and its Search Central post of 2026-10-30 differ in how they date and name a planned shared training token: "in 2027" against "early 2027".

In the EU, Article 4(3) of the DSM Directive accepts machine-readable reservations, though no ruling recorded here names Google-Extended. This page quotes Google's wording with check dates and neither recommends nor discourages blocking.

Every Google statement below carries its check date, and how we source, date and check every figure sets out the method.

What Google-Extended is: a control token, not a crawler

Google-Extended is a standalone product token that Google reads in robots.txt; it has no HTTP request user agent string of its own, so no visitor named Google-Extended ever appears in server logs.

Google's crawler documentation states that the token does not have a separate user agent string and is used in a control capacity (checked 2026-11-10).

Other operators work differently. OpenAI documents GPTBot with an example user agent string and an IP list, while Apple states that Applebot-Extended does not crawl webpages.

Content that Google-Extended governs is collected by Google's existing crawling infrastructure, and the token only decides what Google may do with it afterwards. The hub on AI crawlers and how to allow, block or charge them lists every crawler and control token side by side.

What a Google-Extended disallow covers, in Google's words

Google states that a Google-Extended disallow covers two uses of crawled content: training future generations of Gemini models and grounding, which means providing Google Search index content to a model at prompt time. The table below gives Google's wording per use, and the scope list is Google's own and complete as published on 2026-11-10.

The table compares each use of crawled content with whether Google says a Google-Extended disallow covers it (as of 2026-11-10).

Use of crawled content Covered by a Google-Extended disallow? Google's wording Checked Source
Training future Gemini models (Gemini Apps, Vertex AI API for Gemini) yes "training future generations of Gemini models that power Gemini Apps and the Vertex AI API for Gemini" 2026-11-10 developers.google.com/crawling (common crawlers page)
Grounding (Google Search index content provided to the model at prompt time) yes "grounding (providing content from the Google Search index to the model at prompt time)", "to improve factuality and relevancy" in Gemini Apps and Grounding with Google Search on Vertex AI 2026-11-16 same
Inclusion in Google Search no "does not impact a site's inclusion in Google Search" 2026-11-10 same
Search ranking no "nor is it used as a ranking signal" 2026-11-10 same
Crawling by Googlebot no token used "in a control capacity"; no own user agent 2026-11-10 same
AI Overviews and AI Mode not stated on the crawler page not named in Google's scope for the token 2026-11-10 same

What Google-Extended leaves untouched: Search inclusion, ranking and Search features

Google states that Google-Extended "does not impact a site's inclusion in Google Search nor is it used as a ranking signal" (checked 2026-11-10). This page repeats the sentence as Google wrote it and adds no claim about what blocking does to rankings.

The stated scope names Gemini Apps, the Vertex AI API for Gemini and grounding. AI Overviews and AI Mode do not appear in it, so this page does not attribute either feature to the token, in either direction.

Whether a disallow changes how Search features display a page is not stated in the crawler documentation, and nothing is claimed here beyond that gap.

Googlebot keeps crawling a site that disallows Google-Extended, and the content stays in the Google Search index. Licensed content and account content sit outside the token altogether and are treated in a later section.

Writing the Google-Extended rule next to Googlebot rules

A whole-site opt-out is two lines in robots.txt: User-agent: Google-Extended followed by Disallow: /, placed in its own group. Each token needs its own group, and the syntax, caching and error handling shared by all crawlers are set out in robots.txt rules, limits and examples for AI crawlers.

The table compares four whole-site patterns with what each does and does not do (as of 2026-11-10; source for the Google statements: Google's crawler documentation and Search Central blog).

Pattern Lines What it does (Google's statement) What it does not do Source
Opt out of Gemini training and grounding User-agent: Google-Extended then Disallow: / content Google crawls is not used for the uses in the scope table Googlebot keeps crawling; Search inclusion and ranking unchanged; licensed and account content unaffected developers.google.com/crawling
Leave Google-Extended allowed no Google-Extended group Gemini training and grounding may use crawled content n/a developers.google.com/crawling
Block Googlebot User-agent: Googlebot then Disallow: / stops Google Search crawling not a Gemini-only opt-out; also governs Applebot where Applebot is not named developers.google.com/crawling; support.apple.com
Shared training token (planned) group for __usage_ai-training as the draft specifies; not shown as working syntax Google plans to treat an opt-out as a Google-Extended disallow when Google-Extended is not addressed, "in 2027" or "early 2027" not read by Google yet developers.google.com/search/blog

robots.txt is the Robots Exclusion Protocol (RFC 9309, 2022) and is not legally binding by itself. RFC 9309 says a crawler should not use a cached copy longer than 24 hours unless the file is unreachable, but Google's crawler documentation states no time within which a Google-Extended change takes effect.

Apple states that Applebot follows Googlebot instructions when robots.txt mentions Googlebot but not Applebot, which is why a Googlebot-only disallow reaches further than a Google-Extended rule does (Apple support page published 2026-09-04).

Where Google's own statements on Google-Extended differ, and the 2027 shared token

Google's crawler documentation and its Search Central post of 2026-10-30 agree on what Google-Extended does today and differ only in how precisely they date and name the shared training token Google plans to support. The token is the subject of the shared AI training opt-out, which covers the draft and who plans to read it.

The table sets the two Google texts side by side (as of each column's date; source: Google's crawler documentation and Search Central blog).

Point Crawler documentation (checked 2026-11-10) Search Central blog (2026-10-30)
Support date for the shared training token "early 2027" "We aim to support this new control along with other AI providers in 2027"
Token string pairs the plan with __usage_ai-training does not name the token string

The token comes from an individual Internet-Draft, draft-kelley-aipref-training-preference-signal-00, published 2026-10-30, which expires 2027-05-03 and is not endorsed by the IETF; it is not a standard. Google states a fallback: an opt-out through the shared token without a Google-Extended rule is to be treated as disallowing Google-Extended.

Until Google's support starts, only a rule naming Google-Extended reaches Gemini training. Neither Google text says Google already reads the shared token.

When a Google-Extended disallow counts as an EU reservation

In the EU a rightsholder can reserve online content from the commercial text and data mining exception by machine-readable means, and a Google-Extended disallow is one such signal, although no court decision recorded here has ruled that it qualifies.

Article 4(3) of Directive (EU) 2019/790 (in force since 2019-06-06) lets a reservation be made "in an appropriate manner, such as machine-readable means in the case of content made publicly available online", and EU text and data mining exception and its opt-out sets out the exception itself.

Article 53(1)(c) of Regulation (EU) 2024/1689, which applies from 2025-08-02, requires providers of general-purpose AI models to keep a policy to comply with Union copyright law and to identify and comply with Article 4(3) reservations.

The GPAI Code of Practice, Copyright chapter, published 2025-07-10, is voluntary. Its Measure 1.3 has signatories employ web crawlers that read and follow robots.txt as specified in RFC 9309. The Commission lists Google among 22 signatories (page last updated 2026-10-07, checked 2026-11-09).

In CJEU case C-250/25 a Hungarian court asked how EU copyright law applies to Gemini. The request was lodged on 2025-04-03, and the claimed use period, as the CJEU summarizes it, runs from 2023-06-13 to 2024-05-31. Advocate General Szpunar's opinion of 2026-11-03 is not binding and the judgment is pending (as of 2026-11-09).

It proposes that the two training questions be declared inadmissible and adds that training is, as a rule, permitted under the exception, while chatbot answers reusing press publications may infringe in certain circumstances; the case page for Like Company v. Google tracks its status.

What a Google-Extended block cannot reach

A Google-Extended disallow works forward on content Google crawls from the site; it does not reach content Google licenses from someone else, content held in Google accounts, or anything already used. Three limits follow.

  • Licensed copies: Google's Reddit agreement gives it Reddit Data API access "to display, train on, and otherwise use" Reddit content (announced 2024-02-22), and Google states it does not change Google's use of publicly crawlable content.
  • Account and platform content: Google states that Workspace does not use customer data for training models without the customer's prior permission or instruction (checked 2026-11-09), and YouTube's third-party training setting is off by default, concerns third-party companies on YouTube's list and YouTube states it does not facilitate payments through it, so it is not a Gemini control.
  • Past use: no Google statement on removing content already used is recorded here.

The claimed use period in C-250/25 shows why the date of a block matters, since a disallow written after a crawl does not undo it. Settings on accounts and platforms are a different route, covered in opting out of AI training through platform settings and their limits.

Disallowing Google-Extended: benefits, risks, downsides and alternatives

Disallowing Google-Extended costs nothing and, by Google's statement, leaves Search inclusion and ranking alone, but it also takes the site out of Gemini grounding and earns nothing. The lists below set out both sides without recommending either choice.

Benefits:

  • It covers Gemini training and grounding by Google's own scope statement.
  • It leaves Search inclusion and ranking unchanged, by Google's statement of 2026-11-10.
  • It is a machine-readable signal of the kind Article 4(3) of the DSM Directive accepts in the EU, though no court has ruled on the token.

Risks:

  • robots.txt is not binding by itself.
  • Google's crawler documentation states no time for a change to take effect.
  • A grounding exclusion may remove the site from Gemini Apps answers, as a consequence of the stated grounding scope.

Downsides:

  • It is prospective only and does not undo past use.
  • It does not reach licensed or account content.
  • It covers one operator, so other labs need their own rules.

Alternatives:

  1. Leave Google-Extended allowed.
  2. Use the shared training token once Google supports it in 2027 or early 2027.
  3. License content to Google or other developers. Google states that its News AI pilot has more than 400 participating publications (2026-10-15) and that it pays participating news publications for the delivery and use of enhanced content, with training not stated. The Associated Press delivers a real-time feed for the Gemini app (announced 2025-01-15, value not disclosed), and Google claims over 3,000 commercial partnerships in more than 50 countries (2025-12-10), with training not stated.
  4. Neither block nor license: do nothing, which is part of the question of whether to sell or license data to AI companies at all.

Not blocking and not licensing at all is a valid choice, and the options can be combined.

Other crawler controls, Google as a licensee and the law around Gemini training

Google-Extended is one token among many; the pages below cover the other crawlers, Google's licensing deals and the law around crawling.

This page is information, not legal advice.

Frequently asked questions

Does blocking Google-Extended keep content out of AI Overviews?

Google does not say so: its stated scope for Google-Extended is Gemini training and grounding, and it states the token does not affect inclusion in Google Search (checked 2026-11-10). AI Overviews are not named in that scope.

Is Google-Extended a crawler with its own user agent?

No: Google states Google-Extended has no separate HTTP request user agent string and is used only as a robots.txt control token. Google's other crawlers collect the content.

Does a Google-Extended rule stop Googlebot from crawling?

No: a rule naming Google-Extended governs only the uses Google ties to that token, and Googlebot keeps crawling for Search. Blocking Googlebot is a separate rule.

When will Google read the shared AI training token?

Google says it aims to support it in 2027 (blog, 2026-10-30) or early 2027 (crawler documentation), so it does not read it yet. Only a Google-Extended rule works until then.

Does blocking Google-Extended remove content already used for Gemini?

No Google statement on removing content already used is recorded here; a disallow applies to later use of content Google crawls. The block works forward only.