
Licensed training data is used under a contract with the rights holder that sets scope, term and payment; scraped training data is copied from websites and apps without one, on the developer's own reading of copyright, contract and privacy law.
The difference matters to every holder weighing AI training data licensing against leaving content open, because it decides who is paid, who carries the legal risk and which data a developer can reach at all.
Only one developer has put a number on its licensed share: OpenAI told Canada's privacy regulators that licensed data made up less than 1 % of its pre-training datasets, and the regulators could not verify it (PIPEDA Findings #2026-002, para. 159, 2026-05-06).
Every statement on this page is dated and attributed to the party that made it. DataPacts is independent.
Fact-check note: each figure below is sourced and dated as described in how we source, date and check every figure.
What separates licensed from scraped training data?
The difference is the contract: a license is a permission the holder grants and can price, limit and end, while scraping takes what is reachable and leaves the holder only the controls it set up in advance.
A license gives permission without a sale, so the holder keeps the data and grants use on stated terms.
Scraping gives scale and no counterparty. Licensing reaches data that is not on the open web: Google's post of 2024-02-22 says Reddit's Data API access is "to display, train on, and otherwise use" Reddit content, and clinical records are delivered and priced per record.
The table compares the two routes by attribute (as of 2026-11-10).
| Attribute | Licensed | Scraped | Source |
|---|---|---|---|
| How the developer gets it | contract with the holder; delivery or API access | crawler or third-party crawl archive | Google blog 2024-02-22; OpenAI bot documentation |
| Holder paid | yes, on agreed terms (values rarely stated) | no | Reddit S-1; Reddit 10-Q |
| What the holder controls | scope, term, exclusivity, deletion | crawler rules set in advance, site terms | Tempus 10-Q; Kopinor agreement; OpenAI bot documentation |
| Data not on the open web | possible (APIs, records, archives) | no | Google blog 2024-02-22; Tempus 10-Q |
| After the term ends | trained models kept under the Kopinor agreement | n/a | Kopinor agreement |
| Shown in an EU training summary | Yes or No plus modality only | top 10 % of scraped domain names | Commission template C(2025) 8311 final |
| Typical disputes | contract breach, scope | copyright, terms, anti-circumvention, privacy | Ross, Reddit and OPC records |
Crawling runs outside any contract, and the holder's main tool against it is a crawler rule set in advance: OpenAI states that "disallowing GPTBot indicates a site's content should not be used in training". The AI crawlers that collect training data are listed with their user agents on the crawler hub.
How much AI developers say they license, and how much they crawl
Only one developer has put a number on its licensed share: OpenAI told Canada's privacy regulators that licensed data made up less than 1 % of its pre-training datasets, a figure the regulators could not verify.
The other disclosures name routes, not shares. Anthropic's Form S-1 of 2026-10-26 lists public web data, datasets licensed or commissioned from third parties, and synthetic data generated by its own models.
SpaceXAI's summary for Grok 4.7, placed on the EU market on 2026-09-21, answers Yes to commercial licenses with rights holders for text and image only, and names its own crawler as active from January 2024 to August 2026.
Stability AI states Stable Audio 3.0 is trained on fully licensed data (2026-05-20), and Bria states its models use licensed data only (checked 2026-11-09).
The table sets each developer's own statement side by side, ordered alphabetically (as of 2026-11-10).
| Developer | What it says about licensed and crawled data | Where and when | Source |
|---|---|---|---|
| Anthropic, PBC | public web data, datasets licensed or commissioned from third parties, synthetic data | Form S-1, 2026-10-26 | sec.gov |
| Bria | trained exclusively on licensed data from partners including Getty Images, Alamy and Envato; 30+ licensed data partners | own site, checked 2026-11-09 | bria.ai |
| OpenAI | licensed data less than 1 % of pre-training datasets; the regulators could not verify it | PIPEDA Findings #2026-002, para. 159, 2026-05-06 | priv.gc.ca |
| Stability AI | Stable Audio 3.0 "trained on fully licensed data" | newsroom, 2026-05-20 | stability.ai |
| SpaceXAI (xAI) | commercial licenses Yes for text and image; private third-party datasets Yes; own crawler January 2024 to August 2026 | Grok 4.7 EU summary, 2026-09-21 | media.x.ai |
No share exists across the industry, because the template for EU training content summaries asks only Yes or No plus modality for licensed data, while it asks for the top 10 % of scraped domain names.
The xAI document is the only such summary on record here. Under the EU AI Act Article 53 training data summary (Regulation (EU) 2024/1689, applying from 2025-08-02), even full compliance yields no paid share.
What licensors and their executives have said on the record
Licensors disclose more than developers, because a listed company has to report what its data earns, yet most still give no value per deal.
Reddit's S-1 reports USD 203.0 million of aggregate contract value from arrangements entered in January 2024, with terms of two to three years, counterparties that are not named, and at least USD 66.4 million of revenue expected in 2024.
Reddit's 10-Q of 2026-10-29 says substantially all of its licensing value comes from two partners whose agreements are "approaching the end of their respective current terms".
Named executives add scope without values. Condé Nast's CEO Roger Lynch wrote on 2024-08-20 that the OpenAI partnership carries "proper attribution and compensation", without stating training.
News Corp's CEO Robert Thomson said on 2026-11-04: "We are pleased to have partners in OpenAI and Meta", and Microsoft's Tim Frank wrote on 2026-02-03 that publishers in its marketplace are "paid on delivered value", a grounding program that does not mention training.
The table lists licensor statements by date (as of 2026-11-10).
| Date | Licensor | Statement | Stated by | Source |
|---|---|---|---|---|
| 2023-12-13 | Axel Springer | content used "for advancing the training" of OpenAI's models | Axel Springer release | axelspringer.com |
| 2024-02-22 | Google gets Data API access "to display, train on, and otherwise use" Reddit content | Google blog | blog.google | |
| 2024-02 (S-1) | USD 203.0 million aggregate contract value, January 2024 arrangements; counterparties not named | Reddit S-1 | sec.gov | |
| 2024-08-20 | Condé Nast | "proper attribution and compensation" | CEO Roger Lynch, memo | condenast.com |
| 2025-05-14 | Le Monde | Perplexity deal does not provide for training an LLM | Le Monde | lemonde.fr |
| 2025-07-31 | Gannett | Perplexity deal "represents fair value of our content" | Gannett CEO, 8-K Ex. 99.1 | sec.gov |
| 2026-02-03 | Microsoft (marketplace operator) | publishers "paid on delivered value", grounding | Tim Frank, Microsoft blog | about.ads.microsoft.com |
| 2026-10-29 | two main agreements "approaching the end of their respective current terms" | Reddit 10-Q | sec.gov | |
| 2026-11-04 | News Corp | "We are pleased to have partners in OpenAI and Meta" | CEO Robert Thomson, Q1 FY2027 release | sec.gov |
Scope differs by deal. Axel Springer's release of 2023-12-13 names training, Le Monde's OpenAI deal of 2024-03-13 covers it, and its Perplexity deal of 2025-05-14 excludes it.
On the certification side, Ed Newton-Rex launched Fairly Trained with 9 certified models on 2024-01-22. Each deal with its parties, dates and sources sits in the tracker of AI data licensing deals with parties, dates and sources.
Why developers pay for licensed data when the web can be crawled
Developers pay when a license gives them something a crawl cannot: data that is not public, permission a court may ask about, exclusivity, or the right to say a model is licensed.
- Access to non-public data. Tempus's neurology order form with Pathos (10-Q, 2026-10-22) is worth up to USD 55.0 million, of which USD 50.0 million is USD 100 per delivered record; an earlier Pathos license, per Tempus's 10-K, was USD 200 million over three years for training a foundation model.
- Legal position. The Third Circuit wrote in Ross that "the market for licensing headnotes as text to train AI is rapidly developing", while Judge Chhabria held in Kadrey (Dkt. 598, 2025-06-25) that the plaintiffs were "not entitled to the market for licensing their works as AI training data".
- Exclusivity. The Tempus side letter bars licensing the records to others for 18 months after the later of delivery and 2027-10-22.
- Terms a crawl lacks. Kopinor's agreement with the National Library of Norway (signed 2025-12-19) pays NOK 45,000,000 for 2026 (s.17), lets rights holders opt out with effect for future training (s.16) and leaves models already trained untouched on termination. In the HarperCollins deal, the Authors Guild reports USD 5,000 per title, split 50-50 with the author, and a pledge by the licensee not to scrape piracy sites (2024-11-19).
- Product claims. "Fully licensed" and "exclusively licensed" are marketing terms that only a license supports.
The price does not recur on its own. Wiley's USD 23 million was a one-time content rights project in the quarter ended 2024-04-30, and its AI license revenue in the quarter ended 2026-07-31 was USD 13.7 million against USD 28.9 million a year earlier.
Informa called USD 75m+ of data access revenue "non-recurring" in its 2024 full-year results, and Reddit's two main agreements are near their end. Levels per work, hour, image, token and record are tabulated under what buyers pay per work, hour, image, token and record.
Legal exposure of scraped and licensed data, claim by claim
Scraping carries the claims a license would have settled: copyright, the site's terms, anti-circumvention and privacy, while a license moves most of that risk into a contract between two parties. The table maps each claim to both routes (as of 2026-11-10).
| Claim | Scraped data | Licensed data | Record | Source |
|---|---|---|---|---|
| Copyright (US) | fair use decided case by case; pirated copies not excused | covered by the grant, within scope | Bartz Dkt. 231; Ross 3d Cir. No. 25-2153 | court records |
| Copyright (EU) | lawful only absent a machine-readable reservation | covered by the grant | Directive (EU) 2019/790, Art. 4(3) | eur-lex.europa.eu |
| Site terms and contract | breach of terms claims | breach of license scope | Reddit v. Anthropic | courtlistener.com; sec.gov |
| Anti-circumvention | 17 U.S.C. 1201 claims | not applicable to delivered data | Reddit v. SerpApi, Dkt. 104 | court record |
| Privacy | no implied consent for scraped data | no implied consent unless individuals had a choice | PIPEDA Findings #2026-002 | priv.gc.ca |
| Allocation of risk | on the developer | shifted by warranties and indemnities | Grepped terms | grepped.ai |
On copyright, Judge Alsup held in Bartz (N.D. Cal. 3:24-cv-05417-AMO, Dkt. 231, 2025-06-23) that training was fair use and that scanning purchased books was fair use, but that the pirated library copies were not excused; the case settled for USD 1.5 billion, with final approval on 2026-07-20. Ross was held not fair use on affirmance by the Third Circuit on 2026-09-29, and ROSS filed a rehearing petition dated 2026-11-03. In the EU, a rights holder can reserve online content by machine-readable means under Article 4(3) of Directive (EU) 2019/790. The same questions are analyzed in copyright and AI training across jurisdictions.
On contract and access, Reddit v. Anthropic was remanded to state court on 2026-03-30 because the state claims are not preempted (Dkt. 60).
As of 2026-11-09 it is pending, narrowed in September 2026 to contract, interference with contract and unfair competition, per Anthropic's S-1/A. In Reddit v. SerpApi and Perplexity (S.D.N.Y. 1:25-cv-08736-PAE), Dkt. 104 of 2026-07-31 sustained the 17 U.S.C. 1201(a)(1)(A) claims.
On privacy, the Canadian offices found that OpenAI had no implied consent for web-scraped or for licensed data, and that licensed datasets require that individuals were given a choice (PIPEDA Findings #2026-002, paras 119, 160 and 162, concluded 2026-05-06).
EDPB Guidelines 03/2026 on web scraping were open for consultation until 2026-10-30, with no final version recorded. A licensee also takes on contract exposure: Grepped's terms of 2026-11-08 have the seller indemnify Grepped for third-party and regulator claims.
This page is information, not legal advice. For the scraping side in depth, see whether web scraping for AI training is legal.
License, leave open or block: benefits, risks, downsides and alternatives for a data holder
A holder whose content can be crawled chooses between licensing it, leaving it open and blocking it, and not licensing at all is a valid outcome.
Benefits of licensing
- Payment and terms are on record, as in Reddit's S-1 and the Kopinor agreement.
- The holder sets the scope, such as display or training only, as Le Monde's two deals differ.
- A counterparty exists to enforce against; without a license, Reddit's claims against Anthropic rest on its site terms.
Risks
- A license may still breach privacy law if individuals had no choice (PIPEDA Findings #2026-002).
- Trained models usually survive termination (Kopinor agreement).
- Income concentrates in few buyers and ends with the term (Reddit 10-Q, 2026-10-29).
Downsides
- Few deals state a value.
- Leaving content open returns little traffic: Cloudflare measured Anthropic's crawl-to-refer ratio at 70,900:1 (2025-06-19 to 06-26) and nearly 50,000:1 (2025-08-01 to 08-07), two different windows, and the ratios may be overstated because Claude's app sends no Referer.
Alternatives
- Block training crawlers with robots.txt rules.
- Charge per crawl through a metering service.
- Grant permission without payment where a platform offers it.
- Do not license, keep records private and read whether to sell or license your data at all first.
Beyond this comparison: opt-outs, user terms, deals and the law
This page compares two routes; the pages below cover the other routes, the controls a holder can set and the law behind each claim.
A platform setting keeps content out of that service's own training only, and it is a separate control from a license; the settings are listed under opting out of AI training through platform settings.
- How AI companies get training data: the parent page with all nine routes.
- User data and platform terms: when services train on their users.
- Pirated books and shadow libraries in AI training: the unlicensed book copies in court records.
- Public domain and openly licensed material: works that need no payment.
- Pay-per-crawl and crawler licensing: metered access for crawlers.
- Training data provenance and chain of rights: documentation a buyer asks for.
- How frontier AI labs acquire data: lab by lab.
- Reddit's AI data licensing deals with Google and OpenAI: terms and sources.
- Bartz v. Anthropic and the settlement, Thomson Reuters v. Ross Intelligence, Kadrey v. Meta and Reddit v. Anthropic: the case records.
- The Canadian privacy commissioners' OpenAI investigation: the findings on consent.
- The history of AI training data from scraped corpora to licensing deals: the timeline.
- A rights audit of the data you want to license and AI data licensing agreements, clause by clause: the steps before a license.
Frequently asked questions
Is it illegal to train AI on scraped data?
Not as such: legality depends on copyright, the site's terms and privacy law, and courts differ. Bartz held training on books fair use on 2025-06-23 but not the pirated copies, while Ross held a non-generative use not fair use on 2026-09-29.
What does rights-cleared training data mean?
Rights-cleared training data is data whose holder has granted training rights and whose chain of title is documented. The documentation is the subject of the provenance page above, and no regulator defines the term.
Does licensed data remove privacy obligations?
No. The Canadian offices found on 2026-05-06 that licensed datasets require that individuals were given a choice, and OpenAI lacked implied consent for licensed data (PIPEDA Findings #2026-002).
How much do AI companies pay for licensed data?
Most values are undisclosed, so no typical price exists. Reddit's S-1 aggregate is one exception among filings. Methods and public price points are in what company data is worth to AI companies.