DataPacts

Licensed vs Scraped Training Data

Checked

Independent reference. Every buyer is covered on the same terms, every figure carries its source and check date, and no buyer pays to be listed or placed higher.

Licensed and scraped training data compared row by row: how each is obtained, whether the holder is paid, what the holder controls, access to data off the open web, and the typical disputes.

Licensed training data is used under a contract with the rights holder that sets scope, term and payment; scraped training data is copied from websites and apps without one, on the developer's own reading of copyright, contract and privacy law.

The difference matters to every holder weighing AI training data licensing against leaving content open, because it decides who is paid, who carries the legal risk and which data a developer can reach at all.

Only one developer has put a number on its licensed share: OpenAI told Canada's privacy regulators that licensed data made up less than 1 % of its pre-training datasets, and the regulators could not verify it (PIPEDA Findings #2026-002, para. 159, 2026-05-06).

Every statement on this page is dated and attributed to the party that made it. DataPacts is independent.

Fact-check note: each figure below is sourced and dated as described in how we source, date and check every figure.

What separates licensed from scraped training data?

The difference is the contract: a license is a permission the holder grants and can price, limit and end, while scraping takes what is reachable and leaves the holder only the controls it set up in advance.

A license gives permission without a sale, so the holder keeps the data and grants use on stated terms.

Scraping gives scale and no counterparty. Licensing reaches data that is not on the open web: Google's post of 2024-02-22 says Reddit's Data API access is "to display, train on, and otherwise use" Reddit content, and clinical records are delivered and priced per record.

The table compares the two routes by attribute (as of 2026-11-10).

Attribute Licensed Scraped Source
How the developer gets it contract with the holder; delivery or API access crawler or third-party crawl archive Google blog 2024-02-22; OpenAI bot documentation
Holder paid yes, on agreed terms (values rarely stated) no Reddit S-1; Reddit 10-Q
What the holder controls scope, term, exclusivity, deletion crawler rules set in advance, site terms Tempus 10-Q; Kopinor agreement; OpenAI bot documentation
Data not on the open web possible (APIs, records, archives) no Google blog 2024-02-22; Tempus 10-Q
After the term ends trained models kept under the Kopinor agreement n/a Kopinor agreement
Shown in an EU training summary Yes or No plus modality only top 10 % of scraped domain names Commission template C(2025) 8311 final
Typical disputes contract breach, scope copyright, terms, anti-circumvention, privacy Ross, Reddit and OPC records

Crawling runs outside any contract, and the holder's main tool against it is a crawler rule set in advance: OpenAI states that "disallowing GPTBot indicates a site's content should not be used in training". The AI crawlers that collect training data are listed with their user agents on the crawler hub.

How much AI developers say they license, and how much they crawl

Only one developer has put a number on its licensed share: OpenAI told Canada's privacy regulators that licensed data made up less than 1 % of its pre-training datasets, a figure the regulators could not verify.

The other disclosures name routes, not shares. Anthropic's Form S-1 of 2026-10-26 lists public web data, datasets licensed or commissioned from third parties, and synthetic data generated by its own models.

SpaceXAI's summary for Grok 4.7, placed on the EU market on 2026-09-21, answers Yes to commercial licenses with rights holders for text and image only, and names its own crawler as active from January 2024 to August 2026.

Stability AI states Stable Audio 3.0 is trained on fully licensed data (2026-05-20), and Bria states its models use licensed data only (checked 2026-11-09).

The table sets each developer's own statement side by side, ordered alphabetically (as of 2026-11-10).

Developer What it says about licensed and crawled data Where and when Source
Anthropic, PBC public web data, datasets licensed or commissioned from third parties, synthetic data Form S-1, 2026-10-26 sec.gov
Bria trained exclusively on licensed data from partners including Getty Images, Alamy and Envato; 30+ licensed data partners own site, checked 2026-11-09 bria.ai
OpenAI licensed data less than 1 % of pre-training datasets; the regulators could not verify it PIPEDA Findings #2026-002, para. 159, 2026-05-06 priv.gc.ca
Stability AI Stable Audio 3.0 "trained on fully licensed data" newsroom, 2026-05-20 stability.ai
SpaceXAI (xAI) commercial licenses Yes for text and image; private third-party datasets Yes; own crawler January 2024 to August 2026 Grok 4.7 EU summary, 2026-09-21 media.x.ai

No share exists across the industry, because the template for EU training content summaries asks only Yes or No plus modality for licensed data, while it asks for the top 10 % of scraped domain names.

The xAI document is the only such summary on record here. Under the EU AI Act Article 53 training data summary (Regulation (EU) 2024/1689, applying from 2025-08-02), even full compliance yields no paid share.

What licensors and their executives have said on the record

Licensors disclose more than developers, because a listed company has to report what its data earns, yet most still give no value per deal.

Reddit's S-1 reports USD 203.0 million of aggregate contract value from arrangements entered in January 2024, with terms of two to three years, counterparties that are not named, and at least USD 66.4 million of revenue expected in 2024.

Reddit's 10-Q of 2026-10-29 says substantially all of its licensing value comes from two partners whose agreements are "approaching the end of their respective current terms".

Named executives add scope without values. Condé Nast's CEO Roger Lynch wrote on 2024-08-20 that the OpenAI partnership carries "proper attribution and compensation", without stating training.

News Corp's CEO Robert Thomson said on 2026-11-04: "We are pleased to have partners in OpenAI and Meta", and Microsoft's Tim Frank wrote on 2026-02-03 that publishers in its marketplace are "paid on delivered value", a grounding program that does not mention training.

The table lists licensor statements by date (as of 2026-11-10).

Date Licensor Statement Stated by Source
2023-12-13 Axel Springer content used "for advancing the training" of OpenAI's models Axel Springer release axelspringer.com
2024-02-22 Reddit Google gets Data API access "to display, train on, and otherwise use" Reddit content Google blog blog.google
2024-02 (S-1) Reddit USD 203.0 million aggregate contract value, January 2024 arrangements; counterparties not named Reddit S-1 sec.gov
2024-08-20 Condé Nast "proper attribution and compensation" CEO Roger Lynch, memo condenast.com
2025-05-14 Le Monde Perplexity deal does not provide for training an LLM Le Monde lemonde.fr
2025-07-31 Gannett Perplexity deal "represents fair value of our content" Gannett CEO, 8-K Ex. 99.1 sec.gov
2026-02-03 Microsoft (marketplace operator) publishers "paid on delivered value", grounding Tim Frank, Microsoft blog about.ads.microsoft.com
2026-10-29 Reddit two main agreements "approaching the end of their respective current terms" Reddit 10-Q sec.gov
2026-11-04 News Corp "We are pleased to have partners in OpenAI and Meta" CEO Robert Thomson, Q1 FY2027 release sec.gov

Scope differs by deal. Axel Springer's release of 2023-12-13 names training, Le Monde's OpenAI deal of 2024-03-13 covers it, and its Perplexity deal of 2025-05-14 excludes it.

On the certification side, Ed Newton-Rex launched Fairly Trained with 9 certified models on 2024-01-22. Each deal with its parties, dates and sources sits in the tracker of AI data licensing deals with parties, dates and sources.

Why developers pay for licensed data when the web can be crawled

Developers pay when a license gives them something a crawl cannot: data that is not public, permission a court may ask about, exclusivity, or the right to say a model is licensed.

  • Access to non-public data. Tempus's neurology order form with Pathos (10-Q, 2026-10-22) is worth up to USD 55.0 million, of which USD 50.0 million is USD 100 per delivered record; an earlier Pathos license, per Tempus's 10-K, was USD 200 million over three years for training a foundation model.
  • Legal position. The Third Circuit wrote in Ross that "the market for licensing headnotes as text to train AI is rapidly developing", while Judge Chhabria held in Kadrey (Dkt. 598, 2025-06-25) that the plaintiffs were "not entitled to the market for licensing their works as AI training data".
  • Exclusivity. The Tempus side letter bars licensing the records to others for 18 months after the later of delivery and 2027-10-22.
  • Terms a crawl lacks. Kopinor's agreement with the National Library of Norway (signed 2025-12-19) pays NOK 45,000,000 for 2026 (s.17), lets rights holders opt out with effect for future training (s.16) and leaves models already trained untouched on termination. In the HarperCollins deal, the Authors Guild reports USD 5,000 per title, split 50-50 with the author, and a pledge by the licensee not to scrape piracy sites (2024-11-19).
  • Product claims. "Fully licensed" and "exclusively licensed" are marketing terms that only a license supports.

The price does not recur on its own. Wiley's USD 23 million was a one-time content rights project in the quarter ended 2024-04-30, and its AI license revenue in the quarter ended 2026-07-31 was USD 13.7 million against USD 28.9 million a year earlier.

Informa called USD 75m+ of data access revenue "non-recurring" in its 2024 full-year results, and Reddit's two main agreements are near their end. Levels per work, hour, image, token and record are tabulated under what buyers pay per work, hour, image, token and record.

Scraping carries the claims a license would have settled: copyright, the site's terms, anti-circumvention and privacy, while a license moves most of that risk into a contract between two parties. The table maps each claim to both routes (as of 2026-11-10).

Claim Scraped data Licensed data Record Source
Copyright (US) fair use decided case by case; pirated copies not excused covered by the grant, within scope Bartz Dkt. 231; Ross 3d Cir. No. 25-2153 court records
Copyright (EU) lawful only absent a machine-readable reservation covered by the grant Directive (EU) 2019/790, Art. 4(3) eur-lex.europa.eu
Site terms and contract breach of terms claims breach of license scope Reddit v. Anthropic courtlistener.com; sec.gov
Anti-circumvention 17 U.S.C. 1201 claims not applicable to delivered data Reddit v. SerpApi, Dkt. 104 court record
Privacy no implied consent for scraped data no implied consent unless individuals had a choice PIPEDA Findings #2026-002 priv.gc.ca
Allocation of risk on the developer shifted by warranties and indemnities Grepped terms grepped.ai

On copyright, Judge Alsup held in Bartz (N.D. Cal. 3:24-cv-05417-AMO, Dkt. 231, 2025-06-23) that training was fair use and that scanning purchased books was fair use, but that the pirated library copies were not excused; the case settled for USD 1.5 billion, with final approval on 2026-07-20. Ross was held not fair use on affirmance by the Third Circuit on 2026-09-29, and ROSS filed a rehearing petition dated 2026-11-03. In the EU, a rights holder can reserve online content by machine-readable means under Article 4(3) of Directive (EU) 2019/790. The same questions are analyzed in copyright and AI training across jurisdictions.

On contract and access, Reddit v. Anthropic was remanded to state court on 2026-03-30 because the state claims are not preempted (Dkt. 60).

As of 2026-11-09 it is pending, narrowed in September 2026 to contract, interference with contract and unfair competition, per Anthropic's S-1/A. In Reddit v. SerpApi and Perplexity (S.D.N.Y. 1:25-cv-08736-PAE), Dkt. 104 of 2026-07-31 sustained the 17 U.S.C. 1201(a)(1)(A) claims.

On privacy, the Canadian offices found that OpenAI had no implied consent for web-scraped or for licensed data, and that licensed datasets require that individuals were given a choice (PIPEDA Findings #2026-002, paras 119, 160 and 162, concluded 2026-05-06).

EDPB Guidelines 03/2026 on web scraping were open for consultation until 2026-10-30, with no final version recorded. A licensee also takes on contract exposure: Grepped's terms of 2026-11-08 have the seller indemnify Grepped for third-party and regulator claims.

This page is information, not legal advice. For the scraping side in depth, see whether web scraping for AI training is legal.

License, leave open or block: benefits, risks, downsides and alternatives for a data holder

A holder whose content can be crawled chooses between licensing it, leaving it open and blocking it, and not licensing at all is a valid outcome.

Benefits of licensing

  • Payment and terms are on record, as in Reddit's S-1 and the Kopinor agreement.
  • The holder sets the scope, such as display or training only, as Le Monde's two deals differ.
  • A counterparty exists to enforce against; without a license, Reddit's claims against Anthropic rest on its site terms.

Risks

  • A license may still breach privacy law if individuals had no choice (PIPEDA Findings #2026-002).
  • Trained models usually survive termination (Kopinor agreement).
  • Income concentrates in few buyers and ends with the term (Reddit 10-Q, 2026-10-29).

Downsides

  • Few deals state a value.
  • Leaving content open returns little traffic: Cloudflare measured Anthropic's crawl-to-refer ratio at 70,900:1 (2025-06-19 to 06-26) and nearly 50,000:1 (2025-08-01 to 08-07), two different windows, and the ratios may be overstated because Claude's app sends no Referer.

Alternatives

  • Block training crawlers with robots.txt rules.
  • Charge per crawl through a metering service.
  • Grant permission without payment where a platform offers it.
  • Do not license, keep records private and read whether to sell or license your data at all first.

Beyond this comparison: opt-outs, user terms, deals and the law

This page compares two routes; the pages below cover the other routes, the controls a holder can set and the law behind each claim.

A platform setting keeps content out of that service's own training only, and it is a separate control from a license; the settings are listed under opting out of AI training through platform settings.

Frequently asked questions

Is it illegal to train AI on scraped data?

Not as such: legality depends on copyright, the site's terms and privacy law, and courts differ. Bartz held training on books fair use on 2025-06-23 but not the pirated copies, while Ross held a non-generative use not fair use on 2026-09-29.

What does rights-cleared training data mean?

Rights-cleared training data is data whose holder has granted training rights and whose chain of title is documented. The documentation is the subject of the provenance page above, and no regulator defines the term.

Does licensed data remove privacy obligations?

No. The Canadian offices found on 2026-05-06 that licensed datasets require that individuals were given a choice, and OpenAI lacked implied consent for licensed data (PIPEDA Findings #2026-002).

How much do AI companies pay for licensed data?

Most values are undisclosed, so no typical price exists. Reddit's S-1 aggregate is one exception among filings. Methods and public price points are in what company data is worth to AI companies.