DataPacts

Multilingual and Low-Resource Language Data for AI

Checked

At a glance

Multilingual and low-resource language data is text and speech in languages other than English, above all languages with little digital material, and it is licensed for AI training through vendors, marketplaces, contributor apps and national programs, while several company data programs still ask for English only.

Independent reference. Every buyer is covered on the same terms, every figure carries its source and check date, and no buyer pays to be listed or placed higher.

Request an independent introduction

Tell us what data your company holds. We reply with the buyers that fit and what their terms leave out. Your details reach a buyer only after you agree to a named introduction.

IndependentYour details stay with us until you agreeNot selling is a valid outcome

As of 2026-11-17 the figures on record are language counts that vendors state about themselves, one contributor app that pays less for low-resource audio than for English, and one national license for Norwegian and Sami models.

This page covers language coverage as an attribute of AI training data licensing, the governance of community language material and consent. Translation memories and speech corpora have their own pages, and not licensing is a valid outcome, above all where a community has not decided.

What Counts as Multilingual and Low-Resource Language Data

Multilingual data covers any training data in languages other than English, and low-resource language data is the part of it in languages with little digital text or speech, which is why it is collected, recorded or licensed rather than crawled.

The definition is this page's own: no official list of low-resource languages is on record, so the page gives no count and no list.

The kinds of data are the same as for English, namely speech, text and parallel text. Only the language attribute differs, and with it the pool of holders, speakers and rights.

The table below compares four kinds of language data with one example each, as of 2026-11-17.

Kind of language data What it contains Typical holder Example on record Source
Speech in under-served languages recorded conversation or reading with transcripts contributor apps, vendors, community projects Luel partner audio in Uyghur, Tibetan, Hmong, Jamaican Creole and Guarani luel.ai
Non-English text press, books, web text, reference text publishers, libraries, platforms Norwegian press publications licensed for Norwegian and Sami models kopinor.no
Parallel text source and target segments translation agencies, language teams covered on the translation page n/a
Open multilingual corpora released speech or text in many languages nonprofits, research groups Common Voice Scripted Speech v27.0, 295 languages mozilladatacollective.com

Parallel text and speech have their own pages, and each data type is covered under the data types AI companies buy.

Which Languages Data Programs Accept, and Which Ask for English Only

Company data programs on record mostly ask for English: micro1 wants data primarily in English, and Replay's and The Corner's referral terms ask for English-language companies, while Mercor's publisher and video listings welcome multilingual content and its call-center listing names non-English markets. Every value below is the organization's own statement, checked 2026-11-09.

The table compares the language requirement each program states, as of 2026-11-17, with rows in alphabetical order.

Program or listing Language requirement as stated Source
Luel contributor tasks task cards for American English conversations and for partner audio in Uyghur, Tibetan, Hmong, Jamaican Creole and Guarani; "low-resource language audio" among data wanted luel.ai/contribute
Mercor data listings (published 2026-10-17) publisher and video studio: "Anywhere; multilingual content welcome"; consulting: "Anywhere; English-language work preferred"; call center: India, France and francophone markets, Latin America, U.S. Hispanic markets, Japan api.data.mercor.com listings
micro1 Company Data Partnerships data "primarily in English"; US first, then other Western markets; defunct-company referrals: "English-language data" micro1.ai/data-partnerships; micro1.ai/company-referral
Replay (referral terms) referred companies: "English-language, US preferred" replay.ai
The Corner (micro1 referral page) "70%+ English data" thecorner.ai/data/partnerships

Replay's line comes from its referral terms, not its company offer, and The Corner is a referral page, not a recipient of data. The Mercor call-center listing ties its range to "hours and languages". Luel is the only program on record that names low-resource languages, five of them on its task cards for partner audio.

Organizations Handling Non-English Language Data and Their Language Claims

Vendors, marketplaces and distributors state language coverage in counts that are their own claims: TELUS Digital states 500+ languages and dialects, Centific states 350+ on one page and 200+ on another, SpeechData.ai 60 and David AI 15+.

The table compares what each organization handles and the language claim it makes, as of 2026-11-17, in alphabetical order.

Organization (role) What it acquires or handles Language claim Source
Centific (data vendor) company records under its Enterprise Data program; contributor platform OneForma "350+" languages (about page, checked 2026-11-09); "200+ languages and regional variants" (home page, read 2026-11-16) centific.com
David AI (vendor selling its own datasets) multilingual speech among its own datasets "15+ languages" withdavid.ai
Defined.ai (licensing marketplace) audio, text, image, video and multimodal datasets from data partners none stated defined.ai/partnerships
Language-resource distributors: ELRA, Linguistic Data Consortium, Gengo-Shigen-Kyokai (GSK) speech, text and lexical corpora licensed onward from providers none stated elra.info; ldc.upenn.edu; gsk.or.jp
Luel (contributor app and dataset marketplace) low-resource language audio, conversational audio five low-resource languages named on task cards luel.ai
Mozilla Data Collective (dataset platform) datasets from organizations and communities; launch providers include Karya and Pangeanic, "among others" none stated for Compensated Datasets mozilladatacollective.com
Sieve (data lab licensing archives) podcasts, audiobooks, multilingual recordings among other content none stated sieve.ai/partners
SpeechData.ai (vendor selling its own datasets) conversational, TTS, wake-word and call-center speech "60 languages and dialects"; 73k hours; 7,250 native speakers speechdata.ai
TELUS Digital (AI data services) annotation and data collection for developers "500+" languages and dialects telusdigital.com

A language count describes a vendor's own collection or annotation work, not a program that licenses a holder's data, and TELUS Digital states no public program to license company data. These organizations list, distribute, license onward or handle data; none is a recipient that pays a holder by default.

Open Multilingual Corpora a License Competes With

Large multilingual corpora are free to use under open terms or on their publishers' terms, so a licensed collection has to offer what they lack: Common Voice holds 295 languages under CC0, and FineWeb2 covers 1,868 language-script pairs. The table compares four open corpora with the figures as stated by their publishers, as of 2026-11-17.

Corpus Languages as stated Size and date Terms on record
Common Voice Scripted Speech v27.0 295 languages a little over 42,593 hours, 32,349,999 clips; corpus build 2026-09-11, release post 2026-09-21; Spontaneous Speech v5.0: 80 languages, 302 validated hours CC0
FineWeb2 1,868 language-script pairs about 20 TB, 5 billion documents, over 3 trillion words; v2.1.1, 2025-10-27 not recorded
Institutional Books 1.0 254 languages (summary) and 379 (detected), both on the same page 2025-06-12 not stated here
Wikipedia (Wikimedia Enterprise) over 300 languages more than 65 million articles, 2026-01-15 commercial API access through Wikimedia Enterprise

A licensed collection can add consent documentation, conversational or domain speech, and languages thin in the open corpora. No figure on record shows a gap for any single language.

A National Language License: Training Norwegian and Sami Models

The National Library of Norway licenses Norwegian press publications from Kopinor to build training corpora and train Norwegian and Sami language models, under an agreement signed on 2025-12-19. The license is non-exclusive, the models may be shared publicly under license terms, and rightsholders may opt out for future training under section 16. Termination does not affect models already trained.

The agreement sets NOK 45,000,000 for 2026 (agreement s.17). The Kopinor agreement for Norwegian language models is covered in full on its own page, with scope, term and sources.

The agreement licenses press content, not community-held Sami material, and no license of community or Indigenous language data for AI training is on record.

Community and Indigenous Data Governance: The CARE Principles

The CARE Principles for Indigenous Data Governance (Collective Benefit, Authority to Control, Responsibility, Ethics) were drafted on 8 November 2018 in Gaborone, Botswana, and published in the Data Science Journal on 2020-11-04 to complement the FAIR principles.

The authors are Carroll and colleagues (Data Science Journal 19: 43, DOI 10.5334/dsj-2020-043), and the Global Indigenous Data Alliance records the drafting.

The table quotes each principle's own sentence from the journal article (read 2026-11-17) and pairs it with a question for a training-data license.

Principle The source's own sentence (verbatim) Question for a training-data license
Collective Benefit "Indigenous data must facilitate collective benefit for Indigenous Peoples to achieve inclusive development and innovation" what the community receives
Authority to Control "Recognition of these rights bolsters Indigenous Peoples' authority to control and govern such data" who in the community decides and signs
Responsibility "there is a responsibility to nurture respectful relationships with Indigenous Peoples from whom the data originate" what the acquirer reports back and for how long
Ethics "Paramount to ethics in data practices is representation and participation of Indigenous Peoples" which uses are excluded and who reviews them

CARE are principles, not law. No statute or court ruling in the record makes them binding on a licensee, so a license that follows them does so by agreement.

A licensor or acquirer applies them by asking who in the community decides, what the community receives, which uses are excluded and whether recordings can be withdrawn. The wider questions sit under ethics and risk of selling data for AI training.

Each speaker or writer whose words are in the data needs consent that covers reuse for AI training, and for community-held language material the community's own decision comes on top of individual consent.

Mercor states "Recorded calls with consent that covers reuse" as a requirement on its call-center listing and "Recordings made without consent" as not a fit (checked 2026-11-09). SpeechData.ai states that every speaker signed explicit commercial-use consent, and Defined.ai states data must be "ethically sourced with consent, documentation and right to commercialize" (both checked 2026-11-09).

The community decision follows the CARE principle of Authority to Control. Kopinor's opt-out is an example of a collective license that lets rightsholders withhold works from future training, and no further rule on community consent is stated on this page.

Removing personal data is harder outside English, since Amazon Comprehend documents PII detection for English and Spanish only (checked 2026-11-09). Other languages need human review or other tools, and the steps are in PII removal and de-identification in other languages.

Pay and Price Figures on Record by Language

No program publishes a premium for a rare language; the figures on record are a vendor's per-hour sale prices by language, one listing range that depends on languages, and contributor pay that is lower for low-resource partner audio than for English.

Each figure is the stating party's claim, checked 2026-11-09, in its own unit and never converted or averaged.

Figure Stating party Value as stated Who pays whom Source
Contributor pay, English Luel USD 12 per hour (USD 0.20 per minute), American English three-person conversations Luel pays the contributor luel.ai/contribute
Contributor pay, low-resource partner audio Luel USD 7.50 per hour (USD 0.13 per minute), Uyghur, Tibetan, Hmong, Jamaican Creole, Guarani Luel pays the contributor luel.ai/contribute
Contributor pay, language roles DataAnnotation up to USD 40 per hour across 43 language roles the platform pays the contributor dataannotation.tech
Dataset sale price by language SpeechData.ai USD 60 to 95 per hour of audio; English (US) USD 90, Spanish (MX) USD 65 the buyer pays the vendor speechdata.ai
Listing range Mercor (call-center listing) USD 20,000 to 250,000, depending on hours and languages Mercor to the company api.data.mercor.com

Vendor sale prices are not holder pay, and none of these figures is a price for a holder's language data. Prices across data types are in AI training data prices per hour and record.

Licensing Language Data: Benefits, Risks, Downsides and Alternatives, Including Not Licensing

Licensing language data can fund collection in languages that open corpora cover thinly, but most company programs ask for English, contributor pay for low-resource audio is lower on record, and recordings that reach a trained model cannot be called back.

Benefits

  • Some programs state that other languages are welcome (Mercor's publisher and video listings), and Mercor's call-center listing names non-English markets.
  • Compensated listings let the provider set price and license (Mozilla Data Collective Compensated Datasets).
  • A national licensee exists for a smaller language (Norwegian and Sami models).

Risks

  • Consent gaps for speakers, where recordings were made without reuse consent.
  • Community authority not respected where CARE-style governance applies.
  • De-identification tools document only English and Spanish for one major service.
  • Models already trained survive termination of a license.

Downsides

  • English-only eligibility at micro1, Replay, The Corner and Mercor's consulting listing.
  • Lower contributor pay on record for low-resource partner audio.
  • Open corpora such as Common Voice and FineWeb2 are free competition.

Alternatives

  • Open release under CC0, as Common Voice does.
  • Research-only distribution through a language-resource distributor.
  • Community-governed terms under CARE.
  • Not licensing at all, and keeping the recordings with the community.

The choice belongs to the holder, and for community material it belongs to the community first.

Translation, Speech, Language Work and the Seller Steps for Language Data

This page covers language coverage as an attribute of training data and the governance of community language material; translation memories, speech corpora, contributor language work and each data type have their own pages.

For a holder who proceeds, the steps run as follows.

  1. Read the guide on how to sell or license company data.
  2. Describe languages, speakers and consent in a data manifest describing a dataset.
  3. Negotiate AI data licensing agreements, including field of use and withdrawal clauses.
  4. Agree when AI data buyers pay: upfront, milestones or revenue share.

This page is information, not legal advice.

Frequently Asked Questions

What is low-resource language data?

Low-resource language data is text or speech in a language with little digital material. No official list is on record; Luel names Uyghur, Tibetan, Hmong, Jamaican Creole and Guarani for its partner audio (checked 2026-11-09).

Who buys low-resource language data for AI training?

Contributor apps and vendors name it, and no lab program on record names a low-resource language. Luel, Sieve and SpeechData.ai state language data among what they handle (checked 2026-11-09).

The page on dataset vendors and data labs covers vendors that collect language data.

Do AI companies pay more for rare languages?

No premium for rare languages is on record. Luel's contributor pay for partner audio is USD 7.50 per hour against USD 12 for American English (checked 2026-11-09).

The page on what company data is worth to AI companies covers valuation.

Can an Indigenous community license its language recordings for AI training?

Yes, where the community holds the rights and decides under its own governance. CARE is the framework on record, and no such license is on record.