Anthropic and India Eye Joint AI Benchmarks After G20 Ministerial Talks
India's MoS IT Jitin Prasada met Anthropic co-founder Tom Brown at the G20 Innovation Ministers' Meeting to discuss India-specific AI benchmarks and Claude's second-largest market.
India's Minister of State for Electronics and IT, Jitin Prasada, met Anthropic co-founder Tom Brown at the G20 Innovation Ministers' Meeting in Chapel Hill on September 8, 2026. The two discussed the AI landscape and the possibility of jointly developing benchmarks calibrated to Indian conditions. The meeting signals a deeper policy relationship between India's government and Anthropic, which already counts India as its second-largest market for Claude and opened its first local office in Bengaluru earlier this year.
What happened
| Detail | Fact |
|---|---|
| Event | G20 Innovation Ministers’ Meeting, Chapel Hill |
| Date | September 8, 2026 |
| Participants | Jitin Prasada (MoS Electronics and IT), Tom Brown (Anthropic co-founder) |
| India’s rank for Claude | Second-largest market globally |
| Anthropic’s Bengaluru office opened | February 2026 |
| Languages covered in training data initiative | 10 (Hindi, Bengali, Marathi, Telugu, Tamil, Punjabi, Gujarati, Kannada, Malayalam, Urdu) |
India’s Ministry of Electronics and IT (MeitY) confirmed that Prasada and Brown discussed the potential for collaboration on AI benchmarks built specifically for Indian use cases. According to MeitY, key areas on the table included benchmarks relevant to India’s context, rather than the Western-centric evaluations that most frontier AI labs currently rely on.
Anthropic is already building toward this goal. The company says it is working with Karya and the Collective Intelligence Project to create evaluation datasets that test model performance on locally relevant tasks in agriculture and law. Domain experts from Digital Green and Adalat AI are involved as partners.
Why does India want its own AI benchmarks?
Standard AI benchmarks such as MMLU or HumanEval are built largely on English-language data and Western institutional knowledge. They tell you little about how well a model handles a question about crop disease in a regional Indian language, or how it navigates Indian contract law. For a government that wants to deploy AI in public services at scale, that gap is a real problem.
Anthropic has been aware of this. Last year the company launched an initiative to collect higher-quality, representative training data in 10 widely spoken Indian languages. According to Anthropic, that work has already improved its models. The benchmark collaboration discussed at the G20 meeting would take this a step further, producing formal evaluation tools that other developers could also use.
What else came out of the G20 meetings?
Prasada held bilateral sessions with representatives from the European Union, France, Germany, the African Union, and the United States. In a meeting with US Secretary of Commerce Howard Lutnick, both sides discussed cooperation on semiconductors, AI, and data centers. Prasada also raised the prospect of more US investment in ISM2.0 (India’s semiconductor mission), proposed strengthening the AI technology partnership, and shared a mechanism for AI safeguards.
MeitY framed the engagements as evidence of India’s growing role in shaping global technology policy, pointing to AI, semiconductors, digital public infrastructure, and robotics as areas of strategic focus.
Our take
The benchmark angle is the most substantive part of this story. Government-to-company meetings at international forums often produce little beyond a photo opportunity. This one has a concrete output already taking shape: Anthropic and its NGO partners are actively building India-focused evaluation datasets. If those benchmarks become publicly available, they could raise the floor for AI performance across Indian languages well beyond Anthropic’s own models.
The second-largest market claim is worth noting too. Anthropic does not publish usage numbers, so the ranking is self-reported. But the Bengaluru office, the 10-language training data push, and now a ministerial meeting all point to genuine strategic investment rather than a marketing label. For businesses thinking about AI integration for multilingual or emerging-market audiences, the India benchmark work could eventually provide much better quality signals than anything available today.
Watch whether MeitY and Anthropic publish a formal memorandum of understanding or a public benchmark specification. A stated goal is easy; an open, auditable benchmark is the thing that actually moves the needle. You can track further developments in our AI news coverage as this story develops.
What to do about it
- If you build products for Indian users in any Indian language, check Anthropic’s existing Claude models against your real-world tasks now. Performance may already be better than you expect after last year’s training data update.
- Watch for the public release of evaluation datasets from Karya and the Collective Intelligence Project. These could become the standard for testing multilingual AI quality.
- If semiconductors or data center investment in India is relevant to your business, the ISM2.0 program mentioned in US bilateral talks is worth tracking for procurement and partnership opportunities.
Frequently asked questions
Is India really Anthropic's second-largest market?
According to Anthropic, India is the second-largest market for Claude globally. The company opened its first Indian office in Bengaluru in February 2026 and launched a 10-language Indian training data initiative last year.
What are India-specific AI benchmarks?
They are evaluation datasets designed to test how well AI models perform on tasks relevant to India, such as agriculture questions in regional languages or Indian legal queries. Standard benchmarks like MMLU are largely English-language and Western in scope.
Which Indian languages is Anthropic training Claude on?
Anthropic's initiative covers 10 languages: Hindi, Bengali, Marathi, Telugu, Tamil, Punjabi, Gujarati, Kannada, Malayalam, and Urdu.
Who is Anthropic working with to build Indian AI evaluations?
Anthropic is working with Karya and the Collective Intelligence Project to build the evaluation datasets, with domain experts from Digital Green and Adalat AI providing input on agriculture and law respectively.
