Large Language Models (LLMs) Training in India

Source: The post “Large Language Models (LLMs) Training in India” has been created, based on “How are Indian firms training LLMs? | Explained” published in “The Hindu” on  26th February 2026.

UPSC Syllabus: GS Paper-3-Science and technology

Context:  Bengaluru-based Sarvam AI released two Large Language Models trained on 35 billion and 105 billion parameters at the AI Impact Summit in New Delhi. These models aim to improve performance in Indian languages while maintaining computational efficiency. This initiative aligns with India’s broader effort under the IndiaAI Mission to build indigenous AI capabilities.

Training of Large Language Models (LLMs)

1. Large Language Models are trained using clusters of Graphics Processing Units (GPUs), which handle the intensive parallel computations required for deep learning.
2. The training process requires enormous datasets, most of which are scraped from the internet and are dominated by English and other globally prominent languages.
3. The combined cost of GPUs, electricity, cooling systems, and infrastructure often runs into millions of dollars for a single training run.
4. Indian firms are focusing on building mid-sized models that balance performance and efficiency, rather than immediately attempting trillion-parameter frontier systems.
5. For instance, Sarvam AI trained its models from scratch using subsidised compute access, while BharatGen developed a 17-billion-parameter multilingual model for sectors such as healthcare and education.

Challenges in Training Large Language Models on Indian Soil

 I. Data Scarcity

1. Indian languages are underrepresented on the internet, resulting in limited high-quality datasets for training.
2. Many LLMs rely on translating Indian-language inputs into English internally, which increases token usage and inference cost.
3. Translation-based approaches may reduce efficiency and affect performance in low-resource settings.

II. Capital Constraints

1. The acquisition and operation of high-end GPUs require substantial capital investment, which is often beyond the reach of startups.
2. Indian firms do not yet possess the financial scale of global technology giants that can absorb massive upfront costs.
3. There may be limited immediate commercial returns to justify frontier-scale model training.

III. Infrastructure Limitations

1. Access to advanced semiconductor hardware remains dependent on imports.
2. Establishing and maintaining large-scale AI data centres requires long-term strategic planning and financial commitment.

Government Support under IndiaAI Mission

1. The IndiaAI Mission has commissioned over 36,000 GPUs in domestic data centres to support AI development.
2. The government has collaborated with firms such as Yotta to host this infrastructure.
3. Startups and research institutions can access GPU clusters at subsidised rates for training and inference.
4. Sarvam AI received access to 4,096 GPUs, with an estimated subsidy of nearly ₹100 crore.
5. The total cost of the compute cluster is approximately ₹246 crore, and the infrastructure remains reusable for other developers.
6. The initiative seeks to promote technological sovereignty, enhance AI talent, and ensure that Indian languages receive adequate representation in AI systems.

Cost Efficiency of the Mixture of Experts (MoE) Architecture

1. In traditional dense LLMs, all parameters are activated during inference, making each query computationally expensive.
2. The Mixture of Experts (MoE) architecture activates only a subset of parameters for any given query.
3. This selective activation reduces computational load and lowers energy consumption.
4. As a result, MoE models achieve faster inference speeds at lower operational costs.
5. Therefore, a 105-billion-parameter MoE model can be significantly cheaper to operate than a dense model with the same parameter size.

Way Forward

1. The government should expand initiatives to create high-quality, publicly available datasets in Indian languages through collaboration with academic institutions and state agencies.
2. Greater private sector participation and venture capital investment are required to sustain large-scale AI research and development.
3. India should strengthen its semiconductor manufacturing and GPU supply chain capabilities to reduce import dependence.
4. Transparent benchmarking and open evaluation of domestic models should be encouraged to build global credibility.
5. Sector-specific LLMs tailored for governance, healthcare, education, and agriculture should be prioritised to generate immediate social and economic impact.
6. Long-term public–private partnerships should be institutionalised to ensure continuity of compute infrastructure and research funding.

Conclusion: India’s efforts to develop indigenous Large Language Models represent a strategic move toward digital sovereignty and technological self-reliance. Although challenges related to data availability, capital investment, and infrastructure remain significant, targeted government support through the IndiaAI Mission and the adoption of efficient architectures such as MoE have laid a strong foundation. Sustained investment, ecosystem development, and policy stability will determine India’s ability to compete with global frontier AI systems.

Question:  Examine how Indian firms are developing Large Language Models (LLMs). Discuss the challenges of training them in India, the role of the IndiaAI Mission, the cost advantage of Mixture of Experts architecture, and suggest the way forward.

Source: The Hindu