Tech Mahindra to launch Hindi language model by April; to focus on Bangla next: Nikhil Malhotra

After announcing a major bet on generative artificial intelligence (AI) in the form of Project Indus last year, IT services giant Tech Mahindra is all set to release the first phase of the model by April. For the uninitiated, Project Indus is a language model built grounds up, on the lines of other popular models like OpenAI’s GPT and Meta’s Llama. What makes it unique, however, is that it is built for Hindi language, along with its 37 dialects. 

The Indus model, currently in beta within the company, is designed for Hindi and its lesser-known dialects, including Bhojpuri, Maithli, Braj Bhasha, Khortha, Kurmali, among others. Nikhil Malhotra, global head of Tech Mahindra’s Makers Lab, shared that the Indus model has 539 million parameters and 10 billion tokens. Tokens represent the units of text the model can read or generate.

“The model currently is in a pre-training phase. The 10 billion token model was a good way to start off with because we wanted to see how the model operates in a pre training phase. We are now trying to fine tune the model in terms of question and answer so that it becomes very similar to what ChatGPT looks like, but for Hindi and related dialects. We are not restricting ourselves to 539 million parameters; the aim is that by March end or beginning of April, we release a 1.5 to 3 billion parameter model in open source,” said Malhotra. “Once we release the Hindi language model, we will next focus on Bangla. After Hindi, which is spoken by 615 million people, Bangla is widely spoken by up to 400 million people. While not as diverse as Hindi, Bangla too has several dialects.”

Malhotra said that collecting data for the Indus model was one of the main challenges. To counter them, Project Indus’ team came up with a lot of out of the box techniques to counter it. “One of the things we did was opening a site and asking people to do ‘Bhasha Daan’, which involved giving them general prompts and asking them to respond in their language. We also involved our Tech Mahindra employees, many of whom belonged to communities that speak these languages.” For dialects which were heavily underrepresented, Tech Mahindra dispatched its teams to places like Rajasthan and Madhya Pradesh, collecting data from on ground.

Collaborating with its partner, the Centre for Development of Advanced Computing (C-DAC), Tech Mahindra addressed Project Indus' computational needs. Malhotra noted that the beta phase utilised 48 graphic processing units (GPUs), a notably smaller number compared to OpenAI's ChatGPT, which employed tens of thousands of GPUs for training. This difference is attributed to the smaller scale of the Indus model, as clarified by Malhotra.

He, however, said that the team is experimenting with techniques to bring done to optimise computation. “One of the things we are trying out is a quantum inspired concept called Tensor Networks. If successful, this technique can significantly bring down the computational costs as we plan to roll out bigger models in future.”

Tech Mahindra's decision to adopt a foundational language model, particularly tailored for Hindi and its dialects, stems from its capacity to multitask and cater to specific business needs, especially in sectors like rural finance, agritech, and media and entertainment, offering advantages in proprietary models for diverse purposes. As part of the larger Mahindra Group, the Hindi-based model not only benefits the Indian ecosystem but opens doors to international markets – “We have already received interest in building a similar model for Indonesia Bahasa,” said Malhotra. This strategic move positions Tech Mahindra to efficiently address varied customer needs and expand its global reach.

The race for Large Language Models (LLMs) is expected to intensify, with many players releasing their versions, fostering innovation in the Indian ecosystem, predicts Malhotra. This surge in linguistic experimentation will lead to diverse applications, exploring various languages and use cases. 

To be sure, there have been a spurt of Indic language models. Lighteningspeed Ventures backed Sarvam AI open sourced its Hindi language model called OpenHathi in December. Unlike Tech Mahindra’s Indus, this model is fine tuned on the Llama model from Meta. In the same month, Bhavish Aggrawal-led Ola Group also announced a base language model, Krutrim, built for 22 Indic languages.