Small Language Models: The next frontier in enterprise AI

In the not-so-distant past, natural language processing (NLP) was regarded too intricate for modern artificial intelligence (AI). However, the landscape changed dramatically in November 2022 when OpenAI introduced ChatGPT. Within a week, it garnered more than a million users, making AI accessible to the masses and marking the beginning of a loud, public, and costly Generative AI (GenAI) revolution. 

The rapid evolution of AI capabilities over the past few years has been driven by advances in large language models (LLMs). While LLMs offer impressive capabilities, their massive size leads to efficiency, cost, and customisability challenges. This has paved the way for the rise of small language models (SLMs). 

SLMs are more streamlined versions of LLMs, featuring fewer parameters and simpler designs. They have become attractive to enterprises due to their control, cost-efficiency, and ability to fine-tune for specific domains and ensure data security. 

“Applications like ChatGPT based on LLMs created a frenzy about the possibilities of new age AI. However, as enterprises started to rush towards adapting it in their different product and services, they faced the business reality of cost vs revenue impact. In their quest to reduce cost while maintaining the same or acceptable quality of output, they came across SLMs,” said Ganesh Sahai, CTO of Nagarro. 

Recognising the trend, companies such as Apple, Microsoft, Meta, and Google are now focusing on developing smaller AI models with fewer parameters while maintaining robust capabilities. 

The Shift Towards Smaller Models 

Several factors drive the shift towards smaller models, primarily to address concerns surrounding the adoption of LLMs. In April, Microsoft introduced Phi-3, a family of open AI models that are capable and cost-effective small language models. Hugging Face also launched SmolLM, a new series of compact language models optimised for use on local devices like laptops and phones, eliminating the need for cloud-based resources and significantly reducing energy consumption. 
Just two months after introducing GPT-4o, OpenAI unveiled GPT-4o mini, a cost-efficient small language model. Nvidia and French startup Mistral AI also announced Mistral-NeMo, a small language model designed to bring powerful AI capabilities directly to business desktops. 

The Benefits of Small Language Models 

Language models are AI systems trained on large text datasets, enabling text generation, document summarisation, language translation, and question answering. Small language models fill the same niche but with notably smaller model sizes, typically under 100 million parameters, with some models as small as 1 million parameters.

For example, Meta’s Llama 3 boasts an 8 billion parameter model that rivals larger models like GPT-4. Similarly, Microsoft’s Phi-3-small model, with 7 billion parameters, outperforms previous versions of OpenAI’s model. The smaller sizes make SLMs more efficient, economical, and customisable than their larger counterparts. 

One significant advantage of SLMs is their ability to run on devices with limited processing power, such as smartphones or IoT devices. This edge computing capability contrasts sharply with larger models requiring powerful cloud infrastructure, making SLMs accessible to entrepreneurs and smaller organisations. 

“With SLMs, AI capabilities can be democratised, allowing smaller organisations to leverage powerful NLP tools without massive computational resources. SLMs, with their smaller size and lower computational requirements, are easier to secure and control, minimising the risk of misuse or unauthorised access,” said Pawan Prabhat, Co-Founder of Shorthills AI. 

Challenges with Large Language Models

LLMs have become a major force in the enterprise sector, excelling in data processing, summarising, and analysis, offering valuable insights for decision-making. They also create compelling content and translate foreign languages. However, LLMs have significant disadvantages, including accuracy issues, model bias, and “hallucinations” where the model generates factually incorrect or nonsensical information. 

LLMs can be too generalised due to training on public internet data, leading to gaps in understanding industry-specific jargon, processes, and data nuances. “There are scenarios where we have certain focused areas of operation and corresponding use cases that do not need general-purpose LLMs. We have enterprise-level scenarios needing intelligent bots for internal policies or operational data insights. SLMs fulfill this need well while delivering services on low budgets,” said Sahai. 

This year’s AI Index, an independent initiative at Stanford University claimed OpenAI’s GPT-4 used an estimated $78 million worth of compute to train, while Google’s Gemini Ultra cost $191 million for compute. This marks a dramatic increase from previous years, the report estimates it cost Google only $12 million to train its PaLM model in 2022. 

The Future of Small Language Models 

As AI adoption grows, and tech giants push AI into various offerings, the cost of AI is likely to continue rising. Despite challenges, SLMs represent a significant advancement in AI, offering efficiency and versatility while challenging the dominance of LLMs. These models redefine computational norms with reduced costs and streamlined architectures, proving that size is not the sole determinant of proficiency. Ongoing research and collaborative efforts continue to enhance SLM performance, indicating a promising future for these models in enterprise applications. 

In conclusion, SLMs offer a practical and economical alternative to LLMs, providing significant advantages for enterprises looking to integrate AI into their operations efficiently. As technology evolves, these smaller models are set to play a crucial role in the future of AI.