What are Small Language Models, and are they better than LLMs?

“We’ve achieved peak data,” former OpenAI chief scientist Ilya Sutskever said onstage at the NeurIPS conference last year. If LLMs are intentionally built to achieve Artificial General Intelligence, small language models are made for specific use cases. “Small language models are perfect for edge cases,” said Rahul Dandwate, ML engineer at Adobe. This is similar to how small language models and large language models work,” he said. In a country like India, where the scope of AI adoption is immense but resources are constrained, the diminutiveness of small language models is perfect.