# Small Language Models are the Future of Agentic AI

**Authors:** Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, Pavlo Molchanov  
**arXiv:** [2506.02153](https://arxiv.org/abs/2506.02153)  
**Date:** 2025-06-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2506.02153v1/

## Key points

- Debate between large language models (LLMs) and small language models (SLMs) in the evolving landscape of agentic AI systems
- Shift towards specialized tasks with repetitive functions in agentic AI systems
- Argument that SLMs are powerful, suitable, and cost-effective for many applications in agentic systems
- Detailed process outlined from data curation and filtering to SLM selection, specialized SLM fine-tuning, and continuous iteration and refinement
- Importance of embracing SLMs in agentic AI systems for efficiency, cost reduction, and enhanced performance in specialized task-oriented applications

## Summary

The evolving landscape of agentic AI systems is sparking a debate between large language models (LLMs) and small language models (SLMs). While LLMs are praised for their near-human performance and conversational abilities, the rise of agentic AI systems calls for a shift towards specialized tasks with repetitive functions. This has led to the argument that SLMs are not only powerful but also more suitable and cost-effective for many applications in agentic systems, making them the future of agentic AI. To support this stance, a detailed process is outlined from data curation and filtering (S2) to SLM selection (S4), specialized SLM fine-tuning (S5), and continuous iteration and refinement (S6). The process involves collecting data, clustering tasks, selecting appropriate SLMs based on criteria such as capabilities and performance benchmarks, fine-tuning them with task-specific datasets, and continuously refining the models with new data to adapt to changing patterns. The authors highlight the transformative potential of agentic AI in white-collar work and beyond, emphasizing the importance of cost savings and sustainability in AI infrastructure. They invite contributions and critiques on their position via email at agents@nvidia.com and commit to publishing all correspondence on their website. Overall, this comprehensive summary presents a strong case for embracing SLMs in agentic AI systems to drive efficiency, reduce costs, and enhance performance in specialized task-oriented applications.