# Toolformer: Language Models Can Teach Themselves to Use Tools

**Authors:** Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom  
**arXiv:** [2302.04761](https://arxiv.org/abs/2302.04761)  
**Date:** 2023-02-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.04761v1/

## Key points

- Language models (LMs) have remarkable abilities to solve new tasks with few examples or textual instructions.
- LMs struggle with basic functionality such as arithmetic or factual lookup.
- Toolformer is a model that teaches itself to use external tools via simple APIs in a self-supervised way.
- The model decides which APIs to call, when to call them, what arguments to pass and how best to incorporate the results into future token prediction.
- Various tools are incorporated into Toolformer including a calculator, Q&A system, two different search engines, translation system and calendar.
- With only a handful of demonstrations for each API required, Toolformer achieves substantially improved zero-shot performance across various downstream tasks while maintaining its core language modeling abilities.
- This approach combines the strengths of LMs with those of specialized models designed for specific tasks.
- Overall, Toolformer represents an exciting step towards developing more efficient and effective natural language processing systems that can adapt quickly and easily to new challenges.

## Summary

In recent years, language models (LMs) have demonstrated remarkable abilities to solve new tasks with just a few examples or textual instructions. However, they often struggle when it comes to basic functionality such as arithmetic or factual lookup, where simpler and smaller models tend to perform better. To address this paradox, a team of researchers has developed Toolformer - a model that teaches itself to use external tools via simple APIs in a self-supervised way. The model is trained to decide which APIs to call, when to call them, what arguments to pass and how best to incorporate the results into future token prediction. The team incorporated various tools into Toolformer including a calculator, Q&A system, two different search engines, translation system and calendar. With only a handful of demonstrations for each API required, Toolformer achieves substantially improved zero-shot performance across various downstream tasks while maintaining its core language modeling abilities. This approach offers the best of both worlds by combining the strengths of LMs with those of specialized models designed for specific tasks. Overall, Toolformer represents an exciting step towards developing more efficient and effective natural language processing systems that can adapt quickly and easily to new challenges.