# Gorilla: Large Language Model Connected with Massive APIs

**Authors:** Shishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. Gonzalez  
**arXiv:** [2305.15334](https://arxiv.org/abs/2305.15334)  
**Date:** 2023-05-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.15334v1/

## Key points

- Large Language Models (LLMs) have made significant advancements in various tasks
- LLMs struggle with generating accurate input arguments and often hallucinate incorrect API call usage
- Gorilla is a finetuned LLaMA-based model that outperforms GPT-4 in writing API calls
- Gorilla demonstrates strong adaptability to test-time document changes when combined with a document retriever
- Evaluation of Gorilla's performance is done using the APIBench dataset consisting of HuggingFace, TorchHub, and TensorHub APIs
- Successful integration of the retrieval system with Gorilla improves the reliability and applicability of LLM outputs
- Availability of Gorilla's code, model, data, and demo at https://gorilla.cs.berkeley.edu facilitates its adoption
- Empowering LLMs to use tools through APIs enhances their capabilities and interaction with tools in various domains
- The proposed pipeline for finetuning LLMs to call APIs surpasses GPT-4's performance in massive datasets collected from companies like Samsung SDS, Uber, and VMware

## Summary

Large Language Models (LLMs) have made significant advancements in various tasks, such as mathematical reasoning and program synthesis. However, their ability to effectively use tools via API calls remains a challenge. Even state-of-the-art LLMs like GPT-4 struggle with generating accurate input arguments and often hallucinate incorrect API call usage. To address this issue, the researchers introduce Gorilla, a finetuned LLaMA-based model that outperforms GPT-4 in writing API calls. Gorilla demonstrates strong adaptability to test-time document changes when combined with a document retriever. This enables flexible user updates or version changes and mitigates the problem of hallucination commonly encountered when directly prompting LLMs. To evaluate Gorilla's performance, the researchers introduce APIBench, a comprehensive dataset consisting of HuggingFace, TorchHub, and TensorHub APIs. The successful integration of the retrieval system with Gorilla showcases the potential for LLMs to accurately use tools, keep up with frequently updated documentation, and improve the reliability and applicability of their outputs. The availability of Gorilla's code, model, data, and demo at https://gorilla.cs.berkeley.edu further facilitates its adoption. The paper also highlights the importance of empowering LLMs to use tools for accessing larger and changing knowledge bases and accomplishing complex computational tasks. By integrating plugins that allow LLMs to invoke external tools through APIs, they can become primary interfaces to computing infrastructure and the web. Techniques like Gorilla enhance an LLM's ability to identify appropriate APIs for specific tasks; correct usage of APIs improves an LLM's interaction with tools in various domains. The proposed pipeline for finetuning LLMs to call APIs surpasses GPT-4's performance in massive datasets collected by the researchers from companies like Samsung SDS, Uber, and VMware. Overall, these advancements contribute to expanding the capabilities of LLMs and their potential to revolutionize various industries by providing them with powerful interfaces that enable them to access large knowledge bases quickly while remaining reliable even when faced with frequent updates or version changes.