# HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace

**Authors:** Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang  
**arXiv:** [2303.17580](https://arxiv.org/abs/2303.17580)  
**Date:** 2023-03-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.17580v2/

## Key points

- Solving complex tasks involving different domains and modalities is crucial for advanced AI capabilities
- Existing AI models often struggle with complicated tasks
- Large language models (LLMs) can be used as controllers to manage existing AI models and solve complex tasks
- Language can act as a generic interface to empower LLMs in this role
- HuggingGPT is a framework that leverages LLMs like ChatGPT to connect various AI models in machine learning communities like Hugging Face
- The framework uses ChatGPT to conduct task planning, selects appropriate AI models based on their function descriptions available in Hugging Face, executes each subtask with the selected model, and summarizes the response according to the execution results
- HuggingGPT can tackle numerous sophisticated AI tasks across different modalities and domains by utilizing ChatGPT's strong language capabilities and abundant AI models available in Hugging Face
- The framework achieves impressive results in challenging areas such as language understanding, generation, interaction, reasoning, vision, speech recognition and other complex tasks
- This approach enables LLMs to act as controllers for managing existing AI models effectively and paves the way towards advanced artificial intelligence.

## Summary

In the field of artificial intelligence (AI), solving complex tasks that involve different domains and modalities is a crucial step towards achieving advanced AI capabilities. While there are numerous AI models available for various domains and modalities, they often struggle to handle complicated tasks. To address this issue, researchers propose using large language models (LLMs) as controllers to manage existing AI models and solve complex tasks. They suggest that language could act as a generic interface to empower LLMs in this role. To put this philosophy into practice, the researchers introduce HuggingGPT, a framework that leverages LLMs such as ChatGPT to connect various AI models in machine learning communities like Hugging Face. The framework uses ChatGPT to conduct task planning when it receives a user request, selects appropriate AI models based on their function descriptions available in Hugging Face, executes each subtask with the selected model, and summarizes the response according to the execution results. By utilizing ChatGPT's strong language capabilities and abundant AI models available in Hugging Face, HuggingGPT can tackle numerous sophisticated AI tasks across different modalities and domains. The framework achieves impressive results in challenging areas such as language understanding, generation, interaction, reasoning, vision, speech recognition and other complex tasks. This approach paves the way towards advanced artificial intelligence by enabling LLMs to act as controllers for managing existing AI models effectively.