# A Survey on Multimodal Large Language Models

**Authors:** Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, Enhong Chen  
**arXiv:** [2306.13549](https://arxiv.org/abs/2306.13549)  
**Date:** 2023-06-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.13549v1/

## Key points

- Multimodal Large Language Models (MLLM) utilize powerful Large Language Models (LLMs) for multimodal tasks
- MLLM has remarkable capabilities such as generating stories based on images and performing math reasoning without OCR
- MLLM has the potential to pave the way towards artificial general intelligence
- The survey provides an overview of recent advancements in MLLM
- Key techniques and applications discussed include Multimodal Instruction Tuning (M-IT), Multimodal In-Context Learning (M-ICL), Multimodal Chain of Thought (M-CoT), and LLM-Aided Visual Reasoning (LAVR)
- Existing challenges in MLLM are addressed, and promising research directions are identified
- The authors intend to continuously update the survey to inspire further research
- A GitHub link is provided for access to the latest papers related to MLLM

## Summary

The paper titled "A Survey on Multimodal Large Language Models" explores the emerging research field of Multimodal Large Language Models (MLLM). MLLM utilizes powerful Large Language Models (LLMs) as a cognitive tool to perform various multimodal tasks. The authors highlight the remarkable capabilities of MLLM, such as generating stories based on images and performing math reasoning without OCR, which are not commonly seen in traditional methods. These capabilities suggest that MLLM has the potential to pave the way towards artificial general intelligence. The main objective of this survey is to provide an overview of the recent advancements in MLLM. The authors begin by presenting the formulation of MLLM and explaining its related concepts. They then delve into discussing key techniques and applications within this field, including Multimodal Instruction Tuning (M-IT), Multimodal In-Context Learning (M-ICL), Multimodal Chain of Thought (M-CoT), and LLM-Aided Visual Reasoning (LAVR). Furthermore, the paper addresses existing challenges in MLLM and identifies promising research directions. As this is still an early stage for MLLM, the authors express their intention to continuously update this survey to inspire further research. They provide a GitHub link that collects the latest papers related to MLLM. Overall, this comprehensive survey provides valuable insights into the progress made in MLLM and highlights its potential impact on future research in artificial intelligence.