# Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems

**Authors:** Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, Zhihao Jia  
**arXiv:** [2312.15234](https://arxiv.org/abs/2312.15234)  
**Date:** 2023-12-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.15234v2/

## Key points

- Generative large language models (LLMs) are driving significant advancements in understanding and manipulating human languages.
- Transformer-based architectures like GPT-family, LLaMA-family, DeepSeek-family, OPT, BLOOM, Mistral, DeciLM, Baichuan, and GLM have revolutionized natural language processing (NLP) tasks.
- LLMs have expanded into diverse applications including automated programming, science discovery, personalized digital assistants, creative arts, and next-generation computing architecture.
- Computational intensity and memory consumption of deploying these models present challenges in serving efficiency due to their immense size and complexity.
- Efficient LLM serving methodologies are being explored by the research community to address concerns over energy consumption, scalability, and accessibility for broader adoption in real-world applications.

## Summary

In the rapidly evolving landscape of artificial intelligence (AI), generative large language models (LLMs) have emerged as a driving force behind significant advancements in understanding and manipulating human languages. These LLMs include transformer-based architectures such as the GPT-family, LLaMA-family, DeepSeek-family, and other latest public models like OPT, BLOOM, Mistral, DeciLM, Baichuan, and GLM. They have revolutionized natural language processing (NLP) tasks and expanded into diverse applications including automated programming, science discovery, personalized digital assistants, creative arts, and next-generation computing architecture. Despite their exceptional performance across various domains, the computational intensity and memory consumption of deploying these models present substantial challenges in terms of serving efficiency. The immense model size and complexity require extensive computational resources during inference processes for LLMs. This resource-intensive nature raises concerns over energy consumption, scalability, and accessibility for broader adoption in real-world applications. To address this critical need for efficient LLM serving methodologies from a machine learning system (MLSys) research perspective,<Organization> this survey paper provides an exhaustive exploration of multifaceted strategies proposed by the research community. From cutting-edge algorithmic modifications to groundbreaking changes in system designs such as Speculative Sampling Speculative Decoding (SpecDec), InferMedusa RESTLookahead SequoiaEAGLEOuroboros Triforce Hydra Kangaroo OPT-Tree SpecPrefill LongSpec AdaServe Sequence-based speculative decoding Tree-based speculative decoding among others. The survey aims to offer valuable insights for researchers and practitioners in overcoming the barriers of effective LLM deployment. By providing a comprehensive understanding of the current state and future directions in efficient LLM serving methods from algorithms to systems optimization,<Organization> this survey paper stands at the crux of advanced AI innovations and practical system optimizations reshaping the future of AI technology.