# DynaServe: Unified and Elastic Execution for Dynamic Disaggregated LLM Serving

**Authors:** Chaoyi Ruan, Yinhe Chen, Dongqi Tian, Yandong Shi, Yongji Wu, Jialin Li, Cheng Li  
**arXiv:** [2504.09285](https://arxiv.org/abs/2504.09285)  
**Date:** 2025-04-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2504.09285v2/

## Key points

- DynaServe is a high-performance LLM serving system designed to meet strict latency SLOs while maximizing goodput.
- It utilizes a micro-request abstraction that splits each request into at most two cooperating segments at any token boundary for efficient processing of unbalanced and dynamic workloads.
- A two-level scheduling framework balances the load of micro-requests across unified GPU instances, with global and local schedulers adjusting batch compositions based on workload fluctuations and GPU utilization.
- In real-world traces, DynaServe boosts overall serving capacity by up to 3.07 times, improves goodput by up to 1.91 times, and enhances performance by up to 60% in hybrid workloads under SLO constraints compared to baselines.
- The innovative approach of micro-request segmentation and intelligent scheduling mechanisms enables DynaServe to achieve remarkable improvements in both throughput and latency metrics for demanding LLM inference tasks.

## Summary

, , , , 

DynaServe is a high-performance LLM serving system designed to meet strict latency SLOs while maximizing goodput. It addresses challenges posed by real-world variability in prompt and response lengths, which can skew traditional compute-intensive prefill and memory-bound decode phases. To overcome these challenges, DynaServe utilizes a micro-request abstraction that splits each request into at most two cooperating segments at any token boundary. This approach allows for more efficient processing of unbalanced and dynamic workloads. A two-level scheduling framework balances the load of micro-requests across unified GPU instances, with a global scheduler rapidly selecting per-request split points based on the request's prefill/decode time ratio and current GPU instance loads. Local schedulers on each GPU instance independently form SLO-aware batches, adjusting their composition in response to workload fluctuations, potential latency spikes, and per-GPU under/overutilization. In real-world traces, DynaServe significantly boosts overall serving capacity by up to 3.07 times, improves goodput by up to 1.91 times, and enhances performance by up to 60% in hybrid workloads under SLO constraints compared to state-of-the-art colocated and disaggregated baselines. This innovative approach demonstrates how unified and elastic execution can effectively handle dynamic LLM serving requirements while maintaining optimal performance levels. through micro-request segmentation and intelligent scheduling mechanisms enables DynaServe to achieve remarkable improvements in both throughput and latency metrics, making it a promising solution for demanding LLM inference tasks.