# Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU

**Authors:** Changyue Liao, Mo Sun, Zihan Yang, Kaiqi Chen, Binhang Yuan, Fei Wu, Zeke Wang  
**arXiv:** [2403.06504](https://arxiv.org/abs/2403.06504)  
**Date:** 2024-03-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.06504v1/

## Key points

- Authors: Changyue Liao, Mo Sun, Zihan Yang, Kaiqi Chen, Binhang Yuan, Fei Wu, Zeke Wang
- Challenge: Large language models require vast parameters for optimal performance
- Proposed Solution: Incorporating SSD-CPU communication to enable efficient fine-tuning of 100B models on low-end servers with modest GPU and CPU memory capacities
- Maximizes GPU utilization
- Overcomes hardware limitations for training massive language models
- Achieved high TFLOPS on consumer-grade GPUs compared to ZeRO-Infinity
- Impact: Provides a novel and cost-effective solution for AI researchers with limited budgets to efficiently fine-tune massive language models on commodity servers.

## Summary

In their paper titled "Adding NVMe SSDs to Enable and Accelerate 100B Model Fine-tuning on a Single GPU," authors Changyue Liao, Mo Sun, Zihan Yang, Kaiqi Chen, Binhang Yuan, Fei Wu, and Zeke Wang address the challenges posed by large language models that require vast amounts of parameters for optimal performance. The authors propose a solution that enables efficient fine-tuning of 100B models on low-end servers with modest GPU and CPU memory capacities by incorporating SSD-CPU communication as an optimization dimension. This approach maximizes GPU utilization and overcomes hardware limitations for training massive language models. Experimental results demonstrate the effectiveness of this approach in achieving high TFLOPS on consumer-grade GPUs compared to state-of-the-art solutions like ZeRO-Infinity. Overall, this research presents a novel and cost-effective solution for AI researchers with limited budgets to efficiently fine-tune massive language models on commodity servers.