# Perseus: Removing Energy Bloat from Large Model Training

**Authors:** Jae-Won Chung, Yile Gu, Insu Jang, Luoxi Meng, Nikhil Bansal, Mosharaf Chowdhury  
**arXiv:** [2312.06902](https://arxiv.org/abs/2312.06902)  
**Date:** 2023-12-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.06902v1/

## Key points

- Large AI model training on multiple GPUs consumes a substantial amount of energy.
- Not all energy used in the training process directly contributes to enhancing training throughput.
- A study identified two sources of energy bloat in large model training: intrinsic and extrinsic factors.
- Perseus is an innovative solution designed to address these energy bloat issues.
- Perseus uses an iterative graph cut-based algorithm to optimize energy consumption and schedule computations efficiently.
- Evaluations on models like GPT-3 and Bloom showed that Perseus could reduce overall energy consumption by up to 30%.
- Authors of this work include Jae-Won Chung, Yile Gu, Insu Jang, Luoxi Meng, Nikhil Bansal, and Mosharaf Chowdhury.

## Summary

In the realm of training large AI models on multiple GPUs, the consumption of a substantial amount of energy is a prevalent issue. It has been observed that not all energy utilized during the training process directly contributes to enhancing end-to-end training throughput. A significant portion of this energy can be deemed unnecessary and removed without impeding the pace of training. To address this challenge, a comprehensive study was conducted to pinpoint two distinct sources of energy bloat in large model training: intrinsic and extrinsic factors. In response to these findings, an innovative solution named Perseus was introduced. <Organization>Perseus</Organization> serves as a unified optimization framework designed to effectively mitigate both intrinsic and extrinsic sources of energy bloat in large model training scenarios. By leveraging an efficient iterative graph cut-based algorithm, Perseus is able to determine the "iteration time-energy" Pareto frontier for any given large model training job. Furthermore, it strategically schedules the energy consumption associated with forward and backward computations over time to eliminate intrinsic and extrinsic energy bloat. The efficacy of Perseus was put to the test through rigorous evaluations on prominent large models such as GPT-3 and Bloom. The results were nothing short of impressive, showcasing that Perseus could potentially reduce the overall energy consumption during large model training by up to 30%. This substantial reduction not only signifies significant cost savings but also opens up new possibilities that were previously unattainable in optimizing energy efficiency within AI model training processes. The authors behind this groundbreaking work include Jae-Won Chung, Yile Gu, Insu Jang, Luoxi Meng, Nikhil Bansal, and Mosharaf Chowdhury. Their collective efforts have culminated in the development of Perseus as a cutting-edge solution for removing energy bloat from large model training operations. The open-source availability of Perseus further underscores its potential impact and accessibility within the broader research community.