# Learning Compiler Pass Orders using Coreset and Normalized Value Prediction

**Authors:** Youwei Liang, Kevin Stone, Ali Shameli, Chris Cummins, Mostafa Elhoushi, Jiadong Guo, Benoit Steiner, Xiaomeng Yang, Pengtao Xie, Hugh Leather, Yuandong Tian  
**arXiv:** [2301.05104](https://arxiv.org/abs/2301.05104)  
**Date:** 2023-01-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.05104v2/

## Key points

- Proposed pipeline for finding program-dependent pass sequences to optimize code-size reduction tasks
- Optimal pass sequence can significantly reduce program size and/or improve program efficiency
- Prior works on compilation pass ordering have two major drawbacks: excessive budget or fail to generalize to unseen programs
- Pipeline identifies a coreset of 50 pass sequences via greedy optimization of a submodular function
- Learns a policy with Graph Neural Network (GNN) to pick the optimal sequence by predicting normalized values of the pass sequences in the coreset
- Outperforms default -Oz flag by an average of 4.7% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets
- Proposed technique transforms raw action space into a small one with denser rewards and improves existing human-designed compiler flags
- Hyperparameters such as temperature T, number of layers, embedding dimension, and output dimension are searched over for each method
- Best model is selected based on validation results using mean squared loss
- Effective approach for optimizing code-size reduction tasks that outperforms existing techniques while being simple and efficient

## Summary

This paper proposes a novel pipeline for finding program-dependent pass sequences to optimize code-size reduction tasks. The optimal pass sequence of compilation can lead to a significant reduction in program size and/or improvement in program efficiency. However, prior works on compilation pass ordering have two major drawbacks: they either require an excessive budget (in terms of compilation steps) at compile time or fail to generalize to unseen programs. The proposed pipeline first identifies a coreset of 50 pass sequences via greedy optimization of a submodular function. Then, it learns a policy with Graph Neural Network (GNN) to pick the optimal sequence by predicting the normalized values of the pass sequences in the coreset. Despite its simplicity, this pipeline outperforms the default -Oz flag by an average of 4.7% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets. Previous approaches like reinforcement learning on the raw pass sequence space may take days to train due to sparse reward and may not generalize well in held-out ones from different domains. In contrast, this proposed technique transforms the raw action space into a small one with denser rewards and improves existing human-designed compiler flags. In experiments, hyper-parameters such as temperature T in Eq. 3, number of layers, embedding dimension of node/edge features, and output dimension in hidden layers in MLPs are searched over for each method. The best model is selected based on validation results using mean squared loss. Overall, this paper presents an effective approach for optimizing code-size reduction tasks that outperforms existing techniques while being simple and efficient.