Learning Compiler Pass Orders using Coreset and Normalized Value Prediction

AI-generated keywords: Compilation Pass Sequence Code-Size Reduction Graph Neural Network (GNN) Submodular Function

AI-generated Key Points

  • Proposed pipeline for finding program-dependent pass sequences to optimize code-size reduction tasks
  • Optimal pass sequence can significantly reduce program size and/or improve program efficiency
  • Prior works on compilation pass ordering have two major drawbacks: excessive budget or fail to generalize to unseen programs
  • Pipeline identifies a coreset of 50 pass sequences via greedy optimization of a submodular function
  • Learns a policy with Graph Neural Network (GNN) to pick the optimal sequence by predicting normalized values of the pass sequences in the coreset
  • Outperforms default -Oz flag by an average of 4.7% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets
  • Proposed technique transforms raw action space into a small one with denser rewards and improves existing human-designed compiler flags
  • Hyperparameters such as temperature T, number of layers, embedding dimension, and output dimension are searched over for each method
  • Best model is selected based on validation results using mean squared loss
  • Effective approach for optimizing code-size reduction tasks that outperforms existing techniques while being simple and efficient
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Youwei Liang, Kevin Stone, Ali Shameli, Chris Cummins, Mostafa Elhoushi, Jiadong Guo, Benoit Steiner, Xiaomeng Yang, Pengtao Xie, Hugh Leather, Yuandong Tian

License: CC BY 4.0

Abstract: Finding the optimal pass sequence of compilation can lead to a significant reduction in program size and/or improvement in program efficiency. Prior works on compilation pass ordering have two major drawbacks. They either require an excessive budget (in terms of compilation steps) at compile time or fail to generalize to unseen programs. In this paper, for code-size reduction tasks, we propose a novel pipeline to find program-dependent pass sequences within 45 compilation calls. It first identifies a coreset of 50 pass sequences via greedy optimization of a submodular function, and then learns a policy with Graph Neural Network (GNN) to pick the optimal sequence by predicting the normalized values of the pass sequences in the coreset. Despite its simplicity, our pipeline outperforms the default -Oz flag by an average of 4.7% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets. In comparison, previous approaches like reinforcement learning on the raw pass sequence space may take days to train due to sparse reward, and may not generalize well in held-out ones from different domains. Our results demonstrate that existing human-designed compiler flags can be improved with a simple yet effective technique that transforms the raw action space into a small one with denser rewards.

Submitted to arXiv on 09 Jan. 2023

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2301.05104v2

This paper proposes a novel pipeline for finding program-dependent pass sequences to optimize code-size reduction tasks. The optimal pass sequence of compilation can lead to a significant reduction in program size and/or improvement in program efficiency. However, prior works on compilation pass ordering have two major drawbacks: they either require an excessive budget (in terms of compilation steps) at compile time or fail to generalize to unseen programs. The proposed pipeline first identifies a coreset of 50 pass sequences via greedy optimization of a submodular function. Then, it learns a policy with Graph Neural Network (GNN) to pick the optimal sequence by predicting the normalized values of the pass sequences in the coreset. Despite its simplicity, this pipeline outperforms the default -Oz flag by an average of 4.7% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets. Previous approaches like reinforcement learning on the raw pass sequence space may take days to train due to sparse reward and may not generalize well in held-out ones from different domains. In contrast, this proposed technique transforms the raw action space into a small one with denser rewards and improves existing human-designed compiler flags. In experiments, hyper-parameters such as temperature T in Eq. 3, number of layers, embedding dimension of node/edge features, and output dimension in hidden layers in MLPs are searched over for each method. The best model is selected based on validation results using mean squared loss. Overall, this paper presents an effective approach for optimizing code-size reduction tasks that outperforms existing techniques while being simple and efficient.
Created on 24 Apr. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.