# StructCoder: Structure-Aware Transformer for Code Generation

**Authors:** Sindhu Tipirneni, Ming Zhu, Chandan K. Reddy  
**arXiv:** [2206.05239](https://arxiv.org/abs/2206.05239)  
**Date:** 2022-06-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.05239v1/

## Key points

- The paper addresses automating software engineering tasks using deep learning
- Specifically focuses on code generation from source code in a different language or natural language
- Emphasizes the need for a comprehensive understanding of code syntax and semantics
- Proposes an encoder-decoder Transformer model called StructCoder
- Encoder and decoder components are trained to recognize syntax and data flow in source and target codes respectively
- Encoder is structure-aware by leveraging syntax tree and data flow graph of source code
- Introduces two auxiliary tasks: Abstract Syntax Tree (AST) paths prediction and data flow prediction to preserve syntax and data flow in generated code
- Achieves state-of-the-art performance on various tasks within the CodeXGLUE benchmark
- Pioneering introduction of structure-aware Transformer decoder that enhances generated code quality by modeling target syntax and data flow
- Significant improvements compared to existing models, establishing StructCoder as a leading solution in this field

## Summary

The paper titled "StructCoder: Structure-Aware Transformer for Code Generation" addresses the growing interest in automating software engineering tasks using deep learning. Specifically, it focuses on the problem of code generation, where the objective is to generate target code based on source code written in a different language or described in natural language. While existing deep learning models for code generation primarily rely on training strategies designed for natural language, this work emphasizes the need for a more comprehensive understanding of code syntax and semantics. To tackle this challenge, the authors propose an encoder-decoder Transformer model called StructCoder. Both the encoder and decoder components of StructCoder are trained to recognize the syntax and data flow in the source and target codes respectively. The encoder is made structure-aware by leveraging the syntax tree and data flow graph of the source code. Additionally, two auxiliary tasks are introduced to ensure that the decoder preserves the syntax and data flow of the target code: Abstract Syntax Tree (AST) paths prediction and data flow prediction. This research contributes to advancing automated software engineering by developing a novel approach that leverages deep learning techniques while considering both syntax and semantics during code generation. The proposed StructCoder model achieves state-of-the-art performance on various tasks such as code translation and text-to-code generation within the CodeXGLUE benchmark. Notably, this paper is pioneering in its introduction of a structure-aware Transformer decoder that enhances generated code quality by modeling target syntax and data flow. The results demonstrate significant improvements compared to existing models, establishing StructCoder as a leading solution in this field. In summary, this research makes an important contribution towards improving automated software engineering through its development of an effective approach that considers both syntax and semantics during code generation.