# Data Augmentation Approaches for Source Code Models: A Survey

**Authors:** Terry Yue Zhuo, Zhou Yang, Zhensu Sun, Yufei Wang, Li Li, Xiaoning Du, Zhenchang Xing, David Lo  
**arXiv:** [2305.19915](https://arxiv.org/abs/2305.19915)  
**Date:** 2023-05-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.19915v1/

## Key points

- The paper presents a survey of data augmentation (DA) techniques for source code models.
- DA methods are used to enhance training data and improve the robustness and generalizability of these models.
- A taxonomy of DA for source code model approaches is constructed, followed by a discussion on prominent, methodologically illustrative approaches.
- The authors highlight the general strategies and techniques to optimize the quality of DA, underscore techniques that find utility in widely-accepted source code scenarios and downstream tasks, and outline the prevailing challenges and potential opportunities for future research.
- Specific areas such as Code Summarization, Code Search, and Code Completion are discussed with examples of how DA can be applied in each area.
- The paper provides a valuable collection of general-purpose DA techniques for source code models and serves as an inspiration for further research in this area.
- The authors also provide a continually updated GitHub repository that hosts a list of up-to-date papers on DA for source code models.

## Summary

This paper presents a comprehensive and integrative survey of data augmentation (DA) techniques for source code models. The increasing popularity of source code adoption in critical tasks has motivated the development of DA methods to enhance training data and improve various capabilities such as robustness and generalizability of these models. While several DA methods have been proposed and tailored for source code models, there lacks a comprehensive examination to understand their effectiveness and implications. The paper starts by constructing a taxonomy of DA for source code model approaches, followed by a discussion on prominent, methodologically illustrative approaches. The authors highlight the general strategies and techniques to optimize the quality of DA, underscore techniques that find utility in widely-accepted source code scenarios and downstream tasks, and outline the prevailing challenges and potential opportunities for future research. The authors also present specific areas such as Code Summarization where they discuss how MHM is applied to perturb training examples to improve adversarial training while generating new summaries using retrieval-augmentation frameworks based on similar code-summary pairs. In Code Search, they discuss how soft data augmentation (SoDa) can be used instead of rule-based techniques to manipulate input data representation while still predicting tokens based on dynamic masking or replacement when processing CodeSearchNet. In Code Completion, they suggest that generative source code models are vulnerable to adversarial examples that are perturbed with transformation rules; thus they propose customized transformations specifically for docstrings, function/variable names, syntax/formatting on datasets like PY150 and GitHub Java Corpus. In essence, this paper provides a valuable collection of general-purpose DA techniques for source code models and serves as an inspiration for further research in this area. The authors also provide a continually updated GitHub repository that hosts a list of up-to-date papers on DA for source code models.