# Using Large Language Models to Enhance Programming Error Messages

**Authors:** Juho Leinonen, Arto Hellas, Sami Sarsa, Brent Reeves, Paul Denny, James Prather, Brett A. Becker  
**arXiv:** [2210.11630](https://arxiv.org/abs/2210.11630)  
**Date:** 2022-10-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.11630v1/

## Key points

- Understanding and interpreting programming error messages is a challenge
- Error messages can be difficult to understand, especially for novice programmers
- Error messages are often designed for an audience with programming knowledge
- Researchers have been working on improving programming error messages since the 1960s
- This study leverages large language models to enhance programming error messages
- Large language models can generate explanations of errors and suggestions on how to resolve them
- Enhanced versions of error messages created by large language models sometimes outperform original messages in terms of interpretability and actionability
- Large language models have potential benefits in computing education, particularly in challenging areas for students
- Advantages of large language models include valuable enhancements to error messages
- Concerns about reliance on vast amounts of data and potential biases within datasets exist
- Future research directions suggested for further enhancing programming error messages using large language models

## Summary

The paper titled "Using Large Language Models to Enhance Programming Error Messages" explores the challenge of understanding and interpreting programming error messages. Learning to program involves not only writing code but also being able to identify and fix errors. However, error messages can be difficult to understand, especially for novice programmers who may not have a strong background in programming. One reason for this difficulty is that error messages are often designed for an audience that already has programming knowledge or for specific programming environments that use the information to highlight problematic areas in the code. As a result, these messages may not provide clear explanations of the errors or offer helpful suggestions on how to fix them. Researchers have been working on improving the accessibility of programming error messages since the 1960s, but progress has been slow. In this study, the authors contribute to this ongoing research by leveraging large language models to enhance programming error messages. These models are capable of generating explanations of errors and providing suggestions on how to resolve them. The results of their work demonstrate that large language models can create enhanced and novice-friendly versions of programming error messages that sometimes outperform the original messages in terms of interpretability and actionability. This finding highlights the potential benefits of using large language models in computing education, particularly in areas known to be challenging for students. The paper also discusses both the advantages and disadvantages associated with large language models. While they offer valuable enhancements to error messages, there are concerns about their reliance on vast amounts of data and potential biases present within those datasets. The authors suggest future research directions for further enhancing programming error messages using these models. Overall, this study contributes to addressing a long-standing issue in computer science education by leveraging large language models to improve the understandability and usability of programming error messages.