# RLTF: Reinforcement Learning from Unit Test Feedback

**Authors:** Jiate Liu, Yiqin Zhu, Kaiwen Xiao, Qiang Fu, Xiao Han, Wei Yang, Deheng Ye  
**arXiv:** [2307.04349](https://arxiv.org/abs/2307.04349)  
**Date:** 2023-07-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.04349v1/

## Key points

- Goal of program synthesis or code generation is to generate executable code based on given descriptions
- Growing interest in using reinforcement learning (RL) to improve performance of large language models (LLMs) for code
- RLTF (Reinforcement Learning from Unit Test Feedback) proposed as a novel online RL framework
- RLTF generates data in real-time during training and utilizes fine-grained feedback signals from unit tests
- Overcomes limitations of previous methods by incorporating online reinforcement learning with unit test feedback
- Achieves state-of-the-art performance on benchmarks such as APPS and MBPP
- Extensive experimental results provided to demonstrate effectiveness of RLTF
- Code available on GitHub for further exploration and replication of findings

## Summary

The goal of program synthesis, or code generation, is to generate executable code based on given descriptions. In recent years, there has been a growing interest in using reinforcement learning (RL) to improve the performance of large language models (LLMs) for code. To address the limitations of existing RL methods for code generation that only utilize offline frameworks and do not take into account specific error locations within the generated code, a team of researchers proposed RLTF (Reinforcement Learning from Unit Test Feedback), a novel online RL framework. Unlike offline frameworks, RLTF generates data in real-time during training and utilizes fine-grained feedback signals from unit tests to guide the model towards producing higher-quality code. This approach enables RLTF to overcome the limitations of previous methods and achieve state-of-the-art performance on benchmarks such as APPS and MBPP. The authors provide extensive experimental results to demonstrate the effectiveness of RLTF and make their code available on GitHub for further exploration and replication of their findings. Overall, RLTF offers a promising solution for improving the performance of large language models in generating high-quality executable code by incorporating online reinforcement learning with unit test feedback.