# Fine-tuning Language Models with Generative Adversarial Feedback

**Authors:** Zhang Ze Yu, Lau Jia Jaw, Wong Qin Jiang, Zhang Hui  
**arXiv:** [2305.06176](https://arxiv.org/abs/2305.06176)  
**Date:** 2023-05-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.06176v1/

## Key points

- Authors propose Reinforcement Learning with Generative Adversarial Feedback (RLGAF) as an alternative approach to Reinforcement Learning with Human Feedback (RLHF)
- RLHF is limited by the expertise and productivity of human evaluators
- RLGAF can effectively align LLM outputs without being hindered by the restrictions of RLHF
- Preliminary findings suggest that RLGAF can improve the performance of large language models (LLMs)
- This promising avenue for further research on automating AI alignment could have substantial implications for enhancing the accuracy and reliability of language models in various applications.

## Summary

In their paper titled "Fine-tuning Language Models with Generative Adversarial Feedback," authors Zhang Ze Yu, Lau Jia Jaw, Wong Qin Jiang and Zhang Hui explore an alternative approach to Reinforcement Learning with Human Feedback (RLHF) for improving the performance of large language models (LLMs). RLHF has been demonstrated to align LLM outputs with desired human values; however, it is limited by the expertise and productivity of human evaluators. To address this issue, the authors propose Reinforcement Learning with Generative Adversarial Feedback (RLGAF) as a solution. Their preliminary findings suggest that RLGAF can effectively align LLM outputs without being hindered by the restrictions of RLHF. This promising avenue for further research on automating AI alignment could have substantial implications for enhancing the accuracy and reliability of language models in various applications. The paper consists of 10 pages, 3 figures and 5 tables.