# IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization

**Authors:** Xinghua Zhang, Haiyang Yu, Cheng Fu, Fei Huang, Yongbin Li  
**arXiv:** [2411.06208](https://arxiv.org/abs/2411.06208)  
**Date:** 2024-11-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2411.06208v3/

## Key points

- Large Language Models (LLMs) require the ability to accurately follow complex instructions for various tasks
- TRACE benchmark introduced to enhance and evaluate LLMs' instruction-following capabilities
- TRACE includes 120K training data points and 1K evaluation data points for comprehensive testing
- IOPO alignment method proposed to improve how LLMs align with response preferences and explore instruction preferences in detail
- Extensive experiments show significant improvements with IOPO, including 8.15% and 2.18% enhancements on in-domain data, as well as 6.29% and 3.13% improvements on out-of-domain data compared to existing methods
- Research contributes valuable insights into empowering LLMs with enhanced complex instruction following abilities through innovative benchmarking and alignment techniques like TRACE and IOPO

## Summary

In the realm of Large Language Models (LLMs), the ability to accurately follow complex instructions is becoming increasingly important as more agents and applications rely on LLMs for various tasks. With the complexity of instructions rapidly evolving, there is a growing need to enhance the instruction-following capabilities of these models. To address this gap, this paper introduces TRACE, a benchmark designed to improve and evaluate the ability of LLMs to follow complex instructions. The benchmark includes 120K training data points and 1K evaluation data points for comprehensive testing. Additionally, the paper proposes the IOPO (Input-Output Preference Optimization) alignment method, which considers both input and output preference pairs in order to enhance how LLMs align with response preferences and explore instruction preferences in detail. Extensive experiments conducted on both in-domain and out-of-domain datasets demonstrate the effectiveness of IOPO. The results show significant improvements compared to existing methods, with 8.15% and 2.18% enhancements on in-domain data, as well as 6.29% and 3.13% improvements on out-of-domain data when compared to SFT and DPO alignment methods respectively. Overall, this research contributes valuable insights into empowering LLMs with enhanced complex instruction following abilities through innovative benchmarking and alignment techniques like TRACE and IOPO. These advancements have the potential to significantly improve the performance of LLMs in handling intricate tasks based on complex instructions in various real-world applications.