# Less is More for Long Document Summary Evaluation by LLMs

**Authors:** Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, Estevam Hruschka  
**arXiv:** [2309.07382](https://arxiv.org/abs/2309.07382)  
**Date:** 2023-09-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.07382v1/

## Key points

- Authors address challenges faced by Large Language Models (LLMs) in summary evaluation tasks
- Challenges include high computational costs and Lost-in-the-Middle problem
- Proposed solution: Extract-then-Evaluate approach
- Approach involves extracting key sentences from lengthy source document and evaluating the summary using LLMs
- Results show significant reduction in evaluation costs and higher correlation with human evaluations
- Practical recommendations provided for optimal document length and effective sentence extraction methods
- Research contributes to cost-effective and accurate evaluation methods for text generation using LLMs

## Summary

In their paper titled "Less is More for Long Document Summary Evaluation by LLMs," authors Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, and Estevam Hruschka address the challenges faced by Large Language Models (LLMs) in summary evaluation tasks. These challenges include high computational costs and the Lost-in-the-Middle problem where crucial information in lengthy documents is often overlooked. To overcome these obstacles, the authors propose a novel approach called Extract-then-Evaluate. This method involves extracting key sentences from a lengthy source document and then evaluating the summary using LLMs. The results of their study demonstrate that this approach not only significantly reduces evaluation costs but also shows a higher correlation with human evaluations. Furthermore, the authors provide practical recommendations for determining optimal document length and implementing effective sentence extraction methods. These insights contribute to the development of cost-effective yet more accurate methods for evaluating text generation using LLMs. Overall, this research highlights the importance of refining evaluation processes for LLM-based text generation by focusing on extracting key information from long documents and optimizing evaluation strategies.