# QuALITY: Question Answering with Long Input Texts, Yes!

**Authors:** Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, Samuel R. Bowman  
**arXiv:** [2112.08608](https://arxiv.org/abs/2112.08608)  
**Date:** 2021-12-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2112.08608v1/

## Key points

- Introduction of a new multiple-choice QA dataset called QuALITY
- Enables building and testing models on long-document comprehension
- Context passages in English have an average length of about 5,000 tokens
- Questions are written and validated by contributors who have read the entire passage
- Only half of the questions are answerable by annotators working under tight time constraints
- Current models perform poorly on this task (55.4%)
- Significantly lag behind human performance (93.5%)
- Authors include Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He and Samuel R. Bowman

## Summary

Introduce a new multiple-choice QA dataset called QuALITY to enable building and testing models on long-document comprehension. The context passages in English have an average length of about 5,000 tokens, which is much longer than typical current models can process. Unlike prior work with passages, the questions are written and validated by contributors who have read the entire passage instead of relying on summaries or excerpts. The authors found that only half of the questions are answerable by annotators working under tight time constraints, indicating that skimming and simple search are not enough to consistently perform well. Current models perform poorly on this task (55.4%) and significantly lag behind human performance (93.5%). The paper's authors include Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He and Samuel R. Bowman.