# Trusting Your Evidence: Hallucinate Less with Context-aware Decoding

**Authors:** Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, Scott Wen-tau Yih  
**arXiv:** [2305.14739](https://arxiv.org/abs/2305.14739)  
**Date:** 2023-05-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.14739v1/

## Key points

- Language models struggle with paying enough attention to input context
- Context-aware decoding (CAD) is proposed as a solution
- CAD amplifies the difference between output probabilities with and without context
- CAD significantly improves the faithfulness of language models for summarization tasks
- LLaMA shows a 14.3% gain in factuality metrics with CAD
- CAD overrides a model's prior knowledge when it contradicts the provided context
- CAD leads to substantial improvements in resolving knowledge conflicts
- Hyperparameter α controls the adjustment level, with α = 0.5 generally yielding good results
- CAD outperforms standard decoding algorithms on CNN-DM and XSUM datasets
- CAD improves both the quality and factuality of generated summaries from diverse language models

## Summary

The article discusses the issue of language models struggling to pay enough attention to input context, resulting in unfaithful or hallucinatory texts. To address this problem, the authors propose a solution called context-aware decoding (CAD), which amplifies the difference between output probabilities when a model is used with and without context. The experiments show that CAD significantly improves the faithfulness of different language models for summarization tasks, with a 14.3% gain in factuality metrics for LLaMA. CAD is particularly effective in overriding a model's prior knowledge when it contradicts the provided context, leading to substantial improvements in tasks where resolving knowledge conflicts is essential. The study introduces a hyperparameter α to control the adjustment level, with α = 0.5 generally yielding good results across all settings and datasets. The results on CNN-DM and XSUM datasets demonstrate that CAD outperforms standard decoding algorithms by a large margin, improving both the quality and factuality of generated summaries from diverse language models.