# Contrastive Decoding Improves Reasoning in Large Language Models

**Authors:** Sean O'Brien, Mike Lewis  
**arXiv:** [2309.09117](https://arxiv.org/abs/2309.09117)  
**Date:** 2023-09-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.09117v1/

## Key points

- Contrastive Decoding is a simple and computationally light text generation method proposed by Li et al. (2022)
- It aims to improve the quality of long-form text generation by maximizing the difference in likelihood between strong and weak models
- Contrastive Decoding outperforms greedy decoding on commonsense reasoning and math word reasoning benchmarks
- LLaMA-65B using Contrastive Decoding surpasses other models on HellaSwag commonsense reasoning benchmark and GSM8K math word reasoning benchmark
- It prevents abstract reasoning errors and avoids simpler modes such as copying sections of the input during chain-of-thought
- More effective than nucleus sampling for long-form generation and greedy decoding for reasoning tasks
- Further research needed to optimize the contrastive objective in generating text effectively
- Reranking is considered ineffective for judging the merits of a generation-level contrastive score
- Differentiates from previous works by focusing on training-free contrastive decoding to improve reasoning capability
- Highlights the power of Contrastive Decoding as a general-purpose method for generating text from language models

## Summary

In this study, the authors demonstrate the effectiveness of Contrastive Decoding, a simple and computationally light text generation method proposed by Li et al. (2022), on various reasoning tasks. Contrastive Decoding aims to improve the quality of long-form text generation by searching for strings that maximize the difference in likelihood between strong and weak models. The authors show that Contrastive Decoding outperforms greedy decoding on tasks such as commonsense reasoning and math word reasoning benchmarks. The results indicate that LLaMA-65B using Contrastive Decoding surpasses other models like LLaMA 2, GPT-3.5, and PaLM 2-L on the HellaSwag commonsense reasoning benchmark, as well as LLaMA 2, GPT-3.5, and PaLM-540B on the GSM8K math word reasoning benchmark. Additionally, Contrastive Decoding shows improvements on other tasks as well. The analysis suggests that Contrastive Decoding improves over existing methods by preventing abstract reasoning errors and avoiding simpler modes such as copying sections of the input during chain-of-thought. It is found to be more effective than nucleus sampling for long-form generation and greedy decoding for reasoning tasks. However, further research is needed to optimize the contrastive objective in generating text effectively. Reranking is considered ineffective for judging the merits of a generation-level contrastive score. The related work section discusses steering methods for reasoning, prompting methods for reasoning, sampling methods in language models, and contrastive generation methods. The authors differentiate their approach from previous works by focusing on training-free contrastive decoding to improve reasoning capability rather than anti-toxicity or human judgments of open-ended generations. In conclusion, this study highlights the power of Contrastive Decoding as a general-purpose method for generating text from language models. It achieves significant improvements over greedy decoding on various reasoning tasks and demonstrates its potential in enhancing the quality of long-form text generation. However, further research is required to optimize the contrastive objective and explore better methods for generating text using this approach.