# Chain-of-Verification Reduces Hallucination in Large Language Models

**Authors:** Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston  
**arXiv:** [2309.11495](https://arxiv.org/abs/2309.11495)  
**Date:** 2023-09-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.11495v2/

## Key points

- Authors address the issue of generating plausible yet incorrect factual information in large language models
- Introduce Chain-of-Verification (CoVe) approach to reduce hallucinations in responses
- CoVe involves a multi-step process: initial response drafting, formulating verification questions, fact-checking independently, and generating final verified response
- Experiments conducted across various tasks show implementing CoVe decreases hallucinations in language models
- Implementing CoVe can enhance accuracy and reliability of content generated by large language models
- Study highlights innovative strategies for improving AI-generated information quality through verification processes

## Summary

In their paper titled "Chain-of-Verification Reduces Hallucination in Large Language Models," authors Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston address the issue of generating plausible yet incorrect factual information in large language models. They investigate the capacity of these models to deliberate on their responses and correct errors by introducing the . This approach involves a multi-step process where the model first drafts an initial response, then formulates verification questions to fact-check its draft independently from other responses. Subsequently, it answers these questions to generate a final verified response. The authors conducted experiments to evaluate the effectiveness of CoVe in reducing hallucinations across various tasks such as list-based questions sourced from Wikidata, closed book MultiSpanQA queries, and longform text generation scenarios. Their findings demonstrate that implementing CoVe leads to a decrease in hallucinations within language models. By incorporating this method into their workflow, researchers can enhance the accuracy and reliability of generated content produced by large language models. The study sheds light on innovative strategies for improving the quality of information generated by AI systems and highlights the importance of verification processes in mitigating errors and enhancing overall performance.