# ReSCORE: Label-free Iterative Retriever Training for Multi-hop Question Answering with Relevance-Consistency Supervision

**Authors:** Dosung Lee, Wonjun Oh, Boyoung Kim, Minyoung Kim, Joonsuk Park, Paul Hongsuck Seo  
**arXiv:** [2505.21250](https://arxiv.org/abs/2505.21250)  
**Date:** 2025-05-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2505.21250v1/

## Key points

- Dense retrievers outperform sparse methods like BM25 in multi-hop question answering by leveraging semantic embeddings.
- A challenge in MHQA is the variability of queries throughout reasoning steps, requiring labeled query-document pairs for fine-tuning dense retrievers.
- ReSCORE method introduces Retriever Supervision with Consistency and Relevance, utilizing large language models to capture document relevance and consistency with correct answers.
- ReSCORE enables training of a retriever within an iterative question-answering framework without the need for labeled documents, showcasing significant improvements in retrieval performance.
- Dense retrievers like Contriever are recognized as more effective overall in MHQA due to their reliance on domain-specific query and document embeddings.
- ReSCORE leverages large language models to enhance retrieval accuracy without needing labeled data, offering a solution to labor-intensive and costly training processes.

## Summary

In the realm of multi-hop question answering (MHQA), dense retrievers have shown superior performance compared to sparse methods like BM25 by leveraging semantic embeddings. However, a significant challenge arises in MHQA due to the variability of queries throughout the reasoning steps. This necessitates labeled query-document pairs for fine-tuning dense retrievers. To address this limitation, a novel method called Retriever Supervision with Consistency and Relevance (ReSCORE) has been introduced. utilizes large language models to capture the relevance of each document to the question and its consistency with the correct answer. This enables training of a retriever within an iterative question-answering framework without the need for labeled documents. Experiments conducted on three MHQA benchmarks have demonstrated the effectiveness of ReSCORE, showcasing significant improvements in retrieval performance and subsequently enhancing state-of-the-art MHQA outcomes. The  approach is commonly employed in MHQA systems, where relevant documents are retrieved iteratively to generate partial answers until a final answer is reached. While sparse retrievers like BM25 are frequently used in these systems, dense retrievers such as Contriever have been recognized as more effective overall due to their reliance on query and document embeddings trained specifically for the target domain. Despite this advantage, training dense retrievers for MHQA can be labor-intensive and costly due to the need for labeled documents reflecting their relevance across different iterations. offers a solution by leveraging large language models to streamline this process and enhance retrieval accuracy without requiring labeled data. The implementation of  is publicly available at https://leeds1219.github.io/ReSCORE.