Answering Questions by Meta-Reasoning over Multiple Chains of Thought

AI-generated keywords: Multi-Chain Reasoning Question Answering Meta-Reasoning Decomposition Model Retriever Component

AI-generated Key Points

  • Multi-Chain Reasoning (MCR) is an approach to multi-hop question answering that prompts large language models to meta-reason over multiple chains of thought.
  • MCR examines different reasoning chains, mixes information between them, and selects the most relevant facts in generating an explanation and predicting the answer.
  • MCR outperforms strong baselines on seven multi-hop QA datasets and its explanations exhibit high quality, enabling humans to verify its answers.
  • The process of generating reasoning chains involves a decomposition model that generates intermediate questions based on the original question and previous reasoning steps, and a retriever component that retrieves relevant information from external sources.
  • The generated reasoning chains are then fed into a prompted LLM which generates multiple reasoning chains by interleaving generated intermediate questions with retrieved contexts.
  • The main contribution of this work is introducing a second LLM that is prompted to meta-reason on multiple reasoning chains, collecting evidence facts as its explanation and generating the final answer.
  • This approach considers the relations between intermediate steps across chains and provides a unified explanation for the predicted answer.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, Jonathan Berant

License: CC BY 4.0

Abstract: Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer. Often, multiple chains are sampled and aggregated through a voting mechanism over the final answers, but the intermediate steps themselves are discarded. While such approaches improve performance, they do not consider the relations between intermediate steps across chains and do not provide a unified explanation for the predicted answer. We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought, rather than aggregating their answers. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer. MCR outperforms strong baselines on 7 multi-hop QA datasets. Moreover, our analysis reveals that MCR explanations exhibit high quality, enabling humans to verify its answers.

Submitted to arXiv on 25 Apr. 2023

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2304.13007v1

The paper introduces Multi-Chain Reasoning (MCR), an approach to multi-hop question answering (QA) that prompts large language models to meta-reason over multiple chains of thought, rather than simply aggregating their answers. Unlike existing approaches that break down questions into a sequence of reasoning steps and discard intermediate steps, MCR examines different reasoning chains, mixes information between them, and selects the most relevant facts in generating an explanation and predicting the answer. The authors demonstrate that MCR outperforms strong baselines on seven multi-hop QA datasets and that its explanations exhibit high quality, enabling humans to verify its answers. The paper describes the process of generating reasoning chains using a decomposition model and a retriever component. The decomposition model generates intermediate questions based on the original question and previous reasoning steps while the retriever component retrieves relevant information from external sources. The generated reasoning chains are then fed into a prompted LLM which generates multiple reasoning chains by interleaving generated intermediate questions with retrieved contexts. The main contribution of this work is introducing a second LLM that is prompted to meta-reason on multiple reasoning chains, collecting evidence facts as its explanation and generating the final answer. This approach considers the relations between intermediate steps across chains and provides a unified explanation for the predicted answer. Overall, this paper presents an innovative approach to multi-hop QA that improves performance by considering multiple reasoning chains and providing more comprehensive explanations for predicted answers. It enables humans to verify its answers through high quality explanations generated by MCR's meta-reasoning process over multiple reasoning chains.
Created on 27 Apr. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.