Running summarizing tools on a new article

Abstract: Hallucinations in LLMs are a major issue in developing medical question-answering systems, as the creation of false information can lead to serious safety risks. The Retrieval-Augmented Generation (RAG) model reduces this issue by relying on external knowledge sources. However, in traditional RAG, it is still possible to generate responses even when there is weak or irrelevant evidence.This paper aims to present a novel RAG model that uses cross-encoder re-ranking in conjunction with confidence-gated abstention to ensure that responses are only generated when there is enough evidence. The model was tested using 62 oncology-related questions obtained from a dataset of cancer information in Japanese. The experiment was conducted using closed-domain retrieval of Cancer Information Service (CIS) database information and open-domain retrieval of Google search results. The generative model used in this experiment was GPT-3.5 and GPT-4.The experiment was conducted, and the results show that the proposed model significantly reduces hallucinations compared to traditional RAG. The experiment using open-domain retrieval showed a reduction in conditional hallucinations from 28.0% to 8.82% for GPT-3.5 and from 26.7% to 2.17% for GPT-4. The experiment conducted in this paper proved that combining semantic re-ranking with confidence-gated abstention improves the reliability of AI-based medical information systems.