Exécution de notre outil de synthèse sur un nouvel article

Résumé : Large Language Models (LLM) have greatly enhanced the capacity for natural language understanding and generation; however, they consistently generate "hallucinations," i.e., responses that may be grammatically and/or semantically accurate yet contain unsubstantiated or logically inaccurate information. These hallucinations limit the use of LLMs in high-stake areas (e.g., clinical diagnosis, financial forecasting, etc.) where factual accuracy is paramount. We introduce HAL-RAG, a comprehensive LLM response validation and mitigation framework that integrates both Retrieval-Augmented Generation (RAG) with granular, claim-level Natural Language Inference (NLI) based evidence validation. HAL-RAG consists of three sequential operations: First, it breaks down generated LLM responses into independent, atomic claims that can be verified separately. Second, it retrieves evidence supporting each claim from a well-curated, domain specific knowledge base. Third, it uses the evidence to validate the claim through NLI. Results of experiments conducted using a custom dataset consisting of 487 atomic claims demonstrate that HAL-RAG achieved an exceptional 87.3% accuracy in detecting hallucinations; representing a 16.1% increase in accuracy compared to baseline LLM systems and a 64.2% decrease in the overall number of hallucinations detected. Ablation studies also show that each of the three components of the HAL-RAG architecture contributed significantly and uniquely to the overall performance of the system. Overall, the results suggest that combining retrieval mechanisms with logical evidence verification provide a very effective, interpretable and adaptable solution to the widespread problem of hallucination generated by LLMs.