Exécution de notre outil de synthèse sur un nouvel article

Vérifiez l’article ci-dessous, puis lancez sa synthèse.


Two-tiered Encoder-based Hallucination Detection for Retrieval-Augmented Generation in the Wild

Ilana Zimmerman, Jadin Tredup, Ethan O. Selfridge, Joseph Bradley

Conference on Empirical Methods in Natural Language Processing (EMNLP)
Licence : CC BY

Résumé : Detecting hallucinations, where Large Language Models (LLMs) are not factually consistent with a Knowledge Base (KB), is a challenge for Retrieval-Augmented Generation (RAG) systems.Current solutions rely on public datasets to develop prompts or fine-tune a Natural Language Inference (NLI) model.However, these approaches are not focused on developing an enterprise RAG system; they do not consider latency, train or evaluate on production data, nor do they handle non-verifiable statements such as small talk or questions.To address this, we leverage the customer service conversation data of four large brands to evaluate existing solutions and propose a set of small encoder models trained on a new dataset.We find the proposed models to outperform existing methods and highlight the value of combining a small amount of in-domain data with public datasets.

Publié le 01 Jan. 2024