Running summarizing tools on a new article

Abstract: Detecting hallucinations, where Large Language Models (LLMs) are not factually consistent with a Knowledge Base (KB), is a challenge for Retrieval-Augmented Generation (RAG) systems.Current solutions rely on public datasets to develop prompts or fine-tune a Natural Language Inference (NLI) model.However, these approaches are not focused on developing an enterprise RAG system; they do not consider latency, train or evaluate on production data, nor do they handle non-verifiable statements such as small talk or questions.To address this, we leverage the customer service conversation data of four large brands to evaluate existing solutions and propose a set of small encoder models trained on a new dataset.We find the proposed models to outperform existing methods and highlight the value of combining a small amount of in-domain data with public datasets.