Exécution de notre outil de synthèse sur un nouvel article

Résumé : Abstract Retrieval-Augmented Generation (RAG) improves the accessibility of external knowledge for large language models, but generated responses can remain unsupported, contradictory, or otherwise hallucinatory. Conventional hallucination-detector evaluation largely emphasizes aggregate benchmark performance, which may not reveal whether detector decisions remain reliable when the evidence distribution changes. This study presents a controlled evaluation of hallucination detectors from an evidence-sensitivity and robustness perspective. Three detector configurations - TF-IDF with Logistic Regression, TF-IDF with Linear Support Vector Machine, and DeBERTa-v3-base were evaluated using RAGTruth, HaluBench, and TRIVIA+. In-domain evaluation on RAGTruth showed that DeBERTa achieved an AUROC of 0.8483 and Macro-F1 of 0.7616. However, performance declined substantially under distribution shift, with AUROC decreasing to 0.5348 on HaluBench and 0.5910 on TRIVIA+. Controlled evidence interventions showed substantial detector responsiveness, with mean Evidence Sensitivity Index values of 0.2641 for evidence removal and 0.3268 for cross-example evidence substitution. A context-complexity stress test further showed progressive degradation as irrelevant evidence increased from 0% to 100% within a fixed 512-token observation window, with AUROC decreasing from 0.8306 to 0.7325 and Expected Calibration Error increasing from 0.1375 to 0.4072. Input ablation demonstrated that both generated answers and retrieved evidence contribute to detection performance. These findings show that benchmark accuracy alone is insufficient to characterize hallucination-detector reliability and motivate evaluation frameworks that jointly assess generalization, evidence sensitivity, contextual robustness, and calibration.