Exécution de notre outil de synthèse sur un nouvel article
Vérifiez l’article ci-dessous, puis lancez sa synthèse.
RAG-Guided Large Language Models for Visual Spatial Description with Adaptive Hallucination Corrector
Jun Yu (University of Science and Technology of China), Yunxiang Zhang (University of Science and Technology of China), Zerui Zhang (University of Science and Technology of China), Yang Zhao (Xi'an Jiaotong University), Gongpeng Zhao, Fengzhao Sun (University of Science and Technology of China), Fanrui Zhang (University of Science and Technology of China), Qingsong Liu, Jianqing Sun, Jiaen Liang, Yaohui Zhang (University of Science and Technology of China)
Résumé : Visual Spatial Description (VSD) is an emerging image-to-text task which aims at generating descriptions of the spatial relationships between given objects in an image. In this paper, we apply Retrieval-Augmented Generation (RAG) technology in guiding Multimodal Large Language Models (MLLMs) for the task of VSD, complemented by an Adaptive Hallucination Corrector, and further fine-tuning them to bolster semantic understanding and overall model efficacy. We found that our approach demonstrated higher accuracy and fewer hallucination errors in both spatial relationship classification and visual language description tasks within the VSD task, achieving state-of-the-art results.