# iRAG: An Incremental Retrieval Augmented Generation System for Videos

**Authors:** Md Adnan Arefeen, Biplob Debnath, Md Yusuf Sarwar Uddin, Srimat Chakradhar  
**arXiv:** [2404.12309](https://arxiv.org/abs/2404.12309)  
**Date:** 2024-04-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.12309v1/

## Key points

- The iRAG system introduces an innovative approach to address limitations of traditional Retrieval Augmented Generation (RAG) systems for large corpus of multimodal data.
- iRAG augments RAG with an incremental workflow for interactive querying of multimodal data, avoiding upfront conversion of all content into text descriptions.
- iRAG quickly indexes the multimodal data and extracts relevant details based on user queries, ensuring contextually rich and accurate responses.
- Experimental results show significant improvement in processing speed, with video-to-text ingestion being 23x to 25x faster compared to traditional RAG systems.
- Despite the efficiency gain, the quality of responses provided by iRAG remains comparable to those generated by traditional RAG systems.

## Summary

The iRAG system, proposed by Md Adnan Arefeen, Biplob Debnath, Md Yusuf Sarwar Uddin, and Srimat Chakradhar, introduces an innovative approach to address the limitations of traditional Retrieval Augmented Generation (RAG) systems when dealing with large corpus of multimodal data. RAG systems are known for their ability to combine language generation and information retrieval for applications like chatbots. However, the upfront conversion of all content in multimodal data into text descriptions can lead to high processing times and potential loss of information not captured in the text. Additionally, since user queries are not known apriori, developing a system for interactive querying of multimodal data poses a significant challenge. In response to these limitations, iRAG augments RAG with an incremental workflow that enables interactive querying of large repositories of multimodal data. Unlike traditional RAG systems, iRAG quickly indexes the multimodal data and utilizes this index in an incremental workflow to extract relevant details from select portions of the data based on user queries. This on-demand extraction approach avoids long conversion times and ensures that responses to user queries are contextually rich and accurate. The key innovation of iRAG lies in its ability to support efficient interactive querying of real-world multimodal data by incrementally extracting information as needed, rather than converting all data upfront. Experimental results demonstrate a significant improvement in processing speed, with video-to-text ingestion being 23x to 25x faster compared to traditional RAG systems. Despite this efficiency gain, the quality of responses provided by iRAG remains comparable to those generated by traditional RAG systems. Overall, iRAG represents a groundbreaking advancement in the field of multimodal data processing by introducing an incremental workflow that enhances the effectiveness and responsiveness of retrieval augmented generation systems when dealing with large volumes of diverse multimedia content.