# VerifAI: Verified Generative AI

**Authors:** Nan Tang, Chenyu Yang, Ju Fan, Lei Cao, Yuyu Luo, Alon Halevy  
**arXiv:** [2307.02796](https://arxiv.org/abs/2307.02796)  
**Date:** 2023-07-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.02796v2/

## Key points

- Generative AI advancements have raised concerns about accuracy and reliability
- Inaccuracies in generative AI can lead to serious consequences such as misinformation, privacy violations, and legal liabilities
- Efforts to address risks include explainable AI, transparency, bias mitigation, and social responsibility
- VerifAI framework offers a modularized approach for verifying generative data across various modalities like text files, tables, and knowledge graphs
- VerifAI consists of Indexer module for dataset indexing, Reranker module for fine-tuning rankings of retrieved data sources, and Verifier module for validating generated data objects
- Multi-modal data lakes store diverse structured and unstructured data types including tables and text
- Case study demonstrates how VerifAI verifies textual claims based on retrieved tables using ChatGPT
- Framework integrates local models like PASTA for higher accuracy while maintaining privacy
- VerifAI leverages multi-modal data lakes to ensure correctness of generative AI outputs and promote transparency in decision-making processes

## Summary

, , , , 

Generative AI has made significant advancements, but concerns about its accuracy and reliability persist. Inaccuracies in generative AI can have serious consequences such as misinformation, privacy violations, legal liabilities, and more. Efforts to address these risks include explainable AI and responsible practices like transparency, bias mitigation, and social responsibility. However, verifying the outputs of generative AI from a data management perspective is emerging as a crucial issue. Introducing VerifAI, a framework for verified generative AI that offers a modularized approach for verifying generative data across various modalities such as text files, tables, and knowledge graphs. The framework consists of an Indexer module for indexing datasets, a Reranker module for fine-tuning rankings of retrieved data sources, and a Verifier module for validating generated data objects. Through experiments, VerifAI has shown high accuracy in verifying generated tables and text using multi-modal data lakes. refers to information created by models or algorithms rather than directly observed in the real world. This work specifically focuses on data generated by large language models like ChatGPT using natural language generation techniques. Multi-modal data lakes serve as repositories for storing diverse types of structured and unstructured data including tables and text. A case study presented in Figure 4 demonstrates how VerifAI can verify textual claims based on retrieved tables using ChatGPT. By retrieving relevant tables that either support or refute a claim, users can make informed decisions with explanations provided by the model. The framework's ability to integrate local models like PASTA for higher accuracy while maintaining privacy highlights its versatility in different use cases. Overall, VerifAI showcases the potential of leveraging multi-modal data lakes to ensure the correctness of generative AI outputs and promote transparency in decision-making processes. As open problems persist in addressing challenges related to generative AI verification, continued research efforts are needed to enhance trustworthiness in data sources and improve the overall reliability of machine learning models.