# Large Language Models on Tabular Data -- A Survey

**Authors:** Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego Socolinsky, Srinivasan Sengamedu, Christos Faloutsos  
**arXiv:** [2402.17944](https://arxiv.org/abs/2402.17944)  
**Date:** 2024-02-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.17944v1/

## Key points

- Significant advancements in large language modeling have enabled exploration of applications in tabular data modeling tasks.
- Tasks related to tabular data modeling include prediction, synthesis, question answering, and table understanding.
- There is a lack of a comprehensive review synthesizing and comparing key techniques, metrics, datasets, models, and optimization approaches in this research domain.
- A survey was conducted to consolidate recent developments in these areas.
- The survey provides an examination and taxonomy of datasets, metrics, and methodologies used in tabular data modeling with large language models.
- It highlights strengths and limitations of existing literature and offers insights for future research directions.
- Relevant code references and datasets are provided to assist readers in navigating challenges effectively.
- Researchers can gain necessary tools and knowledge from the review to make significant contributions in this field.
- Random initialization models or leveraging learning dynamics are highlighted for addressing long training times of LLMs through high-quality synthetic data generation.
- Evaluation methods for synthetic data quality are discussed using low-order statistics and high-order metrics.

## Summary

Significant advancements in large language modeling have paved the way for extensive exploration of their applications in various tasks related to tabular data modeling. These tasks include prediction, tabular data synthesis, question answering, and table understanding. However, there is a lack of a comprehensive review that synthesizes and compares key techniques, metrics, datasets, models, and optimization approaches within this research domain. To address this gap, a survey was conducted to consolidate recent developments in these areas. The survey offers a thorough examination and taxonomy of the datasets, metrics, and methodologies utilized in tabular data modeling with large language models. It highlights the strengths and limitations of existing literature while providing insights for future research directions. Additionally, relevant code references and datasets are provided to aid interested readers in navigating and addressing challenges effectively. Through this comprehensive review , researchers can gain necessary tools and knowledge required to make significant contributions in this field. Furthermore, are highlighted for their potential in addressing long training times of LLMs through random initialization models or leveraging learning dynamics for high-quality synthetic data generation. Evaluation methods for synthetic data quality are also discussed from various dimensions such as low-order statistics and high-order metrics. Overall,. It serves as a valuable resource for researchers looking to delve deeper into this vital research area.