# Language Models are Realistic Tabular Data Generators

**Authors:** Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, Gjergji Kasneci  
**arXiv:** [2210.06280](https://arxiv.org/abs/2210.06280)  
**Date:** 2022-10-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.06280v1/

## Key points

- The paper is titled "Language Models are Realistic Tabular Data Generators"
- The authors address the challenge of generating synthetic samples with the original data's characteristics for tabular data
- Less research has been directed towards recent transformer-based large language models (LLMs) for tabular data generation
- The authors propose a new approach called GReaT (Generation of Realistic Tabular data) that exploits an auto-regressive generative LLM to sample synthetic and highly realistic tabular data
- GReaT can model tabular data distributions by conditioning on any subset of features while sampling remaining features without additional computational overhead
- The authors demonstrate the effectiveness of their approach through a series of experiments that quantify the validity and quality of produced data samples from multiple perspectives
- GReaT maintains state-of-the-art performance across many real-world datasets with heterogeneous feature types

## Summary

In their paper titled "Language Models are Realistic Tabular Data Generators," Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci address the challenge of generating synthetic samples with the original data's characteristics for tabular data. While generative models from the computer vision domain have been adapted for tabular data generation, less research has been directed towards recent transformer-based large language models (LLMs), which are also generative in nature. To fill this gap, the authors propose a new approach called GReaT (Generation of Realistic Tabular data) that exploits an auto-regressive generative LLM to sample synthetic and highly realistic tabular data. One of the key features of GReaT is its capability to model tabular data distributions by conditioning on any subset of features while sampling remaining features without additional computational overhead. The authors demonstrate the effectiveness of their approach through a series of experiments that quantify the validity and quality of produced data samples from multiple perspectives. They find that GReaT maintains state-of-the-art performance across many real-world datasets with heterogeneous feature types. Overall, this paper highlights how language models can be utilized to generate realistic tabular data and provides a promising avenue for future research in this area.