# REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers

**Authors:** Aivin V. Solatorio, Olivier Dupriez  
**arXiv:** [2302.02041](https://arxiv.org/abs/2302.02041)  
**Date:** 2023-02-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.02041v1/

## Key points

- Paper introduces REaLTabFormer model for generating synthetic tabular and relational datasets
- Model addresses challenge of capturing relational structures across tables
- REaLTabFormer creates parent table using autoregressive GPT-2 model and generates relational dataset using Seq2Seq model
- Target masking implemented to prevent data copying, $Q_{\delta}$ statistic used to detect overfitting
- Experimental results show REaLTabFormer outperforms baseline models in capturing relational structures
- Achieves state-of-the-art results on prediction tasks for large non-relational datasets without fine-tuning

## Summary

The paper "REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers" by Aivin V. Solatorio and Olivier Dupriez introduces a novel model for generating synthetic tabular and relational datasets called REaLTabFormer. This model addresses the challenge of capturing relational structures across tables, which is often difficult for existing models. To overcome this challenge, REaLTabFormer first creates a parent table using an autoregressive GPT-2 model and then generates the relational dataset conditioned on this parent table using a sequence-to-sequence (Seq2Seq) model. The authors also implement target masking to prevent data copying and introduce the $Q_{\delta}$ statistic along with statistical bootstrapping to detect overfitting in order to enhance the quality of generated data. Experimental results demonstrate that REaLTabFormer outperforms baseline models in accurately capturing relational structures. Additionally, it achieves state-of-the-art results on prediction tasks for large non-relational datasets without requiring fine-tuning. This makes REaLTabFormer a significant advancement in synthetic data generation, particularly in effectively modeling both tabular and relational data. The proposed techniques show promise in improving the quality of generated datasets and addressing challenges associated with capturing complex relationships within data tables. For those interested in exploring or implementing the REaLTabFormer model, the authors provide further details and resources through their GitHub repository.