# Benchmarking the Fidelity and Utility of Synthetic Relational Data

**Authors:** Valter Hudovernik, Martin Jurkovič, Erik Štrumbelj  
**arXiv:** [2410.03411](https://arxiv.org/abs/2410.03411)  
**Date:** 2024-10-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.03411v1/

## Key points

- Importance of synthesizing relational data in research, practice, and industry
- Complexity of synthesizing relational data compared to single tables due to intricate relationships
- Challenges in benchmarking methods for synthesizing relational data
- Lack of empirical evaluation and gaps in understanding how evaluations should be conducted
- Comprehensive review of related work on relational data synthesis, benchmarking datasets, and measuring fidelity and utility of synthetic data
- Development of a benchmarking tool to compare six different methods, including two commercial tools
- Results show no method can produce completely indistinguishable dataset from original data; moderate correlations observed between real and synthetic data for predictive performance and feature importance
- Study sheds light on challenges in synthesizing relational data and provides insights into evaluating fidelity and utility of synthetic datasets
- Benchmarking tool is a valuable resource for researchers, practitioners, and industry professionals looking to improve methods for generating synthetic relational data

## Summary

In their paper titled "Benchmarking the Fidelity and Utility of Synthetic Relational Data," authors Valter Hudovernik, Martin Jurkovič, and Erik Štrumbelj delve into the growing importance of synthesizing relational data in research, practice, and industry. They highlight the inherent complexity of this task compared to synthesizing single tables due to the intricate relationships between various data tables. The authors also point out that benchmarking methods for synthesizing relational data present new challenges that need to be addressed. Motivated by a lack of empirical evaluation of state-of-the-art methods and gaps in understanding how such evaluations should be conducted, the authors conduct a comprehensive review of related work on relational data synthesis, common benchmarking datasets, and approaches to measuring the fidelity and utility of synthetic data. They combine best practices with a novel robust detection approach to develop a benchmarking tool that allows them to compare six different methods, including two commercial tools. The results of their study reveal that while some methods perform better than others in synthesizing relational data, none are able to produce a dataset that is completely indistinguishable from the original data. In terms of utility, moderate correlations are typically observed between real and synthetic data for both model predictive performance and feature importance. Overall, this study sheds light on the challenges associated with synthesizing relational data and provides valuable insights into evaluating the fidelity and utility of synthetic datasets. The authors' benchmarking tool serves as a valuable resource for researchers, practitioners, and industry professionals looking to improve their methods for generating synthetic relational data.