# Generative Models for Effective ML on Private, Decentralized Datasets

**Authors:** Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas  
**arXiv:** [1911.06679](https://arxiv.org/abs/1911.06679)  
**Date:** 2019-11-15  
**Summary page:** https://www.summarizepaper.com/en/paper/1911.06679v2/

## Key points

- Authors explore the use of generative models in improving real-world applications of machine learning
- Manual data inspection is crucial for identifying and rectifying issues, generating hypotheses, and refining labels
- Challenges arise with privacy-sensitive datasets representing real-world behavior
- Generative models trained with formal differential privacy guarantees can address data issues without direct inspection
- Application of differentially private federated RNNs for text and GANs for images demonstrates effectiveness
- Generative models have potential to enhance machine learning on private and decentralized datasets while maintaining privacy

## Summary

In their paper titled "Generative Models for Effective ML on Private, Decentralized Datasets," authors Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, and Blaise Aguera y Arcas explore the use of generative models in improving real-world applications of machine learning. Experienced modelers often rely on intuition about their datasets and models to enhance performance. Manual inspection of raw data plays a crucial role in identifying and rectifying issues within the data, generating new modeling hypotheses, and refining human-provided labels. However, manual data inspection becomes challenging when dealing with privacy-sensitive datasets that represent the behavior of real-world individuals. Additionally, in federated learning settings where raw examples are stored at the edge and modelers can only access aggregated outputs like metrics or model parameters, manual data inspection is not feasible. The authors demonstrate that generative models trained using federated methods with formal differential privacy guarantees can effectively address common data issues even when direct data inspection is not possible. They apply these methods to text using differentially private federated Recurrent Neural Networks (RNNs) and to images through a novel algorithm for differentially private federated Generative Adversarial Networks (GANs). Overall,this research highlights the potential of generative models in enhancing machine learning on private and decentralized datasets by providing solutions to data challenges without compromising privacy or requiring direct access to sensitive information.