# Learning to Generate Image Embeddings with User-level Differential Privacy

**Authors:** Zheng Xu, Maxwell Collins, Yuxiao Wang, Liviu Panait, Sewoong Oh, Sean Augenstein, Ting Liu, Florian Schroff, H. Brendan McMahan  
**arXiv:** [2211.10844](https://arxiv.org/abs/2211.10844)  
**Date:** 2022-11-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.10844v2/

## Key points

- Authors: Zheng Xu, Maxwell Collins, Yuxiao Wang, Liviu Panait, Sewoong Oh, Sean Augenstein, Ting Liu, Florian Schroff, H. Brendan McMahan
- Title: "Learning to Generate Image Embeddings with User-level Differential Privacy"
- Challenge addressed: Training large image-to-embedding feature extractors with user-level differential privacy (DP)
- Proposed solution: DP-FedEmb as a variant of federated learning algorithms incorporating per-user sensitivity control and noise addition
- Techniques used: Virtual clients, partial aggregation, private local fine-tuning, public pretraining
- Applications: Training image embedding models for faces, landmarks, and natural species on benchmark datasets (DigiFace, EMNIST, GLD, iNaturalist)
- Experiment results: Achieved utility under the same privacy budget with epsilon less than 4 while controlling drop in utility within 5%
- Contribution: Enhancing privacy protection in large-scale image processing tasks while maintaining high levels of utility

## Summary

In their paper titled "Learning to Generate Image Embeddings with User-level Differential Privacy," authors Zheng Xu, Maxwell Collins, Yuxiao Wang, Liviu Panait, Sewoong Oh, Sean Augenstein, Ting Liu, Florian Schroff, and H. Brendan McMahan address the challenge of training large image-to-embedding feature extractors with user-level differential privacy (DP). They propose DP-FedEmb as a variant of federated learning algorithms that incorporates per-user sensitivity control and noise addition to overcome this challenge. This approach allows for training from user-partitioned data centralized in the datacenter and leverages techniques such as virtual clients, partial aggregation, private local fine-tuning, and public pretraining to achieve strong trade-offs between privacy and utility. The authors apply DP-FedEmb to train image embedding models for faces, landmarks, and natural species on benchmark datasets including DigiFace, EMNIST, GLD, and iNaturalist. Through their experiments on millions of users participating in the training process, in terms of utility under the same privacy budget with epsilon less than 4 while controlling the drop in utility within 5%. This research contributes valuable insights into enhancing privacy protection in large-scale image processing tasks while maintaining high levels of utility.