# Federated Learning with Matched Averaging

**Authors:** Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, Yasaman Khazaeni  
**arXiv:** [2002.06440](https://arxiv.org/abs/2002.06440)  
**Date:** 2020-02-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2002.06440v1/

## Key points

- Federated learning enables edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store data in the cloud.
- Federated matched averaging (FedMA) algorithm has been proposed for federated learning of modern neural network architectures such as convolutional neural networks (CNNs) and LSTMs.
- FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements with similar feature extraction signatures.
- FedMA outperforms popular state-of-the-art federated learning algorithms and reduces overall communication burden.
- Addressing data bias is important for improving generalizability of machine learning models trained on real-world datasets.
- Various techniques have been proposed such as inclusive images (Doshi 2018).

## Summary

Federated learning has emerged as a promising approach to enable edge devices to collaboratively learn a shared model while keeping the training data on device, thereby decoupling the ability to do model training from the need to store the data in the cloud. In this regard, Federated matched averaging (FedMA) algorithm has been proposed for federated learning of modern neural network architectures such as convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements with similar feature extraction signatures. Specifically, it matches channels for convolution layers, hidden states for LSTM, and neurons for fully connected layers. The performance of FedMA has been evaluated through experiments on deep CNN and LSTM architectures trained on real-world datasets. The results indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms but also reduces overall communication burden. For instance, Table 1 shows that FedMA achieves a final accuracy of 87.53% for VGG-9 trained on CIFAR-10 dataset compared to 86.29% achieved by FedAvg and 85.32% achieved by FedProx Ensemble methods. Addressing data bias is an important aspect of machine learning models since real-world data often exhibit multimodality within each class leading to biases in classification models trained on such data. To address this issue, various techniques have been proposed such as inclusive images (Doshi 2018). In conclusion, Federated matched averaging algorithm offers a promising solution for federated learning of modern neural network architectures while reducing communication burden and achieving better performance than existing state-of-the-art methods. Furthermore, addressing data bias is essential for improving the generalizability of machine learning models trained on real-world datasets.