Federated Learning with Matched Averaging

AI-generated keywords: Federated Learning FedMA CNNs LSTMs Data Bias

AI-generated Key Points

  • Federated learning enables edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store data in the cloud.
  • Federated matched averaging (FedMA) algorithm has been proposed for federated learning of modern neural network architectures such as convolutional neural networks (CNNs) and LSTMs.
  • FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements with similar feature extraction signatures.
  • FedMA outperforms popular state-of-the-art federated learning algorithms and reduces overall communication burden.
  • Addressing data bias is important for improving generalizability of machine learning models trained on real-world datasets.
  • Various techniques have been proposed such as inclusive images (Doshi 2018).
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, Yasaman Khazaeni

Accepted by ICLR 2020
License: CC BY 4.0

Abstract: Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, but also reduces the overall communication burden.

Submitted to arXiv on 15 Feb. 2020

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2002.06440v1

Federated learning has emerged as a promising approach to enable edge devices to collaboratively learn a shared model while keeping the training data on device, thereby decoupling the ability to do model training from the need to store the data in the cloud. In this regard, Federated matched averaging (FedMA) algorithm has been proposed for federated learning of modern neural network architectures such as convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements with similar feature extraction signatures. Specifically, it matches channels for convolution layers, hidden states for LSTM, and neurons for fully connected layers. The performance of FedMA has been evaluated through experiments on deep CNN and LSTM architectures trained on real-world datasets. The results indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms but also reduces overall communication burden. For instance, Table 1 shows that FedMA achieves a final accuracy of 87.53% for VGG-9 trained on CIFAR-10 dataset compared to 86.29% achieved by FedAvg and 85.32% achieved by FedProx Ensemble methods. Addressing data bias is an important aspect of machine learning models since real-world data often exhibit multimodality within each class leading to biases in classification models trained on such data. To address this issue, various techniques have been proposed such as inclusive images (Doshi 2018). In conclusion, Federated matched averaging algorithm offers a promising solution for federated learning of modern neural network architectures while reducing communication burden and achieving better performance than existing state-of-the-art methods. Furthermore, addressing data bias is essential for improving the generalizability of machine learning models trained on real-world datasets.
Created on 12 Jun. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.