# DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

**Authors:** Phillip Rieger, Thien Duc Nguyen, Markus Miettinen, Ahmad-Reza Sadeghi  
**arXiv:** [2201.00763](https://arxiv.org/abs/2201.00763)  
**Date:** 2022-01-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.00763v1/

## Key points

- Federated Learning (FL) allows multiple clients to train a Neural Network (NN) model collaboratively on their private data without revealing the data.
- FL has become vulnerable to targeted poisoning attacks that inject backdoors into the resulting model, allowing adversary-controlled inputs to be misclassified.
- Existing countermeasures against backdoor attacks are inefficient and often exclude benign models of clients with deviating data distributions, causing the aggregated model to perform poorly for such clients.
- DeepSight is a novel defense approach that uses three techniques to characterize the distribution of data used to train model updates and measure fine-grained differences in the internal structure and outputs of NNs.
- DeepSight's filtering scheme combines a classifier and clustering-based similarity estimations to identify suspicious model updates.
- The approach prevents models of benign clients with deviating data distributions from being filtered out, thereby increasing the performance of the aggregated model for these clients.
- DeepSight also proposes a Threshold Exceedings metric that analyzes parameter updates in real-time during training.
- DeepSight effectively mitigates targeted poisoning attacks on FL by combining deep-model-inspection with weight clipping enforcement strategy.

## Summary

Federated Learning (FL) is a technique that enables multiple clients to train a Neural Network (NN) model collaboratively on their private data without revealing the data. However, FL has become vulnerable to targeted poisoning attacks that inject backdoors into the resulting model, allowing adversary-controlled inputs to be misclassified. Existing countermeasures against backdoor attacks are inefficient and often exclude benign models of clients with deviating data distributions, causing the aggregated model to perform poorly for such clients. To address this problem, researchers have proposed DeepSight, a novel defense approach that uses three novel techniques to characterize the distribution of data used to train model updates and measure fine-grained differences in the internal structure and outputs of NNs. DeepSight's filtering scheme combines a classifier and clustering-based similarity estimations to identify suspicious model updates. The individual labels are used to reliably identify clusters with malicious model updates so that not only a model's label is used but also the labels of similar models for deciding on accepting or rejecting a model update. This approach prevents models of benign clients with deviating data distributions from being filtered out, thereby increasing the performance of the aggregated model for these clients. DeepSight also proposes a Threshold Exceedings metric that analyzes parameter updates in real-time during training. The overall structure of DeepSight includes a classifier as its central component based on deep-model-inspection followed by weight clipping enforcement strategy. The first layer realizes classification-based filtering and removes poisoned models with high attack impact where training data is focused on samples for backdoor behavior. The second layer enforces weight clipping strategy which focuses training data on backdoor behavior. DeepSight has been evaluated against several state-of-the-art defense approaches in terms of Backdoor Accuracy (BA), Model Accuracy (MA), precision (PRC), negative predictive value (NPV), indicating the probability that an accepted model is indeed benign, and its complement indicating the probability that a filtered model is indeed poisoned. DeepSight effectively mitigates the attack in both scenarios and is the only approach that is effective in both IID and non-IID data scenarios. In addition, DeepSight has been evaluated against five different NLP and 12 NIDS backdoors, showing that it is not restricted to specific targets. Overall, DeepSight proposes a novel defense approach that effectively mitigates targeted poisoning attacks on FL by combining deep-model-inspection with weight clipping enforcement strategy. It prevents models of benign clients with deviating data distributions from being filtered out, thereby increasing the performance of the aggregated model for these clients.