# Gradients of Counterfactuals

**Authors:** Mukund Sundararajan, Ankur Taly, Qiqi Yan  
**arXiv:** [1611.02639](https://arxiv.org/abs/1611.02639)  
**Date:** 2016-11-08  
**Summary page:** https://www.summarizepaper.com/en/paper/1611.02639v1/

## Key points

- Gradients can be limited in nonlinear deep networks due to saturation
- Interior gradients are proposed as a solution, which are gradients of counterfactual inputs constructed by scaling down the original input
- Interior gradients better capture feature importance compared to traditional methods
- Applicable to a wide variety of deep networks
- Interior gradients possess the attribution property where feature importance scores sum up to the prediction score
- Computing interior gradients is as easy as computing regular gradients, making them more practical for implementation compared to complex previous methods
- Interior gradients are effective across different network architectures and have been empirically demonstrated
- They provide a valuable tool for quantifying feature importance in nonlinear deep networks

## Summary

The paper titled "Gradients of Counterfactuals" by Mukund Sundararajan, Ankur Taly, and Qiqi Yan explores the use of gradients to quantify feature importance in machine learning models. While gradients have been widely used for this purpose, they can be limited in nonlinear deep networks where both individual neurons and the entire network can saturate. This saturation can result in important input features having tiny gradients. To address this issue, the authors propose examining interior gradients, which are gradients of counterfactual inputs constructed by scaling down the original input. By applying their method to various networks, including the GoogleNet architecture for object recognition in images, a ligand-based virtual screening network with categorical features, and an LSTM-based language model for the Penn Treebank dataset, they demonstrate that interior gradients better capture feature importance compared to traditional methods. One key advantage of interior gradients is their applicability to a wide variety of deep networks. Additionally, these gradients possess the attribution property where the feature importance scores sum up to the prediction score. Moreover, computing interior gradients is as easy as computing regular gradients, making them more practical for implementation compared to complex previous methods. In conclusion, this paper introduces interior gradients as a valuable tool for quantifying feature importance in nonlinear deep networks. The authors provide empirical evidence of their effectiveness across different network architectures and highlight their simplicity of computation compared to existing approaches.