# Deep Neural Networks as 0-1 Mixed Integer Linear Programs: A Feasibility Study

**Authors:** Matteo Fischetti, Jason Jo  
**arXiv:** [1712.06174](https://arxiv.org/abs/1712.06174)  
**Date:** 2017-12-17  
**Summary page:** https://www.summarizepaper.com/en/paper/1712.06174v1/

## Key points

- Deep Neural Networks (DNNs) are popular and extensively studied
- DNNs consist of layers of neurons computing affine combinations with nonlinear activation functions like ReLU
- DNNs can be modeled as 0-1 Mixed Integer Linear Programs (0-1 MILP) using continuous and binary variables
- A bound-tightening technique is introduced to improve the efficiency of solving 0-1 MILP models
- Potential applications of 0-1 MILP models include feature visualization and constructing adversarial examples
- Preliminary results show the performance of a state-of-the-art MILP solver on small DNNs for hand-written digit recognition
- Authors Matteo Fischetti and Jason Jo explore leveraging 0-1 MILP models for optimizing deep neural networks, highlighting their practical feasibility

## Summary

Deep Neural Networks (DNNs) have gained widespread popularity and are currently under intense investigation. These networks consist of layers of internal units or neurons that compute an affine combination of the output from the previous layer. A nonlinear operator is then applied to this computation, resulting in an activation value. One commonly used nonlinear operator is the rectified linear unit (ReLU), which outputs the maximum between its input value and zero. In cases where operations such as max pooling involve multiple input values, DNNs can be modeled as 0-1 Mixed Integer Linear Programs (0-1 MILP). This modeling approach uses continuous variables for the output values of each unit and binary variables to represent ReLUs' on/off nature. This paper delves into the unique characteristics of these 0-1 MILP models and introduces a bound-tightening technique aimed at improving their solution efficiency. The study also explores potential applications of the 0-1 MILP model in feature visualization and constructing adversarial examples. Preliminary computational results are presented, focusing on evaluating a state-of-the-art MILP solver's performance when applied to small DNNs in a specific test case: hand-written digit recognition. Authors Matteo Fischetti and Jason Jo provide valuable insights into leveraging 0-1 MILP models for optimizing deep neural networks and highlight their feasibility in various practical scenarios. Their research sheds light on the intricate interplay between mathematical optimization techniques and cutting-edge machine learning algorithms, offering promising avenues for further exploration in this rapidly evolving field.