# Running Neural Networks on the NIC

**Authors:** Giuseppe Siracusano, Salvator Galea, Davide Sanvito, Mohammad Malekzadeh, Hamed Haddadi, Gianni Antichi, Roberto Bifulco  
**arXiv:** [2009.02353](https://arxiv.org/abs/2009.02353)  
**Date:** 2020-09-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2009.02353v1/

## Key points

- Using commodity programmable Network Interface Cards (NICs) for neural network inference tasks in packet monitoring applications
- Data transfer costs to host systems and dedicated machine learning accelerators can exceed processing task costs
- System called N3IC designed and implemented on two different NICs
- N3IC supports three distinct network monitoring use cases requiring machine learning inference
- N3IC performs inference for millions of network flows per second while maintaining a traffic forwarding rate of 40Gb/s
- N3IC offers advantages over equivalent CPU solution: significant reduction in processing latency (up to 100 times) and increased throughput (1.5 times)
- Leveraging NICs for neural network inference tasks enhances network monitoring capabilities efficiently and cost-effectively

## Summary

In their paper titled "Running Neural Networks on the NIC," Giuseppe Siracusano, Salvator Galea, Davide Sanvito, Mohammad Malekzadeh, Hamed Haddadi, Gianni Antichi, and Roberto Bifulco explore the potential of using commodity programmable Network Interface Cards (NICs) to perform neural network inference tasks for packet monitoring applications. They highlight the significance of this approach as data transfer costs to host systems and dedicated machine learning accelerators like GPUs can often exceed the processing task itself. The authors present their system called N3IC, which they design and implement on two different NICs. They demonstrate that N3IC can effectively support three distinct network monitoring use cases that require machine learning inference as a first-class primitive. The system showcases impressive performance by performing inference for millions of network flows per second while maintaining a traffic forwarding rate of 40Gb/s. Comparing N3IC to an equivalent solution implemented on a general-purpose CPU, the authors find that N3IC offers several advantages. It provides a significant reduction in processing latency by up to 100 times and increases throughput by 1.5 times. Overall, this research highlights the potential of leveraging NICs for running neural network inference tasks in packet monitoring applications. By minimizing data transfer costs and achieving high-performance results, N3IC demonstrates its ability to enhance network monitoring capabilities efficiently and cost-effectively.