# Fast Inner-Product Algorithms and Architectures for Deep Neural Network Accelerators

**Authors:** Trevor E. Pogue, Nicola Nicolici  
**arXiv:** [2311.12224](https://arxiv.org/abs/2311.12224)  
**Date:** 2023-11-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.12224v1/

## Key points

- Authors Trevor E. Pogue and Nicola Nicolici introduce the Free-pipeline Fast Inner Product (FFIP) algorithm and its corresponding hardware architecture
- FFIP enhances the fast inner-product algorithm (FIP) proposed by Winograd in 1968, boosting clock frequency and improving throughput without significant hardware cost increase
- FFIP is versatile and applicable to various machine learning model layers involving matrix multiplication like fully-connected, convolutional, recurrent, and attention/transformer layers
- Researchers implement FIP within an ML accelerator for the first time, presenting the FFIP algorithm and its generalized architecture
- FFIP seamlessly integrates into traditional fixed-point systolic array ML accelerators, achieving equivalent throughput with half the number of MAC units or doubling maximum systolic array size under a fixed hardware budget
- FFIP implementation targeting non-sparse ML models with 8 to 16-bit fixed-point inputs demonstrates superior throughput and compute efficiency compared to existing state-of-the-art solutions on similar platforms

## Summary

In their paper titled "Fast Inner-Product Algorithms and Architectures for Deep Neural Network Accelerators," authors Trevor E. Pogue and Nicola Nicolici introduce a novel algorithm known as the Free-pipeline Fast Inner Product (FFIP) along with its corresponding hardware architecture. This innovative approach aims to enhance an underexplored fast inner-product algorithm (FIP) originally proposed by Winograd in 1968. The FFIP algorithm is versatile and applicable to various machine learning (ML) model layers that primarily involve matrix multiplication, such as fully-connected, convolutional, recurrent, and attention/transformer layers. The authors implement FIP within an ML accelerator for the first time and subsequently present the FFIP algorithm and its generalized architecture. These advancements inherently boost FIP's clock frequency, leading to improved throughput without significantly increasing hardware costs. Furthermore, the researchers contribute ML-specific optimizations tailored for both the FIP and FFIP algorithms and architectures. One of the key findings of their study is that FFIP can seamlessly integrate into traditional fixed-point systolic array ML accelerators to achieve equivalent throughput with only half the number of multiply-accumulate (MAC) units required. Alternatively, it enables a doubling of the maximum systolic array size that can be accommodated within devices operating under a fixed hardware budget. Moreover, through their FFIP implementation targeting non-sparse ML models featuring 8 to 16-bit fixed-point inputs, Pogue and Nicolici demonstrate superior throughput and compute efficiency compared to existing state-of-the-art solutions on similar compute platforms. Their research not only enhances performance metrics but also provides valuable insights for optimizing deep neural network accelerators in practical applications.