# DARKSIDE: A Heterogeneous RISC-V Compute Cluster for Extreme-Edge On-Chip DNN Inference and Training

**Authors:** Angelo Garofalo, Yvan Tortorella, Matteo Perotti, Luca Valente, Alessandro Nadalini, Luca Benini, Davide Rossi, Francesco Conti  
**arXiv:** [2303.17954](https://arxiv.org/abs/2303.17954)  
**Date:** 2023-03-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.17954v1/

## Key points

- Machine Learning (ML) and Deep Learning (DL) models are being deployed at the extreme edge of the Internet-of-Things (IoT), known as Tiny Machine Learning (TinyML).
- Heterogeneous clusters have emerged as promising solutions to meet the strict latency, throughput, accuracy, and flexibility requirements for on-chip DNN inference and training at this level.
- DARKSIDE is presented as a System-on-Chip with a heterogeneous cluster consisting of 8 RISC-V cores enhanced with 2-b to 32-b mixed-precision integer arithmetic.
- Three digital accelerators are added to the cluster: a specialized engine for low-data-reuse depthwise convolution kernels capable of up to 30 MAC/cycle; a minimal overhead datamover that marshals 1-b to 32-b data on-the-fly; and a 16-b floating-point Tensor Product Engine (TPE) for tiled matrix multiplication acceleration.
- DARKSIDE achieves peak integer performance of 65 GOPS and peak efficiency of 835 GOPS/W when working on 2-b integer DNN kernels.
- The TPE provides up to 18.2 GFLOPS of performance or 300 GFLOPS/W of efficiency when targeting floating point tensor operations.
- The authors argue that using heterogeneous clusters like DARKSIDE that offer both flexibility and performance boosts through dedicated accelerators is better than catch instructions in certain scenarios.
- Overall, DARKSIDE presents a promising solution for on chip DNN inference and training at the extreme edge of TinyML, with its heterogeneous cluster offering both flexibility and performance boosts through dedicated accelerators.

## Summary

The field of Machine Learning (ML) and Deep Learning (DL) has seen a recent surge in the deployment of models at the extreme edge of the Internet-of-Things (IoT), commonly referred to as Tiny Machine Learning (TinyML). To meet the strict latency, throughput, accuracy, and flexibility requirements for on-chip DNN inference and training at this level, heterogeneous clusters have emerged as promising solutions. These clusters combine the flexibility of DSP-enhanced cores with the performance and energy boost of dedicated accelerators. In this context, DARKSIDE is presented as a System-on-Chip with a heterogeneous cluster consisting of 8 RISC-V cores enhanced with 2-b to 32-b mixed-precision integer arithmetic. To enhance performance and efficiency on key compute-intensive DNN kernels, three digital accelerators are added to the cluster: a specialized engine for low-data-reuse depthwise convolution kernels capable of up to 30 MAC/cycle; a minimal overhead datamover that marshals 1-b to 32-b data on-the-fly; and a 16-b floating-point Tensor Product Engine (TPE) for tiled matrix multiplication acceleration. DARKSIDE is implemented using 65nm CMOS technology and achieves peak integer performance of 65 GOPS and peak efficiency of 835 GOPS/W when working on 2-b integer DNN kernels. When targeting floating point tensor operations, the TPE provides up to 18.2 GFLOPS of performance or 300 GFLOPS/W of efficiency - enough to enable on chip floating point training at competitive speed coupled with ultra low power quantized inference. The authors argue that while catch instructions can improve energy efficiency by reducing memory accesses in certain scenarios, they may not be sufficient for all use cases. Instead they propose using heterogeneous clusters like DARKSIDE that offer both flexibility and performance boosts through dedicated accelerators. Overall, DARKSIDE presents a promising solution for on chip DNN inference and training at the extreme edge of TinyML, with its heterogeneous cluster offering both flexibility and performance boosts through dedicated accelerators. Its implementation in 65nm CMOS technology demonstrates impressive peak integer performance and efficiency when working on 2 b integer DNN kernels while its TPE provides sufficient performance and efficiency to enable on chip floating point training coupled with ultra low power quantized inference.