# Stream-K: Work-centric Parallel Decomposition for Dense Matrix-Matrix Multiplication on the GPU

**Authors:** Muhammad Osama, Duane Merrill, Cris Cecka, Michael Garland, John D. Owens  
**arXiv:** [2301.03598](https://arxiv.org/abs/2301.03598)  
**Date:** 2023-01-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.03598v1/

## Key points

- Muhammad Osama, Duane Merrill, Cris Cecka, Michael Garland, and John D. Owens introduce the Stream-K methodology for parallelizing matrix multiplication on GPUs.
- Stream-K focuses on distributing inner loop iterations evenly among physical processing elements to maximize resource utilization.
- The parallelization of GEMM using Stream-K shows impressive speedup results compared to existing libraries like CUTLASS and cuBLAS.
- Stream-K's work-centric approach leads to near-optimal processor utilization by efficiently dispatching output tiles across physical cores in waves.
- As processors increase in core count and size, the oversubscription phenomenon diminishes, requiring fewer waves for tile production.

## Summary

, , , , 

In their work, titled "Stream-K: Work-centric Parallel Decomposition for Dense Matrix-Matrix Multiplication on the GPU," Muhammad Osama, Duane Merrill, Cris Cecka, Michael Garland, and John D. Owens introduce a novel approach to parallelizing matrix multiplication (GEMM) and related computations in dense linear algebra. The authors present the Stream-K methodology as a promising solution for optimizing dense linear algebra computations on GPUs through its innovative parallel decomposition strategy. Unlike traditional tile-based decompositions, Stream-K focuses on distributing inner loop iterations evenly among physical processing elements to maximize computing resource utilization. On GPU processors, the Stream-K parallelization of GEMM demonstrates impressive results with a peak speedup of up to 14x and 6.7x compared to state-of-the-art math libraries like CUTLASS and cuBLAS. This stands in contrast to existing libraries that rely on complex kernel-selection heuristics from a large pool of kernel variants. The authors highlight how Stream-K's work-centric approach leads to near-optimal processor utilization by efficiently dispatching output tiles across physical cores in waves. As processors continue to increase in core count and size, the oversubscription phenomenon diminishes, requiring fewer waves for tile production. However, this shift towards larger matrix blocking factors may lead to quantization inefficiency and underutilization of processing resources. Overall, the Stream-K methodology offers a promising solution for optimizing dense linear algebra computations on GPUs through its innovative parallel decomposition strategy. By simplifying the configuration process and improving performance consistency across diverse problem sets, Stream-K presents a compelling alternative for researchers and practitioners seeking efficient parallelization techniques in computational mathematics.