# NoProp: Training Neural Networks without Back-propagation or Forward-propagation

**Authors:** Qinyu Li, Yee Whye Teh, Razvan Pascanu  
**arXiv:** [2503.24322](https://arxiv.org/abs/2503.24322)  
**Date:** 2025-03-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2503.24322v1/

## Key points

- Paper introduces NoProp, a novel learning method diverging from traditional deep learning
- NoProp does not use back-propagation or forward-propagation; inspired by diffusion and flow matching methods
- Each layer in NoProp independently learns to denoise a noisy target without hierarchical representations
- Demonstrated effectiveness on image classification benchmarks like MNIST, CIFAR-10, and CIFAR-100 with superior accuracy
- Training involves modeling diffusion dynamics using neural networks with separate embedding pathways for input images and latent variables
- Enables efficient credit assignment within the network without relying on traditional gradient-based methods
- Offers more efficient distributed learning and potential impact on various aspects of the learning process
- Presents a promising direction in gradient-free learning methods as an alternative to back-propagation with superior performance and computational efficiency

## Summary

The paper titled "NoProp: Training Neural Networks without Back-propagation or Forward-propagation" introduces a novel learning method called NoProp that diverges from the traditional deep learning approach. In standard deep learning, gradients are computed at each layer through back-propagation, leading to hierarchical representations where abstract features reside in higher layers. However, NoProp does not rely on forward or backward propagation; instead, it draws inspiration from diffusion and flow matching methods. In NoProp, each layer independently learns to denoise a noisy target without the need for hierarchical representations. The model fixes the representation at each layer beforehand to a noised version of the target, enabling a local denoising process that can be leveraged during inference. The effectiveness of NoProp is demonstrated on popular image classification benchmarks like MNIST, CIFAR-10, and CIFAR-100, showcasing superior accuracy compared to existing back-propagation-free methods. The training procedure involves using neural networks to model diffusion dynamics with separate embedding pathways for input images and latent variables. This approach allows for efficient credit assignment within the network without relying on traditional gradient-based methods. By departing from the conventional learning paradigm, NoProp enables more efficient distributed learning and has the potential to impact various aspects of the learning process. Overall, NoProp presents a promising direction in gradient-free learning methods by offering a viable alternative to back-propagation while achieving superior performance and computational efficiency. The paper's findings suggest that NoProp could pave the way for new approaches in neural network training that do not conform to traditional hierarchical representations.