Running summarizing tools on a new article
Check the paper below, then start its synthesis.
Reformulating the Forward-Forward Algorithm with Layer-wise Similarity-based Objectives
James Gong (University of Auckland), Raymond Luo (University of Auckland), Emma Wang (University of Auckland), Leon Ge (University of Auckland), Bruce Li (University of Auckland), Felix Marattukalam (University of Auckland), Waleed Habib Abdulla (University of Auckland)
Abstract: Backpropagation underpins the success of artificial neural networks, yet it has critical limitations such as backward locking and global error propagation. The Forward-Forward algorithm has been proposed as a more biologically plausible method that replaces the backward pass with an additional forward pass. Nevertheless, the Forward-Forward algorithm significantly trails backpropagation in accuracy, and its optimal form requires multiple forward passes at inference time, resulting in low inference efficiency. In this work, the Forward-Forward algorithm is reformulated within a similarity-learning paradigm. The proposed algorithm is named Forward-forward Algorithm Unified with Similarity-based Training (FAUST). FAUST incorporates similarity-based objectives and, by caching estimates of class centroids, achieves single-pass inference. On matched MLP and CNN architectures, FAUST yields improved accuracy over Forward-Forward baselines and narrows the gap to backpropagation across MNIST, Fashion-MNIST, and CIFAR-10. Applying FAUST on ResNet-18 yields competitive results on ImageNette.