# Barlow Twins: Self-Supervised Learning via Redundancy Reduction

**Authors:** Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, Stéphane Deny  
**arXiv:** [2103.03230](https://arxiv.org/abs/2103.03230)  
**Date:** 2021-03-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2103.03230v1/

## Key points

- Barlow Twins is a self-supervised learning method that aims to learn invariant representations of input samples.
- The approach avoids trivial constant representations, which have been an issue with similar methods in the past.
- The objective function measures cross-correlation between outputs of two identical networks fed with distorted versions of a sample and makes it as close to the identity matrix as possible.
- The method allows for high-dimensional output vectors and does not require large batches or asymmetry between network twins.
- Barlow Twins has shown promising results in semi-supervised classification tasks on ImageNet in low-data regimes and is on par with current state-of-the-art methods for ImageNet classification and transfer tasks.
- The authors behind this research are Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny.

## Summary

Barlow Twins is a self-supervised learning method that aims to learn representations that are invariant to distortions of the input sample. The approach is designed to avoid trivial constant representations, which have been a recurring issue with similar methods in the past. The proposed objective function measures the cross-correlation matrix between the outputs of two identical networks fed with distorted versions of a sample and makes it as close to the identity matrix as possible. This causes the representation vectors of distorted versions of a sample to be similar while minimizing redundancy between their components. The method is named after neuroscientist H. Barlow's redundancy-reduction principle applied to a pair of identical networks. Unlike other self-supervised learning methods, Barlow Twins does not require large batches or asymmetry between network twins such as a predictor network, gradient stopping, or moving average on weight updates. It allows for the use of very high-dimensional output vectors. Barlow Twins has shown promising results in semi-supervised classification tasks on ImageNet in low-data regimes and is on par with current state-of-the-art methods for ImageNet classification with a linear classifier head and transfer tasks of classification and object detection. The authors behind this research are Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. With its ability to avoid trivial constant representations and achieve impressive results across various computer vision benchmarks, Barlow Twins represents an exciting development in self-supervised learning techniques.