# Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

**Authors:** Mehdi Noroozi, Paolo Favaro  
**arXiv:** [1603.09246](https://arxiv.org/abs/1603.09246)  
**Date:** 2016-03-30  
**Summary page:** https://www.summarizepaper.com/en/paper/1603.09246v3/

## Key points

- The paper explores image representation learning without human annotation
- A convolutional neural network (CNN) is proposed to solve Jigsaw puzzles as a pretext task, which requires no manual labeling
- The context-free network (CFN), a siamese-ennead CNN, is introduced to maintain compatibility across tasks and limit the receptive field of its early processing units to one tile at a time
- By training the CFN to solve Jigsaw puzzles, both feature mapping of object parts and their correct spatial arrangement are learned
- The learned features capture semantically relevant content and outperform state-of-the-art methods in several transfer learning benchmarks
- The approach is based on self-supervision principles, enabling learning from unlabeled data without requiring any human annotation
- Noroozi and Favaro's work presents an innovative solution for unsupervised image representation learning using Jigsaw puzzles as a pretext task
- Their proposed CFN architecture offers promising results for solving object classification and detection tasks while requiring minimal human intervention in the training process.

## Summary

In their paper titled "Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles," Mehdi Noroozi and Paolo Favaro explore the problem of image representation learning without human annotation. They propose a convolutional neural network (CNN) that can be trained to solve Jigsaw puzzles as a pretext task, which requires no manual labeling, and then later repurposed to solve object classification and detection. To maintain compatibility across tasks, they introduce the context-free network (CFN), a siamese-ennead CNN that takes image tiles as input and explicitly limits the receptive field of its early processing units to one tile at a time. The authors show that the CFN includes fewer parameters than AlexNet while preserving the same semantic learning capabilities. By training the CFN to solve Jigsaw puzzles, they learn both a feature mapping of object parts as well as their correct spatial arrangement. Their experimental evaluations demonstrate that the learned features capture semantically relevant content. Moreover, their proposed method for learning visual representations outperforms state-of-the-art methods in several transfer learning benchmarks. The authors' approach is based on self-supervision principles, which enables them to learn from unlabeled data without requiring any human annotation. This makes their method highly scalable and applicable to various domains where labeled data is scarce or expensive. Overall, Noroozi and Favaro's work presents an innovative solution for unsupervised image representation learning using Jigsaw puzzles as a pretext task. Their proposed CFN architecture offers promising results for solving object classification and detection tasks while requiring minimal human intervention in the training process.