# Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

**Authors:** Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, Nicolas Ballas  
**arXiv:** [2301.08243](https://arxiv.org/abs/2301.08243)  
**Date:** 2023-01-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.08243v1/

## Key points

- Proposed method: Image-based Joint-Embedding Predictive Architecture (I-JEPA)
- Eliminates the need for hand-crafted data augmentations
- Key idea: Predict multiple target blocks within an image from a single context block
- Predicting multiple target blocks in the image at a large scale (15%-20% of the image)
- Using an informative context block that is spatially distributed
- Empirical results show high scalability when combined with Vision Transformers
- Example: Training ViT-Huge/16 model on ImageNet using 32 A100 GPUs takes less than 38 hours and achieves strong performance on various tasks
- Offers a promising avenue for self-supervised learning from images, generating highly semantic representations without manual data augmentations
- Significant potential for advancing computer vision research and applications

## Summary

Their proposed method, the Image-based Joint-Embedding Predictive Architecture (I-JEPA), eliminates the need for hand-crafted data augmentations to learn highly semantic image representations. The key idea behind I-JEPA is to predict multiple target blocks within an image from a single context block. To ensure the production of meaningful representations, the authors highlight three crucial design choices: predicting multiple target blocks in the image at a large scale (15%-20% of the image) and using an informative context block that is spatially distributed. Empirical results show that when combined with Vision Transformers, I-JEPA exhibits high scalability. For example, training a ViT-Huge/16 model on ImageNet using 32 A100 GPUs takes less than 38 hours and achieves strong performance on various tasks such as linear classification, object counting, and depth prediction. Overall, I-JEPA offers a promising avenue for self-supervised learning from images by generating highly semantic representations without relying on manual data augmentations. This approach has significant potential for advancing computer vision research and applications in diverse domains.