# VL-JEPA: Joint Embedding Predictive Architecture for Vision-language

**Authors:** Delong Chen, Mustafa Shukor, Theo Moutakanni, Willy Chung, Jade Yu, Tejaswi Kasarla, Allen Bolourchi, Yann LeCun, Pascale Fung  
**arXiv:** [2512.10942](https://arxiv.org/abs/2512.10942)  
**Date:** 2025-12-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2512.10942v1/

## Key points

- VL-JEPA is a novel vision-language model operating on a Joint Embedding Predictive Architecture (JEPA) that predicts continuous embeddings of target texts in an abstract representation space.
- The model focuses on task-relevant semantics while abstracting away surface-level linguistic variability, improving efficiency and real-time response requirements.
- VL-JEPA supports selective decoding, reducing operations by 2.85x without sacrificing performance compared to non-adaptive uniform decoding.
- The model's embedding space facilitates open-vocabulary classification, text-to-video retrieval, and discriminative Visual Question Answering (VQA) without architectural modifications.
- VL-JEPA combines the strengths of CLIP and VLMs by leveraging web-scale noisy image-text pairs for open-domain features and supporting conditional generation tasks with a readout text decoder.
- Efforts to improve efficiency in Vision-Language Models include updating only a subset of parameters during training, exploring methods like parameter pruning or token reduction for inference efficiency, and using small VLMs and heuristics for real-time applications.

## Summary

, , , , 


In this study, the authors introduce VL-JEPA, a novel vision-language model that operates on a Joint Embedding Predictive Architecture (JEPA). Unlike traditional Vision-Language Models (VLMs) that generate tokens sequentially, VL-JEPA predicts continuous embeddings of target texts in an abstract representation space. This approach allows the model to focus on task-relevant semantics while abstracting away surface-level linguistic variability. <break>
<break>

To improve efficiency and real-time response requirements, VL-JEPA utilizes a lightweight text decoder only when necessary for translation. The model also supports selective decoding, reducing the number of operations by 2.85x without sacrificing performance compared to non-adaptive uniform decoding. Additionally, VL-JEPA's embedding space naturally facilitates open-vocabulary classification, text-to-video retrieval, and discriminative Visual Question Answering (VQA) without requiring any architectural modifications. <break>
<break>

The proposed VL-JEPA combines the strengths of both CLIP and VLMs by leveraging web-scale noisy image-text pairs for strong open-domain features while supporting conditional generation tasks with a readout text decoder. Furthermore, VL-JEPA is more efficient at learning in latent space compared to generative VLMs that optimize directly in data space. <break>
<break>

Efforts to improve efficiency in Vision-Language Models include updating only a subset of parameters during training and exploring methods such as parameter pruning or token reduction for inference efficiency. Real-time applications benefit from small VLMs and heuristics to reduce query frequency during asynchronous inference. <break>
<break>

Overall, the introduction of VL-JEPA represents a significant advancement in vision-language modeling by addressing issues related to cost-effectiveness, real-time response requirements, and task coverage across various applications such as captioning, retrieval, visual question answering, action tracking, reasoning, and planning.