# BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

**Authors:** Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi  
**arXiv:** [2201.12086](https://arxiv.org/abs/2201.12086)  
**Date:** 2022-01-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.12086v2/

## Key points

- Recent advancements in improving performance of vision-language tasks
- Existing pre-trained models excel in understanding-based or generation-based tasks, but not both
- Introduction of BLIP, a new VLP framework for flexible transfer learning
- BLIP utilizes noisy web data through a two-step process involving a captioner and a filter
- Model architecture enables multi-task pre-training and flexible transfer learning
- Pre-training with three vision-language objectives: image-text contrastive learning, image-text matching, and image-conditioned language modeling
- Dataset boosting technique involves generating synthetic captions and filtering out noisy ones
- Improves performance on vision-language tasks such as image-text retrieval, image captioning, and VQA
- Strong generalization ability in zero-shot transfer to video-language tasks
- Introduces the model architecture and dataset boosting technique as key contributions
- Code, models, and datasets are publicly available for further research and development.

## Summary

In recent years,  has made significant advancements in improving the performance of vision-language tasks. However, most existing pre-trained models excel in either understanding-based tasks or generation-based tasks, but not both. To address these limitations, this paper introduces . BLIP is a new VLP framework that offers flexible transfer learning to both vision-language understanding and generation tasks. It effectively utilizes the noisy web data by employing a two-step process involving a captioner and a filter. The  model architecture enables effective multi-task pre-training and flexible transfer learning by operating as a unimodal encoder, an image-grounded text encoder, or an image-grounded text decoder. It is jointly pre-trained with three vision-language objectives: image-text contrastive learning, image-text matching, and image-conditioned language modeling. The  dataset boosting technique involves generating synthetic captions using the captioner and then filtering out noisy captions using the filter. This process ensures that the noisy web data is effectively utilized while maintaining high-quality supervision. Overall, significantly improves performance on a wide range of vision-language tasks such as image-text retrieval, image captioning, and visual question answering (VQA). Moreover, demonstrates strong generalization ability when directly transferred to video-language tasks in a zero-shot manner. This means that it can perform well on video-related vision-language tasks without any additional training or fine-tuning. In conclusion,  introduces two key contributions: the  model architecture and the  dataset boosting technique. These advancements allow for more effective pre-training and flexible transfer learning in vision-language tasks. The code, models, and datasets are publicly available for further research and development.