# InternVideo: General Video Foundation Models via Generative and Discriminative Learning

**Authors:** Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, Sen Xing, Guo Chen, Junting Pan, Jiashuo Yu, Yali Wang, Limin Wang, Yu Qiao  
**arXiv:** [2212.03191](https://arxiv.org/abs/2212.03191)  
**Date:** 2022-12-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.03191v1/

## Key points

- Introduction of a novel video foundation model, InternVideo, combining generative and discriminative self-supervised video learning techniques
- Focus on masked video modeling and video-language contrastive learning as pretraining objectives
- Achieves state-of-the-art performance on 39 video datasets across various tasks such as action recognition/detection, video-language alignment, and open-world applications
- Efficiency in training with significantly less power consumption compared to other models like CoCa
- Leveraging a unified video representation based on cross-model learning for generalized spatiotemporal representations
- Consistently delivers notable performance improvements even in zero-shot and open-set scenarios
- Potential limitations in handling long-term or high-order video tasks like anticipating plots from movie segments
- Future work could focus on expanding capabilities to address these challenges and enhance generality of video representation learning
- Emphasis on ethical considerations in data collection for training models like InternVideo, ensuring responsible use of curated datasets
- Importance of exploring issues related to bias, risks, fairness, equality, and other social topics for comprehensive impact assessment in the field of AI research

## Summary

In this paper, we introduce , a novel video foundation model that combines generative and discriminative self-supervised video learning techniques to enhance video-level understanding tasks. Unlike existing vision foundation models that primarily focus on image-level pretraining and adaptation,  explores masked video modeling and video-language contrastive learning as pretraining objectives. By selectively coordinating the representations from these two frameworks in a learnable manner,  achieves state-of-the-art performance on 39 video datasets across various tasks such as action recognition/detection, video-language alignment, and open-world applications. stands out for its efficiency in training, consuming significantly less power compared to other models like CoCa. The model leverages a unified video representation based on cross-model learning, leading to generalized spatiotemporal representations with impressive results across different datasets. Even in zero-shot and open-set scenarios,  consistently delivers notable performance improvements. While  excels in current popular video perception tasks using clips, it may face limitations when handling long-term or high-order video tasks like anticipating plots from movie segments. Future work could focus on expanding the model's capabilities to address these challenges and further enhance the generality of video representation learning. Moreover, ethical considerations are paramount in data collection for training models like . Queries used for gathering data are carefully vetted for ethical and legal compliance to ensure responsible use of curated datasets. As research progresses in the field of video understanding, it is essential to explore issues related to bias, risks, fairness, equality, and other social topics for a comprehensive impact assessment. Overall,< kd>InternVideo</kd> represents a significant advancement in the realm of video foundation models by achieving superior performance across diverse tasks while maintaining efficiency in training. Its success underscores the potential for further innovation in enhancing video understanding capabilities and addressing broader societal implications within the field of AI research.