# Efficient Video Classification Using Fewer Frames

**Authors:** Shweta Bhardwaj, Mukundhan Srinivasan, Mitesh M. Khapra  
**arXiv:** [1902.10640](https://arxiv.org/abs/1902.10640)  
**Date:** 2019-02-27  
**Summary page:** https://www.summarizepaper.com/en/paper/1902.10640v1/

## Key points

- The prevalence of video content in the digital age has impacted various aspects of our lives
- Automatic video processing tasks have gained interest, including activity identification, textual description generation, summarization, and question answering
- Balancing the ability to learn from large datasets with the need for low-power device compatibility is crucial
- The ECCV workshop on YouTube-8M focused on developing memory-efficient models using less than 1GB of memory
- Knowledge distillation involves training a teacher network with many parameters to guide a smaller student network for efficient inference
- Researchers at IIT Madras and Robert Bosch Centre propose a compute-efficient video classification approach using a teacher model to train a student model focusing on fewer frames
- This method aims to reduce FLOPs while maintaining performance levels by up to 30% faster inference time and approximately 90% fewer FLOPs

## Summary

In today's digital age, the prevalence of video content on the internet has significantly impacted various aspects of our lives. From education and entertainment to communication and beyond, the surge in video content has sparked a growing interest in automatic video processing tasks. These include activity identification, textual description generation, summarization, question answering, and more. With the availability of large-scale datasets for training complex models with high memory and computational requirements, there is a need to balance the ability to learn from vast amounts of data with the demand for running these models on low-power devices like mobile phones and tablets. These devices have strict constraints on latency, memory usage, and computational costs. To address this challenge, the recent ECCV workshop on YouTube-8M Large-Scale Video Understanding (2018) focused on developing memory-efficient models that utilize less than 1GB of memory. The workshop aimed to discourage ensemble-based methods in favor of single models that are more efficient in terms of memory usage. One approach explored by participants was knowledge distillation - first training a teacher network with a large number of parameters and then using this network to guide a smaller student network with limited memory requirements for efficient inference. This not only reduces memory usage but also decreases floating-point operations (FLOPs) due to smaller weight matrices and hidden representations. Building upon the findings from the ECCV workshop, researchers at Indian Institute of Technology Madras and Robert Bosch Centre for Data Science and AI propose an innovative approach for compute-efficient video classification. They use a compute-heavy teacher model that analyzes all frames in a video to train a compute-efficient student model that focuses on only a small fraction of frames. This novel method aims to reduce FLOPs while maintaining performance levels by up to 30% faster inference time and approximately 90% fewer FLOPs without compromising accuracy. In conclusion, this research contributes to the ongoing efforts towards developing compute-efficient video classification models that strike a balance between learning from extensive data sources and being cost-effective during inference on resource-constrained devices like mobile phones and tablets.