# ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation

**Authors:** Yufei Xu, Jing Zhang, Qiming Zhang, Dacheng Tao  
**arXiv:** [2204.12484](https://arxiv.org/abs/2204.12484)  
**Date:** 2022-04-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.12484v3/

## Key points

- ViTPose is a new baseline model for human pose estimation that uses plain and non-hierarchical vision transformers as backbones to extract features for a given person instance.
- The model demonstrates the surprisingly good capabilities of plain vision transformers for pose estimation from various aspects, including simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models.
- ViTPose can be scaled up from 100M to 1B parameters by taking advantage of the scalable model capacity and high parallelism of transformers, setting a new Pareto front between throughput and performance.
- The authors show that ViTPose is very flexible regarding attention type, input resolution, pre-training and fine-tuning strategy as well as dealing with multiple pose tasks.
- Experimental results show that their basic ViTPose model outperforms representative methods on the challenging MS COCO Keypoint Detection benchmark while the largest model sets a new state-of-the-art.
- ViTPose's lightweight decoder allows it to achieve excellent performance while maintaining simplicity in its design.
- The authors highlight how little effort has been made to reveal the potential of such simple structures for pose estimation tasks despite their excellent performance in visual recognition tasks.
- Code and models are available at https://github.com/ViTAE-Transformer/ViTPose.

## Summary

ViTPose is a new baseline model for human pose estimation that employs plain and non-hierarchical vision transformers as backbones to extract features for a given person instance. The model demonstrates the surprisingly good capabilities of plain vision transformers for pose estimation from various aspects, including simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models. ViTPose can be scaled up from 100M to 1B parameters by taking advantage of the scalable model capacity and high parallelism of transformers, setting a new Pareto front between throughput and performance. The authors show that ViTPose is very flexible regarding attention type, input resolution, pre-training and fine-tuning strategy as well as dealing with multiple pose tasks. They also empirically demonstrate that the knowledge of large ViTPose models can be easily transferred to small ones via a simple knowledge token. Experimental results show that their basic ViTPose model outperforms representative methods on the challenging MS COCO Keypoint Detection benchmark while the largest model sets a new state-of-the-art. Moreover, ViTPose's lightweight decoder allows it to achieve excellent performance while maintaining simplicity in its design. The authors also highlight how little effort has been made to reveal the potential of such simple structures for pose estimation tasks despite their excellent performance in visual recognition tasks. Overall, this paper presents an exciting development in human pose estimation research by demonstrating the effectiveness of plain vision transformers as backbones for feature extraction. The code and models are available at https://github.com/ViTAE-Transformer/ViTPose.