# MixFormer: End-to-End Tracking with Iterative Mixed Attention

**Authors:** Yutao Cui, Cheng Jiang, Gangshan Wu, Limin Wang  
**arXiv:** [2302.02814](https://arxiv.org/abs/2302.02814)  
**Date:** 2023-02-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.02814v2/

## Key points

- MixFormer framework introduces a new approach to visual object tracking combining feature extraction with target information integration
- Utilizes attention operations to propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration
- Two types of MixFormer trackers: hierarchical tracker called MixCvT and non-hierarchical tracker known as MixViT
- Various pre-training methods explored including supervised, self-supervised, and masked techniques
- Asymmetric attention scheme in MAM reduces computational costs when handling multiple target templates during online tracking
- Effective score prediction module proposed to select high-quality templates
- Performance of MixFormer trackers surpasses existing benchmarks on seven tracking datasets

## Summary

, , , , 

The MixFormer framework introduces a new approach to visual object tracking that combines feature extraction with target information integration. It utilizes attention operations to propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration, allowing for the extraction of target-specific discriminative features and extensive communication between the target and search area. Two types of MixFormer trackers are instantiated, including a hierarchical tracker called MixCvT and a non-hierarchical tracker known as MixViT. The paper explores various pre-training methods for these trackers, including supervised, self-supervised, and masked pre-training techniques. Additionally, an asymmetric attention scheme is devised in MAM to reduce computational costs when handling multiple target templates during online tracking. An effective score prediction module is also proposed to select high-quality templates. The performance of MixFormer trackers surpasses existing benchmarks on seven tracking datasets, demonstrating its state-of-the-art performance in visual object tracking. In related work analysis, prevailing tracking methods typically employ a three-stage architecture consisting of a backbone for feature extraction, an integration module for combining target and search region information, and classification/localization heads for determining target states. Siamese-based trackers have gained popularity due to their simplicity and efficiency in modeling appearance similarity between targets and search areas. On the other hand, another family of trackers focuses on learning online target-dependent discriminative models through end-to-end training approaches such as CFNet and DiMP.