# Putting the Object Back into Video Object Segmentation

**Authors:** Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, Alexander Schwing  
**arXiv:** [2310.12982](https://arxiv.org/abs/2310.12982)  
**Date:** 2023-10-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.12982v1/

## Key points

- Cutie is a video object segmentation (VOS) network that aims to improve the performance of VOS algorithms.
- It incorporates object-level memory reading instead of bottom-up pixel-level memory reading.
- Cutie uses a top-down approach with object queries and a query-based object transformer.
- Object queries serve as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation.
- Cutie effectively separates the semantics of the foreground object from the background using foreground-background masked attention.
- It is compared against other VOS algorithms on challenging datasets such as MOSE and BURST.
- On the MOSE dataset, Cutie achieves an improvement of 8.7 J&F over XMem and 4.2 J&F over DeAOT while running faster.
- Cutie provides code availability for further exploration and implementation.
- The authors emphasize that their approach addresses limitations in existing VOS methods and offers promising results in terms of accuracy and efficiency.

## Summary

Cutie is a video object segmentation (VOS) network that aims to improve the performance of VOS algorithms by incorporating object-level memory reading. Unlike existing approaches that use bottom-up pixel-level memory reading, which can be affected by matching noise and distractors, Cutie adopts a top-down approach by utilizing a small set of object queries to interact with the bottom-up pixel features through a query-based object transformer. The object queries in Cutie serve as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation. By combining this top-down memory reading strategy with foreground-background masked attention, Cutie effectively separates the semantics of the foreground object from the background. To evaluate its performance, Cutie is compared against other VOS algorithms on challenging datasets such as MOSE and BURST. On the MOSE dataset, Cutie achieves an improvement of 8.7 J&F over XMem with similar running time and 4.2 J&F over DeAOT while running three times faster. In addition to its superior performance, Cutie also provides code availability for further exploration and implementation. The authors emphasize that their approach addresses some limitations present in existing VOS methods and offers promising results in terms of accuracy and efficiency. Overall, Cutie presents a novel approach to video object segmentation by incorporating object-level memory reading and demonstrates significant improvements in challenging scenarios.