# SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition

**Authors:** Mingxin Huang, Yuliang Liu, Zhenghao Peng, Chongyu Liu, Dahua Lin, Shenggao Zhu, Nicholas Yuan, Kai Ding, Lianwen Jin  
**arXiv:** [2203.10209](https://arxiv.org/abs/2203.10209)  
**Date:** 2022-03-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.10209v1/

## Key points

- End-to-end scene text spotting is gaining attention for combining scene text detection and recognition
- Current methods merge detection and recognition by sharing a backbone, limiting feature interaction
- SwinTextSpotter introduces a novel framework using a transformer encoder for detection and Recognition Conversion mechanism for unifying localization and recognition
- SwinTextSpotter eliminates the need for rectification modules or character-level annotations for arbitrarily-shaped text
- Demonstrated effectiveness through experiments on various datasets, outperforming existing methods significantly
- Ablation studies show the impact of key components like Recognition Conversion in improving performance
- SwinTextSpotter enhances feature interaction between detection and recognition tasks, presenting a promising approach to scene text spotting
- The framework's simplicity and effectiveness make it valuable for researchers and practitioners in the field

## Summary

In recent years, end-to-end scene text spotting has gained significant attention for its ability to leverage the synergy between scene text detection and recognition. However, current state-of-the-art methods typically merge detection and recognition by sharing a backbone, which fails to fully exploit the feature interaction between the two tasks. To address this limitation, a novel end-to-end scene text spotting framework called SwinTextSpotter is introduced in this paper. By utilizing a transformer encoder with dynamic head as the detector and incorporating a Recognition Conversion mechanism, the framework unifies text localization and recognition through explicit guidance via recognition loss. This streamlined design eliminates the need for additional rectification modules or character-level annotations for arbitrarily-shaped text. The effectiveness of SwinTextSpotter is demonstrated through qualitative and quantitative experiments on various datasets including RoIC13, ICDAR 2015, Total-Text, CTW1500, ReCTS (Chinese), and VinText (Vietnamese). The results show that SwinTextSpotter outperforms existing methods significantly. Additionally, ablation studies conducted on Total-Text using ResNet-50 as the baseline backbone confirm the impact of key components such as Recognition Conversion in improving both detection and end-to-end scene text spotting performance. Overall, SwinTextSpotter presents a promising approach to scene text spotting by enhancing feature interaction between detection and recognition tasks. The framework's simplicity and effectiveness make it a valuable tool for researchers and practitioners in the field. Further exploration of its limitations and potential improvements could lead to even more robust performance in real-world applications.