# Can You Read Me Now? Content Aware Rectification using Angle Supervision

**Authors:** Amir Markovitz, Inbal Lavi, Or Perel, Shai Mazor, Roee Litman  
**arXiv:** [2008.02231](https://arxiv.org/abs/2008.02231)  
**Date:** 2020-08-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2008.02231v1/

## Key points

- The ubiquity of smartphone cameras has revolutionized document capture
- Photographed documents often have folds and crumples, causing local variance in text structure
- OCR systems rely on rectifying geometric distortions for accurate recognition
- Previous approaches to rectify document images focus on global features, overlooking content signals
- CREASE is a learned approach for document rectification that leverages the document's content as hints
- CREASE employs pixel-wise angle regression and curvature estimation to optimize the rectification model
- CREASE outperforms previous approaches in OCR accuracy, geometric error, and visual similarity
- This advancement improves OCR accuracy and usability of smartphone-captured documents.

## Summary

The ubiquity of smartphone cameras has revolutionized the way documents are captured, with more and more people opting to photograph documents rather than scan them. However, unlike flatbed scanners, photographed documents often exhibit folds and crumples which lead to significant local variance in text structure. This poses a challenge for Optical Character Recognition (OCR) systems as the accuracy of recognition is heavily influenced by the ability to rectify geometric distortions in the document. While OCR systems have made great strides in recent years, most still rely on a pre-processing step that ensures straight and axis-aligned text lines. Previous works have attempted to rectify document images taken in real-world conditions using various supervision signals and alignment techniques; however these approaches primarily focus on global features extracted from the document's boundaries overlooking valuable signals that could be derived from the document's content. To address this limitation, we introduce CREASE: Content Aware Rectification using Angle Supervision. Our method is the first learned approach for document rectification that leverages the content of the document itself including word location and orientation as hints to assist in the rectification process. We employ a novel pixel-wise angle regression approach and incorporate a curvature estimation side-task to optimize our rectification model. Our method outperforms previous approaches in terms of OCR accuracy, geometric error and visual similarity. By considering both global features from the document's boundaries and local signals obtained from its content CREASE achieves superior performance in rectifying documents captured under challenging conditions. This advancement has significant implications for improving OCR accuracy and enhancing usability of smartphone-captured documents. The authors of this study include Amir Markovitz, Inbal Lavi, Or Perel, Shai Mazor and Roee Litman; it was presented at ECCV 2020 conference.