# TableFormer: Table Structure Understanding with Transformers

**Authors:** Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, Peter Staar  
**arXiv:** [2203.01017](https://arxiv.org/abs/2203.01017)  
**Date:** 2022-03-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.01017v2/

## Key points

- Tables are crucial for organizing content in a concise manner and enhancing predictive capabilities of systems like search engines and Knowledge Graphs.
- Identifying the structure of tables from images is challenging due to various shapes, sizes, and complexities.
- A new table-structure identification model with object detection decoder for table cells has been introduced to improve existing deep learning models.
- The model allows accurate extraction of table content directly from programmatic PDFs without custom OCR decoders, enhancing accuracy and enabling non-English table handling.
- The proposed TableFormer model outperforms existing methods by leveraging tree-editing-distance-score on simple and complex tables.
- Post-processing techniques extract cell content by matching predicted bounding boxes to PDF cells based on overlap and spatial proximity.
- "SynthTabNet" dataset addresses missing characteristics in other datasets, valuable for future research in document understanding and table extraction.

## Summary

, , , , 

Tables play a crucial role in organizing valuable content in a concise and compact manner, enhancing the predictive capabilities of systems such as search engines and Knowledge Graphs. However, tables come in various shapes and sizes, with complex configurations like multi-line rows, different separation lines, and missing entries. Identifying the structure of a table from an image is a challenging task. In this paper, a new table-structure identification model is introduced to improve upon existing deep learning models. The new model incorporates a novel object detection decoder for table cells, allowing for accurate extraction of table content directly from programmatic PDFs without the need for custom OCR decoders. This architectural change enhances table-content extraction accuracy and enables the handling of non-English tables. has greatly advanced document understanding by improving table extraction from documents through addressing challenges related to locating tables on document pages and determining their structure. While table-location has been effectively solved using object-detection networks like YOLO and Mask-RCNN, table-structure decomposition remains a longstanding problem in document understanding. In this study, we propose an innovative solution called TableFormer that overcomes limitations present in current approaches while advancing the field of document understanding through improved  Our approach is language agnostic and efficiently leverages data from original PDF documents while establishing direct links between table cells and their bounding boxes in images. The proposed model outperforms existing state-of-the-art methods by a wide margin thanks to its use of  which have shown significant improvements in tree-editing-distance-score on both simple and complex tables. Qualitative analysis showcases the model's ability to predict bounding boxes for all table cells, including empty ones. Post-processing techniques extract cell content by matching predicted bounding boxes to PDF cells based on overlap and spatial proximity. One of the key contributions of this study is the introduction of a  called "SynthTabNet," which addresses missing characteristics present in other datasets. This dataset will be valuable for future research in document understanding and table extraction. In conclusion, our research team has developed an end-to-end transformer-based approach for predicting table structures and cell bounding boxes from images.