# RelTR: Relation Transformer for Scene Graph Generation

**Authors:** Yuren Cong, Michael Ying Yang, Bodo Rosenhahn  
**arXiv:** [2201.11460](https://arxiv.org/abs/2201.11460)  
**Date:** 2022-01-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.11460v1/

## Key points

- The paper presents a novel approach to scene graph generation called RelTR
- RelTR is an end-to-end model that adopts an encoder-decoder architecture
- The encoder component uses visual feature context to reason about relationships between objects in the scene
- The decoder component infers a fixed-size set of triplets using attention mechanisms with subject and object queries
- RelTR is trained using a set prediction loss that matches ground truth triplets with predicted ones
- RelTR is a one-stage method that predicts relationships solely based on visual appearance without combining entities or labeling all possible predicates
- Extensive experiments on Visual Genome and Open Images V6 datasets show that RelTR achieves superior performance in terms of accuracy
- RelTR also offers fast inference capabilities

## Summary

The paper titled "RelTR: Relation Transformer for Scene Graph Generation" presents a novel approach to scene graph generation. In a scene, different objects are related to each other in varying degrees and the authors propose an end-to-end model called RelTR which adopts an encoder-decoder architecture. The encoder component of RelTR leverages visual feature context to reason about the relationships between objects in the scene while the decoder component infers a fixed-size set of triplets consisting of subject-predicate-object using various attention mechanisms with coupled subject and object queries. To train this model, the authors design a set prediction loss that matches the ground truth triplets with the predicted ones. One notable aspect of RelTR is that it is a one-stage method which directly predicts relationships solely based on visual appearance without combining entities or labeling all possible predicates. Extensive experiments were conducted on two benchmark datasets: Visual Genome and Open Images V6 which demonstrate that RelTR achieves superior performance in terms of accuracy and offers fast inference capabilities. In conclusion, "RelTR: Relation Transformer for Scene Graph Generation" introduces an innovative encoder-decoder model that effectively generates scene graphs by predicting relationships between objects in a one-stage manner. The proposed method outperforms existing approaches on benchmark datasets and demonstrates efficient inference times.