# Towards Open World Object Detection

**Authors:** K J Joseph, Salman Khan, Fahad Shahbaz Khan, Vineeth N Balasubramanian  
**arXiv:** [2103.02603](https://arxiv.org/abs/2103.02603)  
**Date:** 2021-03-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2103.02603v2/

## Key points

- Humans have a natural instinct to identify unknown object instances in their environments
- Curiosity about unknown instances helps us learn about them when we eventually have the corresponding knowledge
- A model is tasked with identifying unknown objects without explicit supervision
- The model should be able to incrementally learn these identified unknown categories without forgetting previously learned classes
- ORE (Open World Object Detector) is a novel solution based on contrastive clustering and energy-based unknown identification
- ORE introduces a strong evaluation protocol to assess its efficacy in achieving open-world objectives
- Experimental evaluation and ablation studies analyze the performance of ORE and its ability to achieve open-world objectives
- Identifying and characterizing unknown instances helps reduce confusion in incremental object detection setting
- State-of-the-art performance is achieved without additional methodological effort
- The work aims to attract further research into the direction of open-world object detection.

## Summary

Humans have a natural instinct to identify unknown object instances in their environments. This curiosity about unknown instances helps us learn about them when we eventually have the corresponding knowledge. In this problem, a model is tasked with identifying objects that it has not been introduced to as "unknown," without explicit supervision to do so. Additionally, the model should be able to incrementally learn these identified unknown categories without forgetting previously learned classes when it receives the corresponding labels progressively. To address this problem, we introduce a novel solution called ORE: Open World Object Detector. ORE is based on contrastive clustering and energy-based unknown identification. We also formulate a strong evaluation protocol to assess the efficacy of ORE in achieving open-world objectives. In our experimental evaluation and ablation studies, we analyze the performance of ORE and its ability to achieve open-world objectives. Interestingly, we find that identifying and characterizing unknown instances also helps reduce confusion in an incremental object detection setting. As a result, we achieve state-of-the-art performance without any additional methodological effort. We hope that our work will attract further research into this newly identified but crucial research direction of open-world object detection.