# Open-World Semantic Segmentation Including Class Similarity

**Authors:** Matteo Sodano, Federico Magistri, Lucas Nunes, Jens Behley, Cyrill Stachniss  
**arXiv:** [2403.07532](https://arxiv.org/abs/2403.07532)  
**Date:** 2024-03-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.07532v1/

## Key points

- Accurate interpretation of camera data is crucial for autonomous systems, such as autonomous vehicles
- Open-world semantic segmentation allows for interpreting image data with previously unseen objects
- The proposed approach excels in precise closed-world semantic segmentation and can identify new categories without additional training data
- The method provides a valuable similarity measure for newly discovered classes compared to known categories, aiding downstream tasks like planning or mapping
- The model achieves state-of-the-art results on both trained classes and anomaly segmentation, as well as distinguishing between various unknown classes
- This research has been accepted at CVPR 2024 and the code is available on GitHub at https://github.com/PRBonn/ContMAV

## Summary

In the realm of autonomous systems, such as autonomous vehicles, accurate interpretation of camera data is crucial. Vision systems operating in real-world environments must be able to understand their surroundings and navigate through novel situations. This paper explores open-world semantic segmentation, which involves interpreting image data where previously unseen objects may appear. The authors propose an innovative approach that not only excels in precise closed-world semantic segmentation but also has the ability to identify new categories without additional training data. Furthermore, their method provides a valuable similarity measure for newly discovered classes compared to known categories, which can aid downstream tasks like planning or mapping. Through rigorous experimentation, Matteo Sodano, Federico Magistri, Lucas Nunes, Jens Behley and Cyrill Stachniss demonstrate that their model achieves state-of-the-art results on both trained classes and anomaly segmentation. It also shows impressive capability in distinguishing between various unknown classes. This research has been accepted at CVPR 2024 and the code is available on GitHub at https://github.com/PRBonn/ContMAV. These findings hold significant promise for advancing the fields of computer vision and autonomous systems by enhancing their adaptability and performance in dynamic real-world scenarios.