# RomniStereo: Recurrent Omnidirectional Stereo Matching

**Authors:** Hualie Jiang, Rui Xu, Minglang Tan, Wenjie Jiang  
**arXiv:** [2401.04345](https://arxiv.org/abs/2401.04345)  
**Date:** 2024-01-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.04345v1/

## Key points

- Omnidirectional Stereo Matching (OSM) is crucial for providing accurate $360^{\circ}$ depth information.
- Existing state-of-the-art methods for stereo matching rely on complex 3D encoder-decoder blocks, leading to sub-optimal results.
- The new algorithm Recurrent Omnidirectional Stereo Matching (RomniStereo) bridges the gap between OSM and RAFT by introducing an adaptive weighting scheme and incorporating grid embedding and adaptive context feature generation techniques.
- RomniStereo outperforms previous methods by improving the average Mean Absolute Error metric by 40.7% across five datasets.
- RomniStereo produces more accurate depth maps with fewer artifacts compared to other methods like OmniMVS+ in datasets such as OmniThings and OmniHouse, especially excelling in close-range regions crucial for robot navigation.

## Summary

, , , , 

In the field of depth sensing, <kw>Omnidirectional Stereo Matching (OSM)</kw> plays a crucial role in providing accurate and reliable $360^{\circ}$ depth information. However, existing state-of-the-art (SOTA) methods for stereo matching often rely on complex 3D encoder-decoder blocks to regularize cost volumes, leading to sub-optimal results. A recent approach based on <kw>Recurrent All-pairs Field Transforms (RAFT)</kw> has shown significant improvements in image-matching tasks like optical flow and stereo matching by employing recurrent updates in 2D. To bridge the gap between OSM and RAFT, a new algorithm called <kw>Recurrent Omnidirectional Stereo Matching (RomniStereo)</kw> is proposed. This innovative approach introduces an opposite adaptive weighting scheme to seamlessly transform the outputs of spherical sweeping from OSM into the required inputs for recurrent updates. Additionally, RomniStereo incorporates two novel techniques - grid embedding and adaptive context feature generation - further enhancing its performance. The RomniStereo algorithm outperforms previous SOTA methods by improving the average Mean Absolute Error (<kw>MAE</kw>) metric by 40.7% across five datasets. Visualizations of the results demonstrate clear advantages of RomniStereo over synthetic and realistic examples. The code for RomniStereo is publicly available at https://github.com/HalleyJiang/RomniStereo. Furthermore, qualitative comparisons show that RomniStereo produces more accurate depth maps with fewer artifacts compared to other methods like OmniMVS+ on datasets such as OmniThings and OmniHouse. In real-world scenarios, RomniStereo excels in producing cleaner and more accurate depth maps, especially in close-range regions crucial for robot navigation. Overall, RomniStereo offers a refined and efficient solution for <kw>omnidirectional stereo matching</kw>, combining the strengths of OSM with the advancements of RAFT to achieve superior depth sensing capabilities without sacrificing accuracy.