# ChiTransformer:Towards Reliable Stereo from Cues

**Authors:** Qing Su, Shihao Ji  
**arXiv:** [2203.04554](https://arxiv.org/abs/2203.04554)  
**Date:** 2022-03-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2203.04554v3/

## Key points

- Existing stereo matching techniques face challenges such as restricted searching space, occluded regions, and sheer size.
- Single image depth estimation can overcome these challenges by utilizing monocular cues extracted from a single view.
- However, the lack of stereoscopic relationship makes the monocular prediction less reliable, particularly in dynamic or cluttered environments.
- A novel self-supervised binocular depth estimation method called ChiTransformer is proposed to address these issues in both scenarios.
- ChiTransformer incorporates a vision transformer (ViT) with gated positional cross-attention (GPCA) layers to enable feature-sensitive pattern retrieval between views and preserve extensive context information through self-attentions.
- Monocular cues from a single view are rectified using a blending layer that incorporates retrieved pattern pairs, mimicking the optic-chasma structure and enhancing the reliability of monocular predictions.
- ChiTransformer outperforms state-of-the-art self-supervised stereo approaches by achieving an improvement of 11%.
- It can be applied to both rectilinear and non-rectilinear images like fisheye images.
- The authors provide experimental evidence to demonstrate the effectiveness of ChiTransformer and have made their project available on GitHub for further exploration.

## Summary

The existing stereo matching techniques face challenges such as restricted searching space, occluded regions, and sheer size. On the other hand, single image depth estimation can overcome these challenges by utilizing monocular cues extracted from a single view. However, the lack of stereoscopic relationship makes the monocular prediction less reliable, particularly in dynamic or cluttered environments. To address these issues in both scenarios, a novel self-supervised binocular depth estimation method called ChiTransformer is proposed. Inspired by the optic-chiasm structure in the human visual system, ChiTransformer incorporates a vision transformer (ViT) with gated positional cross-attention (GPCA) layers. This design enables feature-sensitive pattern retrieval between views while preserving extensive context information through self-attentions. In ChiTransformer, monocular cues from a single view are rectified using a blending layer that incorporates retrieved pattern pairs. This crossover design mimics the optic-chasma structure and enhances the reliability of monocular predictions. Notably, ChiTransformer outperforms state-of-the-art self-supervised stereo approaches by achieving an improvement of 11%. Furthermore, it can be applied to both rectilinear and non-rectilinear images like fisheye images. The authors provide experimental evidence to demonstrate the effectiveness of ChiTransformer and have made their project available on GitHub for further exploration. The paper "ChiTransformer: Towards Reliable Stereo from Cues" authored by Qing Su and Shihao Ji provides detailed insights into this innovative approach for robust stereo depth estimation.