# OmniGS: Omnidirectional Gaussian Splatting for Fast Radiance Field Reconstruction using Omnidirectional Images

**Authors:** Longwei Li, Huajian Huang, Sai-Kit Yeung, Hui Cheng  
**arXiv:** [2404.03202](https://arxiv.org/abs/2404.03202)  
**Date:** 2024-04-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.03202v2/

## Key points

- OmniGS is a groundbreaking system that revolutionizes photorealistic 3D reconstruction.
- It leverages direct omnidirectional screen-space splatting for fast radiance field reconstruction using omnidirectional images.
- The authors present a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting to develop OmniGS.
- They implement a new GPU-accelerated omnidirectional rasterizer for rendering without cube-map rectification or tangent-plane approximation.
- Extensive experiments show that OmniGS achieves state-of-the-art reconstruction quality and high rendering speeds with omnidirectional images.
- The method outperforms existing techniques in both quality and rendering FPS, even with less training time.
- Evaluations in perspective rendering demonstrate superior results compared to traditional methods by cropping rendered omnidirectional images into perspective views.
- Future advancements may focus on improving quality through factors like the periodicity of trigonometric functions in screen-space splatting algorithms, leading to higher-quality reconstructions at faster speeds.

## Summary

OmniGS is a groundbreaking system that revolutionizes the field of photorealistic 3D reconstruction by leveraging the speed and efficiency of direct omnidirectional screen-space splatting. This innovative approach allows for fast radiance field reconstruction using omnidirectional images, a significant advancement from existing systems. In their paper, Longwei Li, Huajian Huang, Sai-Kit Yeung, and Hui Cheng present a comprehensive theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting to lay the foundation for the development of OmniGS. By implementing a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto equirectangular screen space for omnidirectional image rendering, they achieve differentiable optimization of the radiance field without the need for cube-map rectification or tangent-plane approximation. Extensive experiments conducted in both egocentric and roaming scenarios demonstrate that OmniGS not only achieves state-of-the-art reconstruction quality but also delivers high rendering speeds using omnidirectional images. The authors highlight that their method outperforms existing techniques in terms of both reconstruction quality and rendering FPS even with less training time. Additionally, evaluations in perspective rendering show that by cropping rendered omnidirectional images into perspective views, OmniGS can generate superior results compared to traditional 3D Gaussian Splatting methods. Looking ahead, the authors believe that OmniGS has immense potential for further advancements. They suggest exploring ways to improve quality by considering factors like the periodicity of trigonometric functions in screen-space splatting algorithms. This could lead to even higher-quality reconstructions at faster inference speeds, opening up new possibilities for robotics applications. In conclusion, OmniGS represents a significant leap forward in the realm of 3D reconstruction technology. With its innovative approach and impressive results, this system has the potential to drive advancements in various research areas and benefit the broader research community.