# OmniGS: Fast Radiance Field Reconstruction using Omnidirectional Gaussian Splatting

**Authors:** Longwei Li, Huajian Huang, Sai-Kit Yeung, Hui Cheng  
**arXiv:** [2404.03202](https://arxiv.org/abs/2404.03202)  
**Date:** 2024-04-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.03202v3/

## Key points

- 3D Gaussian Splatting has shown great potential in photorealistic reconstruction across various domains
- Introduction of OmniGS, a novel omnidirectional Gaussian splatting system for fast radiance field reconstruction
- Theoretical analysis of spherical camera model derivatives to inform the development of OmniGS
- Implementation of a new GPU-accelerated omnidirectional rasterizer for rendering omnidirectional images without cube-map rectification or tangent-plane approximation
- Extensive experiments conducted to evaluate the performance of OmniGS in egocentric and roaming scenarios, demonstrating state-of-the-art reconstruction quality and high rendering speed with omnidirectional images
- Public availability of authors' code for further research and development in the field of 3D structure reconstruction from observed environments using large field-of-view (FoV) images
- Shift towards NeRF-based methods for scene reconstruction from omnidirectional imagery, offering promising results in achieving realistic renderings and immersive experiences

## Summary

In the field of photorealistic reconstruction, the use of 3D Gaussian Splatting has shown great potential across various domains. However, the current system only supports radiance field reconstruction using undistorted perspective images. To address this limitation, a novel omnidirectional Gaussian splatting system called OmniGS has been introduced in this paper. By taking advantage of omnidirectional images, OmniGS enables fast radiance field reconstruction. The authors conducted a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting to inform the development of OmniGS. They implemented a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. This approach allows for differentiable optimization of the omnidirectional radiance field without the need for cube-map rectification or tangent-plane approximation. Extensive experiments were conducted in both egocentric and roaming scenarios to evaluate the performance of OmniGS. The results demonstrate that this method achieves state-of-the-art reconstruction quality and high rendering speed when using omnidirectional images. Additionally, the authors have made their code publicly available for further research and development. In recent advancements within the realm of 3D structure reconstruction from observed environments, leveraging large field-of-view (FoV) images has become crucial for applications such as environmental monitoring, virtual reality, localization, navigation, path planning, and other high-level perception tasks. The utilization of omnidirectional images that capture the entire environment in each shot has been explored by various approaches like 360Roam, 360FusionNerf, and PaniGRF to enable photorealistic reconstruction and immersive scene roaming. Unlike traditional methods focused on precise geometry reconstruction, these approaches leverage neural radiance field (NeRF) techniques that employ multi-layer perceptrons (MLPs) to implicitly model scenes by accumulating density and view-dependent color per ray. This shift towards NeRF-based methods offers a new paradigm in scene reconstruction from omnidirectional imagery with promising results in achieving realistic renderings and immersive experiences for users.