AI summary · arXiv: 2308.04079v1 · cs.GR
3D Gaussian Splatting for Real-Time Radiance Field Rendering
The essentials
- Radiance Field methods have revolutionized novel view synthesis for scenes captured with multiple photos or videos.
- Achieving high visual quality in real-time remains a challenge due to the costly training and rendering requirements of neural networks.
- Faster methods sacrifice quality for speed, while existing methods struggle with real-time display rates at 1080p resolution.
- Representing the scene with 3D Gaussians derived from sparse points obtained during camera calibration.
- Interleaved optimization and density control of the 3D Gaussians, optimizing anisotropic covariance to improve fidelity.
- Development of a fast visibility-aware rendering algorithm that supports anisotropic splatting, accelerating both training and real-time rendering processes.
- The method achieves state-of-the-art visual quality and real-time rendering on established datasets.
- Compared to previous approaches, it offers a balance between speed and visual quality, bridging the gap between interactive rendering times and real-time rendering at high resolutions.
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Continue with Google Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View SettingsLorenzo Lazzaroni, Riccardo Bollati, Dániel Baráth et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet
Abstract
Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, …
-
On Reducing the Number of Gaussians for Radiance Field Real-Time RenderingOmnia Mahmoud, M. Gendrin · 2024 · 2024 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Elect · cited 3 times · Not summarized yet
Abstract
Various Radiance Field methods have been recently explored for photo-realistic novel-view synthesis of scenes captured with multiple views given a set of images. However, none of them has provided the high visual quality with the real-time rendering of 3D Gaussian Splatting (3D-GS) without a trade-off in neither the speed nor the quality. On the other hand, the astonishing improvements of 3D Gaussian Splatting come at the cost of having a huge storage footprint due to the massive number of gaussians per model. In this paper, we focus on further optimizing the creation of gaussians throughout the training iterations to control the number of gaussians needed to represent the model. We propose …
-
Recent advances in 3D Gaussian splattingTong Wu, Yu-Jie Yuan, Lingxiao Zhang et al. · 2024 · Computational Visual Media · cited 256 times · Not summarized yet
Abstract
The emergence of 3D Gaussian splatting (3DGS) has greatly accelerated rendering in novel view synthesis. Unlike neural implicit representations like neural radiance fields (NeRFs) that represent a 3D scene with position and viewpoint-conditioned neural networks, 3D Gaussian splatting utilizes a set of Gaussian ellipsoids to model the scene so that efficient rendering can be accomplished by rasterizing Gaussian ellipsoids into images. Apart from fast rendering, the explicit representation of 3D Gaussian splatting also facilitates downstream tasks like dynamic reconstruction, geometry editing, and physical simulation. Considering the rapid changes and growing number of works in this field, we …
-
DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth NormalizationJia-He Li, J. Zhang, Xiao Bai et al. · 2024 · Computer Vision and Pattern Recognition · cited 392 times · Not summarized yet
Abstract
Radiance fields have demonstrated impressive performance in synthesizing novel views from sparse input views, yet prevailing methods suffer from high training costs and slow inference speed. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian radiance fields, offering real-time and high-quality few-shot novel view synthesis at low costs. Our motivation stems from the highly efficient representation and surprising quality of the recent 3D Gaussian Splatting, despite it will encounter a geometry degradation when input views decrease. In the Gaussian radiance fields, we find this degradation in scene geometry primarily lined to the positioning of Gaussian primit…
-
3D Gaussian Splatting with Deferred ReflectionKeyang Ye, Qiming Hou, Kun Zhou · 2024 · Computer Vision and Pattern Recognition · cited 95 times · AI summary available
Abstract
The advent of neural and Gaussian-based radiance field methods have achieved great success in the field of novel view synthesis. However, specular reflection remains non-trivial, as the high frequency radiance field is notoriously difficult to fit stably and accurately. We present a deferred shading method to effectively render specular reflection with Gaussian splatting. The key challenge comes from the environment map reflection model, which requires accurate surface normal while simultaneously bottlenecks normal estimation with discontinuous gradients. We leverage the per-pixel reflection gradients generated by deferred shading to bridge the optimization process of neighboring Gaussians, …
-
Compact 3D Gaussian Representation for Radiance FieldJoo Chan Lee, Daniel Rho, Xiangyu Sun et al. · 2023 · Computer Vision and Pattern Recognition · cited 569 times · AI summary available
Abstract
Neural Radiance Fields (NeRFs) have demonstrated remarkable potential in capturing complex 3D scenes with high fidelity. However, one persistent challenge that hinders the widespread adoption of NeRFs is the computational bottleneck due to the volumetric rendering. On the other hand, 3D Gaussian splatting (3DGS) has recently emerged as an alternative representation that leverages a 3D Gaussisan-based representation and adopts the rasterization pipeline to render the images rather than volumetric rendering, achieving very fast rendering speed and promising image quality. However, a significant drawback arises as 3DGS entails a substantial number of 3D Gaussians to maintain the high fidelity o…
-
WL-GS: A Gaussian Splatting Method for Unconstrained Photo Collection under Resource-Limited ConditionsHong-Fei Wang, Sisi Zeng, Qingcang Yu · 2025 · IEEE International Conference on Systems, Man and Cybernetics · Not summarized yet
Abstract
Neural Radiance Fields (NeRFs) have demonstrated impressive performance in novel view synthesis from unconstrained image collections. However, their substantial computational overhead and limited rendering efficiency hinder practical deployment. In contrast, 3D Gaussian Splatting (3DGS) enhances training speed and rendering quality through explicit scene representation but suffers from high memory consumption and poor scalability in in-the-wild settings. To address these limitations, we propose WL-GS, a novel 3D reconstruction framework that adopts a modeling strategy decoupled from dynamic appearance, along with a visibility mask to effectively suppress transient occlusions. Dynamic appeara…
-
4D Gaussian Splatting for Real-Time Dynamic Scene RenderingGuanjun Wu, Taoran Yi, Jiemin Fang et al. · 2024 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conferen · cited 840 times · Not summarized yet
Abstract
Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is…
Prior work it builds on
-
Differentiable Point-Based Radiance Fields for Efficient View SynthesisQian Zhang, Seung-Hwan Baek, S. Rusinkiewicz et al. · 2022 · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia · cited 121 times · Not summarized yet
Abstract
We propose a differentiable rendering algorithm for efficient novel view synthesis. By departing from volume-based representations in favor of a learned point representation, we improve on existing methods more than an order of magnitude in memory and runtime, both in training and inference. The method begins with a uniformly-sampled random point cloud and learns per-point position and view-dependent appearance, using a differentiable splat-based renderer to train the model to reproduce a set of input training images with the given pose. Our method is up to 300 × faster than NeRF in both training and inference, with only a marginal sacrifice in quality, while using less than 10 MB of memory …
-
Baking Neural Radiance Fields for Real-Time View SynthesisPeter Hedman, P. Srinivasan, B. Mildenhall et al. · 2021 · IEEE International Conference on Computer Vision · cited 668 times · Not summarized yet
Abstract
Neural volumetric representations such as Neural Radiance Fields (NeRF) have emerged as a compelling technique for learning to represent 3D scenes from images with the goal of rendering photorealistic images of the scene from unobserved viewpoints. However, NeRF’s computational requirements are prohibitive for real-time applications: rendering views from a trained NeRF requires querying a multilayer perceptron (MLP) hundreds of times per ray. We present a method to train a NeRF, then precompute and store (i.e. "bake") it as a novel representation called a Sparse Neural Radiance Grid (SNeRG) that enables real-time rendering on commodity hardware. To achieve this, we introduce 1) a reformulati…
-
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields ReconstructionCheng Sun, Min Sun, Hwann-Tzong Chen · 2021 · Computer Vision and Pattern Recognition · cited 1,431 times · Not summarized yet
Abstract
We present a super-fast convergence approach to reconstructing the per-scene radiance field from a set of images that capture the scene with known poses. This task, which is often applied to novel view synthesis, is recently revolution-ized by Neural Radiance Field (NeRF) for its state-of-the-art quality and fiexibility. However, NeRF and its variants require a lengthy training time ranging from hours to days for a single scene. In contrast, our approach achieves NeRF-comparable quality and converges rapidly from scratch in less than 15 minutes with a single GPU. We adopt a representation consisting of a density voxel grid for scene geometry and a feature voxel grid with a shallow network fo…
-
Plenoxels: Radiance Fields without Neural NetworksSara Fridovich-Keil, Alex Yu, Matthew Tancik et al. · 2021 · Computer Vision and Pattern Recognition · cited 2,398 times · Not summarized yet
Abstract
We introduce Plenoxels (plenoptic voxels), a systemfor photorealistic view synthesis. Plenoxels represent a scene as a sparse 3D grid with spherical harmonics. This representation can be optimized from calibrated images via gradient methods and regularization without any neural components. On standard, benchmark tasks, Plenoxels are optimized two orders of magnitude faster than Neural Radiance Fields with no loss in visual quality. For video and code, please see https://alexyu.net/plenoxels.
-
PlenOctrees for Real-time Rendering of Neural Radiance FieldsAlex Yu, Ruilong Li, Matthew Tancik et al. · 2021 · IEEE International Conference on Computer Vision · cited 1,367 times · Not summarized yet
Abstract
We introduce a method to render Neural Radiance Fields (NeRFs) in real time using PlenOctrees, an octree-based 3D representation which supports view-dependent effects. Our method can render 800×800 images at more than 150 FPS, which is over 3000 times faster than conventional NeRFs. We do so without sacrificing quality while preserving the ability of NeRFs to perform free-viewpoint rendering of scenes with arbitrary geometry and view-dependent effects. Real-time performance is achieved by pre-tabulating the NeRF into a PlenOctree. In order to preserve view-dependent effects such as specularities, we factorize the appearance via closed-form spherical basis functions. Specifically, we show tha…
Papers that cite it
-
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature FieldsShijie Zhou, Haoran Chang, Sicheng Jiang et al. · 2024 · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conferen · cited 234 times · Not summarized yet
Abstract
3D scene representations have gained immense popularity in recent years. Methods that use Neural Radiance fields are versatile for traditional tasks such as novel view synthesis. In recent times, some work has emerged that aims to extend the functionality of NeRF beyond view synthesis, for semantically aware tasks such as editing and segmentation using 3D feature field distillation from 2D foundation models. However, these methods have two major limitations: (a) they are limited by the rendering speed of NeRF pipelines, and (b) implicitly represented feature fields suffer from continuity artifacts reducing feature quality. Recently, 3D Gaussian Splatting has shown state-of-the-art performanc…
-
2D Gaussian Splatting for Geometrically Accurate Radiance FieldsBinbin Huang, Zehao Yu, An-Pei Chen et al. · 2024 · International Conference on Computer Graphics and Interactive Techniques · cited 1,804 times · Not summarized yet
Abstract
3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed. However, 3DGS fails to accurately represent surfaces due to the multi-view inconsistent nature of 3D Gaussians. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. Our key idea is to collapse the 3D volume into a set of 2D oriented planar Gaussian disks. Unlike 3D Gaussians, 2D Gaussians provide view-consistent geometry while modeling surfaces intrinsically. To accurately recover thin surfaces and achieve stable optimization, we introduce a per…
-
PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D ReconstructionDavid Charatan, S. Li, Andrea Tagliasacchi et al. · 2023 · Computer Vision and Pattern Recognition · cited 890 times · Not summarized yet
Abstract
We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images. Our model features real-time and memory-efficient rendering for scalable training as well as fast 3D reconstruction at inference time. To overcome local minima inherent to sparse and locally supported representations, we predict a dense probability distribution over 3D and sample Gaussian means from that probability distribution. We make this sampling operation differentiable via a reparameterization trick, allowing us to back-propagate gradients through the Gaussian splatting representation. We benchmark our method on wide-baseline novel v…
-
BudSplat: Feed-forward 3D Gaussian Splatting under a Rendering BudgetMu-Yu Xu, Fangneng Zhan, Yu Wei et al. · 2026 · Not summarized yet
Abstract
Feed-forward 3D Gaussian Splatting has recently emerged as a promising alternative to per-scene optimization, enabling fast novel view synthesis by predicting Gaussian primitives directly from input images. However, it quickly faces a bottleneck of rendering efficiency while scaling from sparse to dense multi-view inputs, which offer more geometric constraints but produce a rapidly growing set of candidate Gaussians. Existing solutions cannot reliably distinguish redundant primitives, leading to many redundant primitives from repeatedly observed regions and increased rendering cost without much gain in visual quality. We propose BudSplat, a compact feed-forward 3DGS framework that learns to …
-
Scaffold-GS: Structured 3D Gaussians for View-Adaptive RenderingTao Lu, Mulin Yu, Linning Xu et al. · 2023 · Computer Vision and Pattern Recognition · cited 511 times · AI summary available
Abstract
Neural rendering methods have significantly advanced photo-realistic 3D scene rendering in various academic and industrial applications. The recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed combining the benefits of both primitive-based representations and volumetric representations. However, it often leads to heavily redundant Gaussians that try to fit every training view, neglecting the underlying scene geometry. Consequently, the resulting model becomes less robust to significant view changes, texture-less area and lighting effects. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians, and predicts their attri…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 4 November 2026.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.