Does Gaussian Splatting need SFM Initialization?
Authors: Yalda Foroutan, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi
Abstract
3D Gaussian Splatting has recently been embraced as a versatile and effective method for scene reconstruction and novel view synthesis, owing to its high-quality results and compatibility with hardware rasterization. Despite its advantages, Gaussian Splatting's reliance on high-quality point cloud initialization by Structure-from-Motion (SFM) algorithms is a significant limitation to be overcome. To this end, we investigate various initialization strategies for Gaussian Splatting and delve into how volumetric reconstructions from Neural Radiance Fields (NeRF) can be utilized to bypass the dependency on SFM data. Our findings demonstrate that random initialization can perform much better if carefully designed and that by employing a combination of improved initialization strategies and structure distillation from low-cost NeRF models, it is possible to achieve equivalent results, or at times even superior, to those obtained from SFM initialization.
The papers around it, by date and closeness
Higher is closer to this paper; left is older. Lines join papers close to each other: groups of lines are sub-topics. Select a point to see why it is there. Back to the summary of this paper
Papers around this one
Ranked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
The Role of Initialization in 3D Gaussian SplattingIvan Desiatov, Torsten Sattler · 2026 · Not summarized yet
Abstract
3D Gaussian Splatting (3DGS) has become the method of choice for photo-realistic novel view synthesis (NVS), due to its efficiency and compelling visual quality. 3DGS represents the scene as a set of 3D Gaussians, parameterized by their position, spatial extent, and view-dependent color. Starting from an initial point cloud, 3DGS refines the Gaussians'parameters to reconstruct a set of training images as accurately as possible. Typically, a sparse Structure-from-Motion point cloud is used as initialization. Thus, in order to obtain a full scene representation, 3DGS methods rely on a densification stage. In this paper, we systematically study how initialization affects 3DGS NVS performance an…
-
Relaxing Accurate Initialization Constraint for 3D Gaussian SplattingJaewoo Jung, Ji-Sang Han, Honggyu An et al. · 2024 · arXiv.org · preprint, not peer reviewed · cited 64 times · Not summarized yet
Abstract
3D Gaussian splatting (3DGS) has recently demonstrated impressive capabilities in real-time novel view synthesis and 3D reconstruction. However, 3DGS heavily depends on the accurate initialization derived from Structure-from-Motion (SfM) methods. When the quality of the initial point cloud deteriorates, such as in the presence of noise or when using randomly initialized point cloud, 3DGS often undergoes large performance drops. To address this limitation, we propose a novel optimization strategy dubbed RAIN-GS (Relaing Accurate Initialization Constraint for 3D Gaussian Splatting). Our approach is based on an in-depth analysis of the original 3DGS optimization scheme and the analysis of the S…
-
High-Fidelity Gaussian Splatting from MVS Clouds: An Iterative Spatial Decomposition FrameworkZong-Hua Yu, Jun-Huai Li, Huai-Jun Wang et al. · 2026 · ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) · Not summarized yet
Abstract
3D Gaussian Splatting has become a main technique for fast 3D scene reconstruction and editing, leveraging an efficient and flexible explicit representation for high-fidelity real-time rendering. However, the quality of the point clouds used to initialize Gaussians remains a key factor that limits fine-grained geometry reconstruction. To address this limitation, we propose an Iterative Spatial Decomposition (ISD) framework that bridges dense geometric priors from Multi-View Stereo (MVS) with Gaussian Splatting. ISD mitigates the mismatch between dense MVS point clouds and Gaussian sparsity by iteratively partitioning the scene into voxels of adaptive granularity and performing density-aware …
-
Optimizing 3D Gaussian Splatting via Point Cloud UpsamplingAdrian Ramlal, Yan-Song Hu, John S. Zelek · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet
Abstract
3D Gaussian Splatting (3DGS) is a technique for creating and rendering 3D scenes, however its performance depends heavily on the quality of initial seed points. To improve 3DGS initialization, this study presents and evaluates several point cloud upsampling approaches: linear interpolation, triangular interpolation, spline-based surface reconstruction, moving least squares surface fitting, and Voronoi-based point generation. Additionally, this research introduces a depth-guided point lifting method that leverages depth maps to maintain geometric consistency with Structure-from-Motion (SfM) reconstructions. Through extensive experiments on the Mip-NeRF360 and Replica datasets, the proposed me…
-
Does 3D Gaussian Splatting Need Accurate Volumetric Rendering?A. Celarek, George Kopanas, G. Drettakis et al. · 2025 · Computer graphics forum (Print) · cited 29 times · Not summarized yet
Abstract
Since its introduction, 3D Gaussian Splatting (3DGS) has become an important reference method for learning 3D representations of a captured scene, allowing real‐time novel‐view synthesis with high visual quality and fast training times. Neural Radiance Fields (NeRFs), which preceded 3DGS, are based on a principled ray‐marching approach for volumetric rendering. In contrast, while sharing a similar image formation model with NeRF, 3DGS uses a hybrid rendering solution that builds on the strengths of volume rendering and primitive rasterization. A crucial benefit of 3DGS is its performance, achieved through a set of approximations, in many cases with respect to volumetric rendering theory. A n…
-
Deep Gaussian from Motion: Exploring 3D Geometric Foundation Models for Gaussian SplattingYu Chen, Rolandos Alexandros Potamias, Evangelos Ververas et al. · 2025 · Neural Information Processing Systems · Not summarized yet
Abstract
Neural radiance fields (NeRF) and 3D Gaussian Splatting (3DGS) are popular techniques to reconstruct and render photorealistic images. However, the prerequisite of running Structure-from-Motion (SfM) to get camera poses limits their completeness. Although previous methods can reconstruct a few unposed images, they are not applicable when images are unordered or densely captured. In this work, we propose a method to train 3DGS from unposed images. Our method leverages a pre-trained 3D geometric foundation model as the neural scene representation. Since the accuracy of the predicted pointmaps does not suffice for accurate image registration and high-fidelity image rendering, we propose to miti…
-
Pi-GS: Sparse-View Gaussian Splatting with Dense π^3 InitializationManuel Hofer, M. Steinberger, Thomas Köhler · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet
Abstract
Novel view synthesis has evolved rapidly, advancing from Neural Radiance Fields to 3D Gaussian Splatting (3DGS), which offers real-time rendering and rapid training without compromising visual fidelity. However, 3DGS relies heavily on accurate camera poses and high-quality point cloud initialization, which are difficult to obtain in sparse-view scenarios. While traditional Structure from Motion (SfM) pipelines often fail in these settings, existing learning-based point estimation alternatives typically require reliable reference views and remain sensitive to pose or depth errors. In this work, we propose a robust method utilizing {\pi}^3, a reference-free point cloud estimation network. We i…
-
DiGS: Depth-Initialized Gaussian Splatting for Single-Object ReconstructionJacopo Meglioraldi, Pasquale Cascarano, Gustavo Marfia · 2026 · Journal of Imaging · Not summarized yet
Abstract
Gaussian Splatting is a state-of-the-art technique for 3D reconstruction. In this paper, we investigate how different initialization strategies influence the optimization process within the Gaussian Splatting framework, showing that more accurate initial point clouds can greatly influence the quality of object reconstruction. We introduce the Depth-initialized Gaussian Splatting (DiGS) approach, a pipeline that leverages depth-based initialization. By incorporating depth data from a calibrated stereo camera setup, the proposed method significantly enhances model performance, particularly during the early optimization stages. DiGS is particularly effective for reconstructing isolated single o…
Prior work it builds on
-
3D Gaussian Splatting for Real-Time Radiance Field RenderingBernhard Kerbl, Georgios Kopanas, Thomas Leimkühler et al. · 2023 · ACM Transactions on Graphics, volume 42(4), July 2023 · cited 11,235 times · AI summary available
Abstract
Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting …
-
3D Gaussian Splatting as Markov Chain Monte CarloShakiba Kheradmand, Daniel Rebain, Gopal Sharma et al. · 2024 · Neural Information Processing Systems · cited 306 times · Not summarized yet
Abstract
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization. In this work, we rethink the set of 3D Gaussians as a random sample drawn from an underlying probability distribution describing the physical representation of the scene-in other words, Markov Chain Monte Carlo (MCMC) samples. Under this view, we show that the 3D Gaussian updates can be converted as Stochastic Gradient Langevin Dynamics (SGLD) updates by simply introducing noise. We then rewrite the densification and pruning strategies…
-
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content CreationJiaxiang Tang, Jiawei Ren, Hang Zhou et al. · 2023 · International Conference on Learning Representations · cited 1,112 times · Not summarized yet
Abstract
Recent advances in 3D content creation mostly leverage optimization-based 3D generation via score distillation sampling (SDS). Though promising results have been exhibited, these methods often suffer from slow per-sample optimization, limiting their practical usage. In this paper, we propose DreamGaussian, a novel 3D content generation framework that achieves both efficiency and quality simultaneously. Our key insight is to design a generative 3D Gaussian Splatting model with companioned mesh extraction and texture refinement in UV space. In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significan…
-
Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian SplattingVladimir Yugay, Yue Li, T. Gevers et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 285 times · Not summarized yet
Abstract
We present a dense simultaneous localization and mapping (SLAM) method that uses 3D Gaussians as a scene representation. Our approach enables interactive-time reconstruction and photo-realistic rendering from real-world single-camera RGBD videos. To this end, we propose a novel effective strategy for seeding new Gaussians for newly explored areas and their effective online optimization that is independent of the scene size and thus scalable to larger scenes. This is achieved by organizing the scene into sub-maps which are independently optimized and do not need to be kept in memory. We further accomplish frame-to-model camera tracking by minimizing photometric and geometric losses between th…
-
4D Gaussian Splatting for Real-Time Dynamic Scene RenderingGuan-Jun Wu, Taoran Yi, Jie-Min Fang et al. · 2023 · Computer Vision and Pattern Recognition · cited 1,723 times · Not summarized yet
Abstract
Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is…
Papers that cite it
-
Liberated-Gs: 3D Gaussian Splatting Independent From Sfm Point CloudsWeihong Pan, Xiao-Yu Zhang, Hong-Jia Zhai et al. · 2025 · IEEE International Conference on Computer Vision · cited 7 times · Not summarized yet
-
Is Gaussian Splatting Becoming Neural Again? A Taxonomy and Controlled Study of Learned ParameterizationYuan-Hang Wang, Xin Cao, Yi Zhang · 2026 · Not summarized yet
Abstract
Three-dimensional Gaussian Splatting (3DGS) combines explicit primitives with efficient rasterization, yet recent systems increasingly use neural networks to generate or share Gaussian parameters. We characterize this trend along five axes: attribute decoding, spatial sharing, view-conditioned decoding, topology generation, and amortized inference. An analysis of 19 representative methods shows that these choices address different limitations and cannot be reduced to a binary neural label. We also isolate three forms of neural parameterization in a controlled mip-NeRF 360 study. Sharing appearance and opacity improves reconstruction quality, while decoding geometric structure offers no furth…
-
RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real- Time Rendering with 900+ FPSMichael Niemeyer, F. Manhardt, Marie-Julie Rakotosaona et al. · 2024 · International Conference on 3D Vision · cited 156 times · Not summarized yet
Abstract
Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive ren-dering speeds. While radiance field-based methods achieve state-of-the-art quality in challenging scenarios such as in-the-wild captures and large-scale scenes, they often suf-fer from excessively high compute requirements linked to volumetric rendering. Gaussian Splatting-based methods, on the other hand, rely on rasterization and naturally achieve real-time rendering but suffer from brittle opti-mization heuristics that underperform on more challenging scenes. In this work, we present RadSplat, a lightweight method for robust real-time rendering of complex scenes. Our main contr…
-
Turbo-GS: Accelerating 3D Gaussian Fitting for High-Quality Radiance FieldsTao Lu, Ankit Dhiman, R. Srinath et al. · 2024 · arXiv.org · preprint, not peer reviewed · cited 18 times · Not summarized yet
Abstract
Novel-view synthesis plays a crucial role in computer vision with applications in 3D reconstruction, mixed reality, and robotics. Recent approaches, such as 3D Gaussian Splatting (3DGS), have emerged as state-of-the-art solutions, offering high-quality novel view synthesis in real time. However, training 3DGS models remains slow, particularly for high-resolution images, often requiring hours to fit a scene with 200 views. In this work, we aim to accelerate the fitting process by reducing computational overhead and improving learning efficiency. Specifically, we introduce a dilated rendering technique that renders only a subset of pixels instead of the full image, significantly reducing compu…
-
GaussFusion: Improving 3D Reconstruction in the Wild with A Geometry-Informed Video GeneratorLiyuan Zhu, M. Narayana, Michal Starý et al. · 2026 · Computer Vision and Pattern Recognition · cited 9 times · Not summarized yet
Abstract
We present GaussFusion, a novel approach for improving 3D Gaussian splatting (3DGS) reconstructions in the wild through geometry-informed video generation. GaussFusion mitigates common 3DGS artifacts, including floaters, flickering, and blur caused by camera pose errors, incomplete coverage, and noisy geometry initialization. Unlike prior RGB-based approaches limited to a single reconstruction pipeline, our method introduces a geometry-informed video-to-video generator that refines 3DGS renderings across both optimization-based and feed-forward methods. Given an existing reconstruction, we render a Gaussian primitive video buffer encoding depth, normals, opacity, and covariance, which the ge…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 3 November 2026.