GSDF: 3DGS Meets SDF for Improved Neural Rendering and Reconstruction

Mulin Yu*1, Tao Lu*1, Linning Xu2, Lihan Jiang4,1, Yuanbo Xiangli3, Bo Dai5,1
1 Shanghai Artificial Intelligence Laboratory, 2 The Chinese University of Hong Kong, 3 Cornell University 4 University of Science and Technology of China 5 The University of Hong Kong
Paper Code

TL;DR: We introduce GSDF, a dual-branch architecture that combines the benefits of 3D Gaussian Splatting (3DGS) with neural Signed Distance Fields (SDF) to boost both rendering and reconstruction quality.

From randomly initialized points, GSDF can arrive better rendering quality.


Method Overview

Overview of Dual-branch Guidance. Our dual-branch framework includes a GS-branch for rendering and an SDF-branch for learning neural surfaces. This design preserves the efficiency and fidelity of Gaussian primitive for rendering while accurately approximating scene surfaces from an SDF field adapted from NeuS. Specifically: (1) The GS-branch renders depth maps to guide SDF-branch ray sampling, querying absolute SDF values |s| and sampling points within 2k|s| (e.g., k = 4). (2) Predicted SDF values guide GS-branch density control, growing Gaussians near surfaces and pruning deviated ones. (3) Mutual geometry consistency is enforced by comparing depth and normal maps from both branches, ensuring coherent alignment between Gaussians and surfaces.



Reconstruction Results




Instantnsr GSDF

Reconstruction results with 30k training iterations.

Instantnsr GSDF

Reconstruction results with 500k training iterations.

GSDF yields more surface-aligned primitives with regularized structure.

Rendering Results


GSDF and its randomly initialized variant retains finer details in the texture-less region and show depth maps better aligned to the potential surfaces.

Using PhysGaussian to simulate GSDF checkpoints.

GSDF outperforms baselines, especially in modeling delicate geometries (1st & 2nd row), texture-less and less observations regions (3rd & 4th row) that are quite common in larger scenes.

More obviously improvements for GSDF are shown when the 3D Gaussians are rnadomly initialized.