TL;DR: We propose GSDF, a dual-branch system that enhances rendering and reconstruction at the same time, leveraging the mutual geometry regularization and guidance between Gaussain primitives and neural surface.
From randomly initialized points, GSDF achieves better rendering quality, especially in the texture-less areas (i.e., grass).
Method Overview
Framework. Our dual-branch framework comprises a GS-branch dedicated to rendering and an SDF-branch focusing on learning neural surfaces. Our design effectively preserves the superiority of rendering with Gaussian primitives in terms of efficiency and fidelity, and also more accurately approximates scene surfaces from an SDF field adapted from NeuS. Concretely, (1) we leverage the efficiency and flexibility advantages of the GS-branch, to render depth maps and guide the ray sampling process of the SDF-branch. For each depth position, we query the SDF-branch to obtain its absolute SDF value |s|, and uniformly sample points within 2k|s| (e.g. k = 4). (2) The predicted SDF values from the SDF-branch are in turn used to guide the density control of the GS-branch to grow Gaussian primitives in near-surface regions and prune the ones that are far-away. (3) We further enforce mutual geometry consistency by comparing the depth and normal maps from each branch to encourage more coherent physical alignment between Gaussian primitives and surfaces.
