HaloGS: Loose Coupling of Compact Geometry and Gaussian Splats for 3D Scenes

Changjian Jiang^{1,2 *}, Kerui Ren^{2,3 *}, Linning Xu⁴, Jiong Chen⁵, Jiangmiao Pang², Yu Zhang¹, Bo Dai⁶ Mulin Yu²

¹ Zhejiang University, ² Shanghai Artificial Intelligence Laboratory, ³ Shanghai Jiao Tong University, ⁴ The Chinese University of Hong Kong, ⁵ Inria, ⁶ The University of Hong Kong

Paper Code

TL;DR: We introduce HaloGS, a dual-representation that loosely couples triangles for geometry with Gaussians for appearance, enabling high-fidelity rendering with compact geometry.

Our design yields a compact yet expressive model capable of photo-realistic rendering across both indoor and outdoor environments, seamlessly adapting to varying levels of scene complexity.

Method Overview

Overview of HaloGS. Our proposed dual-representation is illustrated in (a), where learnable triangles explicitly fit the scene geometry, and neural Gaussians decoded from these triangles render the appearance. In (b), we depict our coarse-to-fine training strategy: during the coarse stage, monocular geometric priors supervise the positions and shapes of the triangles. Subsequently, in the fine stage, neural Gaussians decoded from these half-trained triangles are optimized using ground truth images. Concurrently, depth and normal maps rendered from the neural Gaussians provide additional refinement feedback to further enhance the triangle representation.

Reconstruction Results

Representation of scene geometry.

Planar shape extraction.

Starting from the optimized triangles, we reconstruct compact mesh. This figure illustrates the resulting mesh alongside the triangle soup and planar primitives, demonstrating how HaloGS accurately preserves geometric fidelity.

HaloGS demonstrates robust applicability across diverse scenarios, including both indoor and outdoor environments.

Rendering Results

HaloGS outperforms baselines, faithfully reconstructing fine structures and complex planar surfaces, such as wall-mounted mirrors and intricate ceiling ornaments, which existing baselines struggle to capture.