TL;DR: We introduce Octree-GS, featuring an LOD-structured 3D Gaussian approach supporting level-of-detail decomposition for scene representation that contributes to the final rendering results.
Our method can guarantee continuous real-time rendering while achieving better visual quality. (The white points represent the distribution of neural Gaussian to reflect quantity variance)
Abstract
The recent 3D Gaussian splatting (3D-GS) has shown remarkable rendering fidelity and efficiency compared to NeRF-based neural scene representations. While demonstrating the potential for real-time rendering, 3D-GS encounters rendering bottlenecks in large scenes with complex details due to an excessive number of Gaussian primitives located within the viewing frustum. This limitation is particularly noticeable in zoom-out views and can lead to inconsistent rendering speeds in scenes with varying details. Moreover, it often struggles to capture the corresponding level of details at different scales with its heuristic density control operation. Inspired by the Level-of-Detail (LOD) techniques, we introduce Octree-GS, featuring an LOD-structured 3D Gaussian approach supporting level-of-detail decomposition for scene representation that contributes to the final rendering results. Our model dynamically selects the appropriate level from the set of multi-resolution anchor points, ensuring consistent rendering performance with adaptive LOD adjustments while maintaining high-fidelity rendering results.
Method Overview
Illustration of our proposed Octree-GS: Starting from a sparse point cloud, we construct an octree for the bounded 3D space. Each octree level provides a set of anchor Gaussians assigned to the corresponding LOD level. Unlike conventional 3D-GS methods treating all Gaussians equally, our approach involves anchor Gaussians with different LODs. During novel view rendering, we determine the required LOD level ℓ for each occupied anchor voxel within the octree from the observation center and invoke all anchor Gaussians up to that level for final rendering. This process, shown in the middle, results in an increased level of detail by gradually fetching anchors from higher LODs in an accumulation manner. Our model is trained with standard image reconstruction loss and additional regularization loss following the practice of Scaffold-GS.
