BungeeNeRF: Progressive Neural Radiance Field for
Extreme Multi-scale Scene Rendering
ECCV 2022
(Previous version: CityNeRF: Building NeRF at City Scale)
Abstract
Neural Radiance Field (NeRF) has achieved outstanding performance in modeling 3D objects and controlled scenes, usually under a single scale. In this work, we make the first attempt to bring NeRF to city-scale, with views ranging from satellite-level that captures the overview of a city, to ground-level imagery showing complex details of an architecture. The wide span of camera distance to the scene yields multi-scale data with different levels of detail and spatial coverage, which casts great challenges to vanilla NeRF and biases it towards compromised results. To address these issues, we introduce CityNeRF, a progressive learning paradigm that grows the NeRF model and training set synchronously. Starting from fitting distant views with a shallow base block, as training progresses, new blocks are appended to accommodate the emerging details in the increasingly closer views. The strategy effectively activates high-frequency channels in the positional encoding and unfolds more complex details as the training proceeds. We demonstrate the superiority of CityNeRF in modeling diverse city-scale scenes with drastically varying views, and its support for rendering views in different levels of detail.
Framework
Overview of CityNeRF. (a) An illustration of the multi-scale data in city-scale scenes, where we use L ∈ { 1 , 2 , 3 , . . . , } to denote each scale. At each stage, our model grows in synchronization with the training set. (b) New residual blocks are appended to the network as the training proceeds, supervised by the union of samples from the most remote scale up to the current scale. The structure of a residual block is shown in the dashed box. (c) Level-of-detail rendering results obtained at different residual blocks. From shallow to deep, details are added bit by bit. ( ©2021 Google )