ScaNeRF: Scalable Bundle-Adjusting Neural Radiance Fields
for Large-Scale Scene Rendering

SIGGRAPH Asia 2023
(Journal track, selected in technical papers trailer)

Xiuchao Wu1     Jiamin Xu2     Xin Zhang1    Hujun Bao1    Qixing Huang3    Yujun Shen4    James Tompkin5    Weiwei Xu1


1Zhejiang University           2Hangzhou Dianzi University           3University of Texas at Austin           4Ant Group           5Brown University  




















Abstract

High-quality large-scale scene rendering requires a scalable representation and accurate camera poses. This research combines tile-based hybrid neural fields with parallel distributive optimization to improve bundleadjusting neural radiance fields. The proposed method scales with a divideand-conquer strategy. We partition scenes into tiles, each with a multiresolution hash feature grid and shallow chained diffuse and specular multilayer perceptrons (MLPs). Tiles unify foreground and background via a spatial contraction function that allows both distant objects in outdoor scenes and planar reflections as virtual images outside the tile. Decomposing appearance with the specular MLP allows a specular-aware warping loss to provide a second optimization path for camera poses. We apply the alternating direction method of multipliers (ADMM) to achieve consensus among camera poses while maintaining parallel tile optimization. Experimental results show that our method outperforms state-of-the-art neural scene rendering method quality by 5%-10% in PSNR, maintaining sharp distant objects and view-dependent reflections across six indoor and outdoor scenes.

Rendering Results

Bar (1280 x 720)
Bar
Coffee Shop (1280 x 720)
Coffee Shop
Street (1280 x 720)
Street
Shady Path (1280 x 720)
Shady Path
Community (1280 x 720)
Community
Park (1280 x 720)
Park

Method

We address the bundle-adjusting NeRF problem for large-scale 3D scenes by combining a tile-based hybrid neural field scene representation with parallel distributive training via the alternating direction method of multipliers (ADMM).


















Rendering Results (Public Data)

We test our method on dataset rubble (from Mega-NeRF) and poly-tech from (UrbanScene3D).

Our method doesn't handle the lighting differences across images. This can be addressed by appearance embedding (like Mega-NeRF&Block-NeRF).

Poly-tech
Rubble

Comparisons

We compare our method to other state-of-the-art novel view synthesis methods on indoor and outdoor scenes



Full Video







Download

Paper: ScaNeRF: Scalable Bundle-Adjusting Neural Radiance Fields for Large-Scale Scene Rendering
  
    @article{wu2023scanerf,
      title={ScaNeRF: Scalable Bundle-Adjusting Neural Radiance Fields for Large-Scale Scene Rendering},
      author={Wu, Xiuchao and Xu, Jiamin and Zhang, Xin and Bao, Hujun and Huang, Qixing and Shen, Yujun and Tompkin, James and Xu, Weiwei},
      journal={ACM Transactions on Graphics (TOG)},
      year={2023}}
   











Dataset

Access data Bar and Coffee Shop here


























Acknowledgements

Supported by Information Technology Center and State Key Lab of CAD&CG, Zhejiang University.

Thanks to Chi Wang for providing the place (community data) for shooting data.