# NeRFuser: Large-Scale Scene Representation by NeRF Fusion

**Authors:** Jiading Fang, Shengjie Lin, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Adrien Gaidon, Gregory Shakhnarovich, Matthew R. Walter  
**arXiv:** [2305.13307](https://arxiv.org/abs/2305.13307)  
**Date:** 2023-05-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.13307v1/

## Key points

- NeRFuser is a novel architecture that addresses the challenge of operating on implicit visual data structures like Neural Radiance Fields (NeRFs)
- NeRFs offer memory efficiency by allowing large scenes to be stored and shared as small neural nets instead of collections of images
- NeRFuser proposes a solution for NeRF registration and blending, assuming access only to pre-generated NeRFs
- Two key techniques introduced by NeRFuser are registration from re-rendering and sample-based inverse distance weighting for blending
- Registration from re-rendering infers the transformation between NeRFs based on synthesized images, enabling accurate alignment between different views of a scene
- Sample-based inverse distance weighting at the ray-sample level is used for effective fusion of visual information from multiple NeRFs, resulting in high-quality blended representations
- Experiments conducted on public benchmarks and a self-collected dataset demonstrate the robustness of NeRFuser, even in challenging scenarios
- Overall, NeRFuser enables efficient registration and blending operations on pre-generated NeRFs, opening up possibilities for large-scale scene representation and sharing using compact neural networks.

## Summary

NeRFuser is a novel architecture that addresses the challenge of operating on implicit visual data structures like Neural Radiance Fields (NeRFs). NeRFs offer the practical benefit of memory efficiency, allowing large scenes to be stored and shared as small neural nets instead of collections of images. However, working with these implicit visual data structures requires extending classical image-based vision techniques such as registration and blending from image sets to neural fields. The authors propose NeRFuser as a solution for NeRF registration and blending, assuming access only to pre-generated NeRFs rather than the potentially large sets of images used to generate them. They introduce two key techniques: registration from re-rendering and sample-based inverse distance weighting for blending. Registration from re-rendering involves inferring the transformation between NeRFs based on images synthesized from individual NeRFs. This technique enables accurate alignment between different views of a scene represented by NeRFs. For blending, NeRFuser employs sample-based inverse distance weighting at the ray-sample level. This approach allows for effective fusion of visual information from multiple NeRFs, resulting in high-quality blended representations. To evaluate their method, the authors conducted experiments on public benchmarks and a self-collected object-centric indoor dataset. The results demonstrate the robustness of NeRFuser, even in challenging scenarios where rendering views from individual source NeRFs is difficult. Overall, NeRFuser offers an innovative solution for handling implicit visual data structures like Neural Radiance Fields. By enabling efficient registration and blending operations on pre-generated NeRFs, it opens up possibilities for large-scale scene representation and sharing using compact neural networks instead of image collections.