# ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields

**Authors:** Nagabhushan Somraj, Rajiv Soundararajan  
**arXiv:** [2305.0041](https://arxiv.org/abs/2305.0041)  
**Date:** 2023-04-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.0041v1/

## Key points

- The paper presents a novel approach called ViP-NeRF to address the limitations of Neural Radiance Fields (NeRFs) in synthesizing photo-realistic novel views.
- NeRFs require a large number of images per scene, making them computationally expensive and impractical for many applications.
- Training NeRFs on sparse input views often leads to overfitting and incorrect scene depth estimation, resulting in artifacts in rendered novel views.
- Previous approaches have used dense depth estimated from pre-trained networks as supervision, but these depth priors may be inaccurate due to generalization issues.
- The authors propose using the visibility of pixels in different input views as more reliable dense supervision by computing a visibility prior using plane sweep volumes.
- By incorporating this visibility prior into the NeRF training process, they successfully train NeRFs with only a few input views.
- They reformulate the NeRF model to directly output the visibility of a 3D point from a given viewpoint, reducing training time while enforcing the visibility constraint.
- The proposed ViP-NeRF model outperforms existing sparse input NeRF models on multiple datasets.
- ViP-NeRF leverages visibility priors instead of depth priors and reduces computational requirements for view synthesis using sparse input views.

## Summary

The paper titled "ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields" presents a novel approach to address the limitations of Neural Radiance Fields (NeRFs) in synthesizing photo-realistic novel views. NeRFs have shown impressive performance in view synthesis by encoding neural representations of a scene. However, they require a large number of images per scene, making them computationally expensive and impractical for many applications. One major challenge with NeRFs is that training them on sparse input views often leads to overfitting and incorrect scene depth estimation, resulting in artifacts in the rendered novel views. To overcome this limitation, previous approaches have used dense depth estimated from pre-trained networks as supervision to regularize the training process. However, these depth priors may be inaccurate due to generalization issues. In this paper, the authors propose an alternative solution by hypothesizing that the visibility of pixels in different input views can provide more reliable dense supervision. They compute a visibility prior using plane sweep volumes which does not require any pre-training. By incorporating this visibility prior into the NeRF training process they successfully train NeRFs with only a few input views. Furthermore, the authors reformulate the NeRF model to directly output the visibility of a 3D point from a given viewpoint which reduces the training time while enforcing the visibility constraint. The proposed model outperforms existing sparse input NeRF models including those that use learned priors on multiple datasets. Overall, this paper introduces ViP-NeRF as an effective method for improving view synthesis using sparse input views by leveraging visibility priors instead of depth priors and reducing computational requirements. The source code for their model is available on their project page for further exploration and implementation.