# GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

**Authors:** Barbara Roessle, Norman Müller, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder, Matthias Nießner  
**arXiv:** [2306.06044](https://arxiv.org/abs/2306.06044)  
**Date:** 2023-06-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.06044v1/

## Key points

- The paper presents a joint solution to mitigate imperfections in 3D scene reconstruction using GANs and NeRF.
- The authors propose learning the patch distribution of a scene using an adversarial discriminator that provides feedback to the radiance field reconstruction to improve realism in a 3D-consistent fashion.
- This approach repairs rendering artifacts directly in the underlying 3D representation by imposing multi-view path rendering constraints.
- The authors condition a generator with multi-resolution NeRF renderings that are adversarially trained to further enhance rendering quality.
- The proposed approach significantly improves rendering quality compared to Nerfacto, nearly halving LPIPS scores while improving PSNR by 1.4dB on advanced indoor scenes of Tanks and Temples.
- The authors' contribution lies in leveraging discriminators to optimize NeRFs for realistic image synthesis.
- The proposed method is also capable of repairing rendering artifacts directly in the underlying 3D representation while imposing multi-view path rendering constraints.
- Overall, this paper presents an innovative solution that combines GANs and NeRFs for enhanced realism in 3D scene reconstruction.
- The proposed approach has significant potential for applications such as virtual reality and gaming where high-quality image synthesis is crucial for immersive experiences.

## Summary

The paper titled "GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields" by Barbara Roessle, Norman Müller, Lorenzo Porzi, Samuel Rota Bulò, Peter Kontschieder, and Matthias Nießner presents a joint solution to mitigate imperfections in 3D scene reconstruction using generative adversarial networks (GANs) and Neural Radiance Fields (NeRF). Although NeRF has shown impressive results in novel view synthesis, even thorough recordings can yield imperfections due to poorly observed areas or minor lighting changes. The authors propose learning the patch distribution of a scene using an adversarial discriminator that provides feedback to the radiance field reconstruction to improve realism in a 3D-consistent fashion. This approach repairs rendering artifacts directly in the underlying 3D representation by imposing multi-view path rendering constraints. Additionally, the authors condition a generator with multi-resolution NeRF renderings that are adversarially trained to further enhance rendering quality. The proposed approach significantly improves rendering quality compared to Nerfacto, nearly halving LPIPS scores while improving PSNR by 1.4dB on advanced indoor scenes of Tanks and Temples. The authors' contribution lies in leveraging discriminators to optimize NeRFs for realistic image synthesis. They demonstrate that their approach effectively mitigates imperfections caused by various sources such as poorly observed areas or minor lighting changes. The proposed method is also capable of repairing rendering artifacts directly in the underlying 3D representation while imposing multi-view path rendering constraints. Overall, this paper presents an innovative solution that combines GANs and NeRFs for enhanced realism in 3D scene reconstruction. The proposed approach has significant potential for applications such as virtual reality and gaming where high-quality image synthesis is crucial for immersive experiences.