# Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields

**Authors:** Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, Jing Liao  
**arXiv:** [2305.11588](https://arxiv.org/abs/2305.11588)  
**Date:** 2023-05-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.11588v1/

## Key points

- Text-driven 3D scene generation has applications in video gaming, film industry, and metaverse domains
- Existing text-to-3D methods produce simple geometries and dreamlike styles lacking realism
- Researchers presented Text2NeRF - a method for generating complicated 3D scenes with high-fidelity textures from a text prompt
- They used Neural Radiance Fields (NeRF) as the 3D representation and a pre-trained text-to-image diffusion model to constrain the 3D reconstruction of NeRF to reflect the scene description
- The proposed method requires only a natural language description of the scene as input and no additional training data
- Text2NeRF outperforms existing methods in producing photo-realistic, multi-view consistent, and diverse 3D scenes from various natural language prompts.
- The research findings hold significant implications for industries such as video gaming, film-making, virtual reality experiences where realistic 3D scenes are crucial components.

## Summary

Text-driven 3D scene generation has numerous applications in the video gaming, film industry, and metaverse domains. To overcome the limitation of existing text-to-3D generation methods which produce 3D objects with simple geometries and dreamlike styles that lack realism, a team of researchers comprising Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao have presented Text2NeRF - a novel method for generating a wide range of 3D scenes with complicated geometric structures and high-fidelity textures purely from a text prompt. The researchers have adopted Neural Radiance Fields (NeRF) as the 3D representation and leveraged a pre-trained text-to-image diffusion model to constrain the 3D reconstruction of NeRF to reflect the scene description. Specifically, they employed the diffusion model to infer the text-related image as the content prior and used a monocular depth estimation method to offer the geometric prior. Both content and geometric priors were utilized to update the NeRF model. To ensure textured and geometric consistency between different views, they introduced a progressive scene inpainting and updating strategy for novel view synthesis of the scene. The proposed method requires no additional training data but only a natural language description of the scene as input. Extensive experiments demonstrate that Text2NeRF outperforms existing methods in producing photo-realistic, multi-view consistent, and diverse 3D scenes from various natural language prompts. The research findings hold significant implications for industries such as video gaming, film-making, virtual reality experiences where realistic 3D scenes are crucial components.