AG3D: Learning to Generate 3D Avatars from 2D Image Collections
AI-generated Key Points
- Zijian Dong and Xu Chen propose a novel method for generating realistic 3D avatars from unstructured 2D image collections.
- The authors' approach is to learn generative models of 3D avatars from abundant unstructured 2D image collections, which can capture shape and deformation of the body and loose clothing.
- The proposed method outperforms previous methods in terms of geometry and appearance, as demonstrated through systematic ablation studies.
- The authors' generator design uses a monolithic approach that models humans holistically in a canonical space using an efficient tri-plane representation.
- The proposed method also incorporates multiple discriminators specialized for improving geometric detail as well as perceptually important regions like the face.
- Normal information is used for guiding geometry in the generative setting by discriminating normal maps rendered from their generative model against off-the-shelf monocular estimators applied to images of human subjects.
- The authors contribute a generative model of articulated 3D humans with state-of-the-art appearance and geometry, a new generator that is efficient and can generate and deform loose clothing, and several specialized discriminators that significantly improve visual and geometric fidelity.
Authors: Zijian Dong, Xu Chen, Jinlong Yang, Michael J. Black, Otmar Hilliges, Andreas Geiger
Abstract: While progress in 2D generative models of human appearance has been rapid, many applications require 3D avatars that can be animated and rendered. Unfortunately, most existing methods for learning generative models of 3D humans with diverse shape and appearance require 3D training data, which is limited and expensive to acquire. The key to progress is hence to learn generative models of 3D avatars from abundant unstructured 2D image collections. However, learning realistic and complete 3D appearance and geometry in this under-constrained setting remains challenging, especially in the presence of loose clothing such as dresses. In this paper, we propose a new adversarial generative model of realistic 3D people from 2D images. Our method captures shape and deformation of the body and loose clothing by adopting a holistic 3D generator and integrating an efficient and flexible articulation module. To improve realism, we train our model using multiple discriminators while also integrating geometric cues in the form of predicted 2D normal maps. We experimentally find that our method outperforms previous 3D- and articulation-aware methods in terms of geometry and appearance. We validate the effectiveness of our model and the importance of each component via systematic ablation studies.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.