Results of the summarizing process for the arXiv paper: 2112.09127v1

In the paper "ICON: Implicit Clothed humans Obtained from Normals," authors Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J. Black address the limitations of current methods for learning realistic and animatable 3D clothed avatars. The proposed method utilizes implicit functions to capture intricate details such as hair and clothing in the avatar reconstruction process. To overcome challenges with diverse human poses, the authors introduce ICON, a novel framework that leverages local features instead of global ones. By reconstructing multiple frames of a subject in different poses and utilizing SCANimate, the method can create an animatable avatar with superior performance compared to state-of-the-art techniques. Evaluation on AGORA and CAPE datasets demonstrates ICON's robustness to out-of-distribution samples like in-the-wild poses/images and out-of-frame cropping. This approach represents a significant advancement towards achieving robust 3D clothed human reconstruction from real-world images.

• - Authors address limitations of current methods for learning 3D clothed avatars
• - Proposed method uses implicit functions to capture intricate details like hair and clothing
• - Introduces ICON framework leveraging local features for diverse human poses
• - Utilizes multiple frames and SCANimate for creating animatable avatars with superior performance
• - Evaluation on AGORA and CAPE datasets shows robustness to out-of-distribution samples
• - Represents significant advancement in 3D clothed human reconstruction from real-world images

Summary- Authors talk about problems with current ways of learning 3D clothed avatars. - They suggest a new way that uses special functions to capture small details like hair and clothes. - They introduce a framework called ICON that focuses on different human poses. - By using multiple frames and SCANimate, they can make avatars that move well and look good. - Testing on AGORA and CAPE datasets shows the new method works even with different kinds of images. Definitions- Avatars: Digital characters representing people in games or virtual worlds. - Implicit functions: Mathematical tools used to describe shapes or surfaces without directly listing their points. - Framework: A basic structure or plan for doing something. - Diverse: Showing a lot of variety or differences. - Robustness: Ability to remain strong and effective even when faced with challenges.

Introduction

In recent years, there has been a growing interest in creating realistic and animatable 3D clothed avatars for various applications such as virtual try-on, gaming, and animation. However, current methods face limitations in accurately capturing intricate details of clothing and hair while also being able to handle diverse human poses. In their paper "ICON: Implicit Clothed humans Obtained from Normals," authors Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J. Black propose a novel framework that overcomes these challenges by utilizing implicit functions.

The Limitations of Current Methods

Traditional methods for reconstructing 3D clothed avatars rely on explicit representations such as meshes or point clouds. While these techniques can capture the overall shape of a subject's body and clothing, they struggle with fine details like wrinkles or folds in fabric. Additionally, they often require manual annotation or parameter tuning to achieve satisfactory results. Another limitation is the difficulty in handling diverse human poses. Most existing methods use global features to represent the entire body at once, which can lead to errors when dealing with extreme poses or occlusions.

The Proposed Method: ICON

To address these limitations, the authors introduce ICON (Implicit Clothed humans Obtained from Normals), a novel framework that leverages implicit functions for avatar reconstruction. Implicit functions are mathematical equations that describe surfaces without explicitly representing them as meshes or points. The key idea behind ICON is to use local features instead of global ones to represent different parts of the body separately. This approach allows for more accurate reconstruction of intricate details like hair and clothing while also being robust to diverse poses.

Reconstruction Process

The first step in the reconstruction process is obtaining multiple frames of a subject captured in different poses using a standard RGB camera setup. These frames are then used to generate a set of normal maps, which represent the surface orientation at each pixel. Next, the authors use a neural network to learn an implicit function that can map from 2D image coordinates and normals to 3D points on the avatar's surface. By using local features instead of global ones, this network can handle diverse poses and capture fine details in clothing and hair.

Animation with SCANimate

To create an animatable avatar, the authors utilize SCANimate, a method for transferring motion from one subject to another. This technique uses a deep learning model trained on motion capture data to transfer pose and shape information from a source subject to the reconstructed avatar. By combining ICON's reconstruction process with SCANimate's animation capabilities, the authors are able to create realistic and animatable 3D clothed avatars with superior performance compared to state-of-the-art methods.

Evaluation Results

The authors evaluate their method on two datasets: AGORA and CAPE. Both datasets contain images of subjects captured in different poses wearing various types of clothing. The results show that ICON outperforms existing methods in terms of accuracy and robustness when dealing with out-of-distribution samples like in-the-wild poses/images or out-of-frame cropping.

Robustness to Diverse Poses

ICON achieves higher accuracy than other methods when reconstructing avatars in extreme poses or occluded body parts. This is due to its ability to leverage local features instead of global ones, allowing it to handle challenging cases more effectively.

Realistic Clothing Details

Compared to traditional explicit representation methods, ICON produces more accurate representations of intricate details like wrinkles or folds in fabric. This is because implicit functions can better capture these fine details without requiring manual annotation or parameter tuning.

Conclusion

In conclusion, "ICON: Implicit Clothed humans Obtained from Normals" presents a novel framework for reconstructing realistic and animatable 3D clothed avatars. By utilizing implicit functions and local features, the method overcomes limitations of current techniques in capturing fine details and handling diverse poses. Evaluation results on two datasets demonstrate its superior performance compared to state-of-the-art methods. This approach represents a significant advancement towards achieving robust 3D clothed human reconstruction from real-world images.