# PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images

**Authors:** Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, Yebin Liu  
**arXiv:** [2207.06400](https://arxiv.org/abs/2207.06400)  
**Date:** 2022-07-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2207.06400v1/

## Key points

- Regression-based methods for estimating body, hand, and full-body models from monocular images
- Challenges with minor deviations in parameters leading to misalignment between estimated meshes and input images
- Introduction of Pyramidal Mesh Alignment Feedback (PyMAF) loop within regression network to rectify predicted parameters based on mesh-image alignment status
- Extension of PyMAF to PyMAF-X for recovery of expressive full-body models by adjusting elbow-twist rotations through adaptive integration strategy
- Utilization of auxiliary dense supervision and spatial alignment attention to improve alignment accuracy and global context awareness within the network
- Validation of efficacy on benchmark datasets for body-only and full-body mesh recovery, achieving new state-of-the-art results
- Project page for PyMAF-X with access to code and video results available at https://www.liuyebin.com/pymaf-x

## Summary

Regression-based methods have shown promise in estimating body, hand, and full-body models from monocular images by directly mapping raw pixels to model parameters in a feed-forward manner. However, even minor deviations in these parameters can result in noticeable misalignment between the estimated meshes and input images. This is particularly challenging when it comes to recovering full-body meshes. To tackle this issue, researchers have introduced a novel approach called Pyramidal Mesh Alignment Feedback (PyMAF) loop within the regression network. The core idea behind PyMAF is to leverage a feature pyramid and rectify predicted parameters based on mesh-image alignment status to ensure well-aligned human mesh recovery. Building upon the success of PyMAF, researchers have extended this method to PyMAF-X for the recovery of expressive full-body models. The main concept behind PyMAF-X is to adjust elbow-twist rotations through an adaptive integration strategy. This not only produces natural wrist poses but also maintains the well-aligned performance of part-specific estimations. By explicitly extracting mesh-aligned evidence from finer-resolution features and providing parameter rectification feedback, PyMAF-X enhances alignment perception and global context awareness within the network. To further improve alignment accuracy, auxiliary dense supervision is employed to offer guidance on mesh-image correspondence. Additionally, spatial alignment attention is introduced to enable the network's awareness of global contexts during the recovery process. The efficacy of both PyMAF and PyMAF-X has been validated on various benchmark datasets for body-only and full-body mesh recovery. These approaches have demonstrated significant improvements in mesh-image alignment and have achieved new state-of-the-art results in this domain. The project page for PyMAF-X with access to code and video results can be found at https://www.liuyebin.com/pymaf-x. Authors involved in this research include Hongwen Zhang, Yating Tian, Yuxiang Zhang, Mengcheng Li, Liang An, Zhenan Sun, and Yebin Liu. Overall, PyMAF-X represents a significant step towards achieving well-aligned full-body model regression from monocular images through innovative techniques that address challenges related to parameter deviation and misalignment issues commonly encountered in such tasks.