# Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance

**Authors:** Haiwen Feng, Peter Kulits, Shichen Liu, Michael J. Black, Victoria Abrevaya  
**arXiv:** [2304.10528](https://arxiv.org/abs/2304.10528)  
**Date:** 2023-04-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.10528v1/

## Key points

- The paper proposes a new approach called ArtEq for fitting a human body model to point cloud data.
- Traditional optimization-based methods require careful initialization and are prone to becoming trapped in local optima, while learning-based methods do not generalize well when the input pose is far from those seen during training.
- ArtEq is a part-based SE(3)-equivariant neural architecture that learns a part detection network by leveraging local SO(3) invariance and regresses shape and pose using articulated SE(3) shape-invariant and pose-equivariant networks, all trained end-to-end.
- Experimental results show that ArtEq can generalize to poses not seen during training, outperforming state-of-the-art methods by 74.5%, without requiring an optimization refinement step.
- Compared with competing works, ArtEq is more than three orders of magnitude faster during inference and has 97.3% fewer parameters.
- This method has significant implications for applications such as virtual try-on, motion capture, and augmented reality where accurate estimation of human body shape and pose from point cloud data is crucial.
- The code and model will be available for research purposes at https://arteq.is.tue.mpg.de., making it accessible for further research in this field.

## Summary

This paper proposes a novel approach called ArtEq for fitting a parametric human body model (SMPL) to point cloud data. Traditional optimization-based methods require careful initialization and are prone to becoming trapped in local optima, while learning-based methods do not generalize well when the input pose is far from those seen during training. The authors extend the idea of leveraging SE(3)-equivariant networks, which have achieved remarkable generalization for rigid point clouds, to articulated objects such as human bodies. ArtEq is a part-based SE(3)-equivariant neural architecture that learns a part detection network by leveraging local SO(3) invariance and regresses shape and pose using articulated SE(3) shape-invariant and pose-equivariant networks, all trained end-to-end. The novel equivariant pose regression module leverages the permutation-equivariant property of self-attention layers to preserve rotational equivariance. Experimental results show that ArtEq can generalize to poses not seen during training, outperforming state-of-the-art methods by 74.5%, without requiring an optimization refinement step. Compared with competing works, ArtEq is more than three orders of magnitude faster during inference and has 97.3% fewer parameters. This method has significant implications for applications such as virtual try-on, motion capture, and augmented reality where accurate estimation of human body shape and pose from point cloud data is crucial. The code and model will be available for research purposes at https://arteq.is.tue.mpg.de., making it accessible for further research in this field.