# Human Motion Diffusion as a Generative Prior

**Authors:** Yonatan Shafir, Guy Tevet, Roy Kapon, Amit H. Bermano  
**arXiv:** [2303.01418](https://arxiv.org/abs/2303.01418)  
**Date:** 2023-03-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.01418v1/

## Key points

- The paper explores the use of a pre-trained diffusion-based model as a generative prior for motion generation tasks.
- Data scarcity is a challenge in this field due to the expensive acquisition process of motion, biasing data towards short single-person sequences.
- The authors demonstrate that their approach can be effective for fine-tuning in a few- and even zero-shot manner.
- For zero-shot setting, they introduce DoubleTake, an inference-time method that generates up to 10-minute long animations with meaningful and controlled transitions using a prior trained for 10-second generations.
- For few-shot setting, they learn text-guided two-person motion generation for the first time using their ComMDM slim communication block.
- Their approach consistently outperforms state-of-the art models designed and trained for those tasks by evaluating their method on three cases: long sequence motion generation, multi person motion generation, and semantically complete motions from a single prescribed joint.
- They use Motion Diffusion Model (MDM) as a prior which has already been demonstrated to generalize well to conditions from other domains and corrections performed between iterations.

## Summary

This paper explores the use of a pre-trained diffusion-based model as a generative prior for motion generation tasks. While denoising diffusion models have recently been introduced to motion generation, the main challenge in this field remains the low availability of data. The expensive acquisition process of motion biases the already modest data towards short single-person sequences, leaving more elaborate generative tasks behind. To mitigate this gap, the authors demonstrate that their approach can be effective for fine-tuning in a few- and even zero-shot manner. For zero-shot setting, they introduce DoubleTake, an inference-time method that generates up to 10-minute long animations with meaningful and controlled transitions using a prior trained for 10-second generations. For few-shot setting, they learn text-guided two-person motion generation for the first time using their ComMDM slim communication block. The authors also show that their approach consistently outperforms state-of-the art models designed and trained for those tasks by evaluating their method on three cases: long sequence motion generation, multi person motion generation, and semantically complete motions from a single prescribed joint. In particular, they use Motion Diffusion Model (MDM) as a prior which has already been demonstrated to generalize well to conditions from other domains and corrections performed between iterations. They further introduce DoubleTake method which enables efficient generation of long motion sequences in a zero shot manner while controlling each interval using different text conditions with realistic transitions between them. Overall, this paper presents an innovative approach to overcome data scarcity challenges in motion generation tasks by utilizing pre trained diffusion based models as priors and demonstrating its effectiveness through various experiments.