Results of the summarizing process for the arXiv paper: 2209.14916v2

The generation of natural and expressive human motion is a challenging task in computer animation due to the diversity of possible motions, human perceptual sensitivity, and the difficulty of accurately describing it. Current generative solutions are either low-quality or limited in expressiveness. However, diffusion models have shown remarkable generative capabilities in other domains and are promising candidates for human motion due to their many-to-many nature. The Motion Diffusion Model (MDM) is introduced as a carefully adapted classifier-free diffusion-based generative model for the human motion domain. MDM is transformer-based, combining insights from motion generation literature. A notable design choice is the prediction of the sample rather than the noise in each diffusion step, facilitating the use of established geometric losses on the locations and velocities of the motion such as foot contact loss. MDM's generic approach enables different modes of conditioning and different generation tasks. The MDM framework achieves state-of-the-art quality in several motion generation tasks while requiring only about three days of training on a lightweight resource. In text-to-motion tasks, MDM generates coherent motions that achieve state-of-the-art results on leading benchmarks for text-to-motion and action-to-motion. In addition, human evaluators prefer MDM generated motions over real motions 42% of the time. In action to motion tasks, MDM outperforms state of the art models designed specifically for this task on common benchmarks. MDM also demonstrates completion and editing by adapting diffusion image inpainting to set a motion prefix and suffix and using our model to fill in gaps under textual conditions while maintaining semantic input. By performing inpainting in joints space rather than temporally, MDM also demonstrates semantic editing of specific body parts without changing others. Overall, Motion Diffusion Model introduces a promising solution to generate high quality diverse human motions with controllability while being lightweight and efficient compared to existing methods. The code can be found at https://github.com/GuyTevet/motiondiffusionmodel .

• Natural and expressive human motion generation is challenging in computer animation due to diversity, perceptual sensitivity, and difficulty in accurately describing it.
• Current generative solutions are low-quality or limited in expressiveness.
• Diffusion models have shown remarkable generative capabilities in other domains and are promising candidates for human motion due to their many-to-many nature.
• Motion Diffusion Model (MDM) is introduced as a carefully adapted classifier-free diffusion-based generative model for the human motion domain.
• MDM is transformer-based, combining insights from motion generation literature.
• MDM predicts the sample rather than the noise in each diffusion step, facilitating the use of established geometric losses on the locations and velocities of the motion such as foot contact loss.
• MDM achieves state-of-the-art quality in several motion generation tasks while requiring only about three days of training on a lightweight resource.
• In text-to-motion tasks, MDM generates coherent motions that achieve state-of-the-art results on leading benchmarks for text-to-motion and action-to-motion.
• Human evaluators prefer MDM generated motions over real motions 42% of the time.
• In action to motion tasks, MDM outperforms state of the art models designed specifically for this task on common benchmarks.
• MDM also demonstrates completion and editing by adapting diffusion image inpainting to set a motion prefix and suffix and using our model to fill in gaps under textual conditions while maintaining semantic input.
• By performing inpainting in joints space rather than temporally, MDM also demonstrates semantic editing of specific body parts without changing others.
• Overall, Motion Diffusion Model introduces a promising solution to generate high quality diverse human motions with controllability while being lightweight and efficient compared to existing methods.

Summary: Scientists have created a computer program that can make people move like humans do. This is hard to do because everyone moves differently and it's tricky to describe how they move. Other programs that try to do this aren't very good. The new program, called Motion Diffusion Model (MDM), is really good at making people move in lots of different ways. It only takes three days to teach the program how to do this. Definitions- Natural and expressive human motion generation: creating movements that look like those made by real people - Computer animation: using computers to create moving pictures - Diversity: being different from each other - Perceptual sensitivity: being able to notice small differences or changes - Generative solutions: computer programs that create things automatically - Diffusion models: a type of mathematical model used in science and engineering - Classifier-free: not using a system for sorting things into categories - Transformer-based: using a type of neural network called a transformer - Geometric losses: measures used in math to compare shapes and sizes - Foot contact loss: measuring how well feet touch the ground when walking or running - Text-to-motion tasks: creating movements based on written words - Action-to-motion tasks: creating movements based on actions, like kicking a ball - Semantic input/editing/completion/inpainting: changing or finishing something while keeping its meaning intact

Generating Natural and Expressive Human Motion with the Motion Diffusion Model

Computer animation is a complex task, as it requires accurately replicating natural human motion. This is due to the diversity of possible motions, human perceptual sensitivity, and the difficulty of accurately describing them. Current generative solutions are either low-quality or limited in expressiveness. However, diffusion models have shown remarkable generative capabilities in other domains and are promising candidates for human motion due to their many-to-many nature.

The Motion Diffusion Model (MDM) is a carefully adapted classifier-free diffusion-based generative model for the human motion domain that has recently been introduced. It combines insights from motion generation literature with transformer-based technology to create an efficient yet powerful solution for generating high quality diverse motions with controllability while being lightweight and efficient compared to existing methods.

Design Choices

A notable design choice of MDM is its prediction of samples rather than noise in each diffusion step, which facilitates the use of established geometric losses on locations and velocities of motions such as foot contact loss. Additionally, MDM's generic approach enables different modes of conditioning and different generation tasks such as text-to-motion tasks or action-to-motion tasks.

Performance Results

MDM has achieved state-of-the art quality in several motion generation tasks while requiring only about three days of training on a lightweight resource. In text-to-motion tasks, MDM generates coherent motions that achieve state-of–the–art results on leading benchmarks for text–to–motion and action–to–motion; furthermore, human evaluators prefer MDM generated motions over real motions 42% of the time! In action to motion tasks, MDM outperforms state –of –the –art models designed specifically for this task on common benchmarks. MDM also demonstrates completion and editing by adapting diffusion image inpainting to set a motion prefix and suffix; using our model to fill gaps under textual conditions while maintaining semantic input; performing inpainting in joints space rather than temporally; demonstrating semantic editing of specific body parts without changing others; etc…

Conclusion

Overall, Motion Diffusion Model introduces a promising solution to generate high quality diverse human motions with controllability while being lightweight and efficient compared to existing methods. The code can be found at https://github.com/GuyTevet/motiondiffusionmodel .