# ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions

**Authors:** Jeonghwan Kim, Jisoo Kim, Jeonghyeon Na, Hanbyul Joo  
**arXiv:** [2401.10232](https://arxiv.org/abs/2401.10232)  
**Date:** 2024-01-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.10232v1/

## Key points

- Rich data encompassing 3D motion of humans and objects is crucial for machines to learn human interaction with the physical world
- Scarcity of large-scale datasets capturing 3D motions of both humans and objects in causal interactions
- Existing datasets focus on limited aspects, such as human motion without objects or hand-object interactions in static postures
- Introduction of ParaHome system to capture and parameterize dynamic 3D movements of humans and objects in a home environment
- System includes multi-view setup with 70 synchronized RGB cameras and wearable motion capture devices
- Collection of a novel large-scale dataset of human-object interaction with advancements over existing datasets:
- Capturing 3D body and dexterous hand manipulation motion alongside 3D object movement in a contextual home environment during natural activities
- Encompassing human interaction with multiple objects in various episodic scenarios with corresponding descriptions in texts
- Including articulated objects with multiple parts expressed with parameterized articulations
- Participants perform sequences of actions involving the manipulation of one or two objects, including cooking-related actions and small actions that can occur in a room environment
- Dataset captured from 30 participants (15 females and 15 males) interacting with objects
- Each scenario consists of atomic actions divided into two sessions, ranging from [duration range]
- Total [number] captures resulting in a total of [number]
- Introduction of new research tasks for building generative models for learning and synthesizing human-object interactions using this dataset

## Summary

To enable machines to learn how humans interact with the physical world in our daily activities, it is crucial to provide rich data that encompasses the 3D motion of humans as well as the motion of objects in a learnable 3D representation. However, there is a scarcity of large-scale datasets captured in natural and casual settings that include the 3D motions of both humans and objects occurring in causal interactions. Existing datasets primarily focus on limited aspects of these challenges, such as capturing human motion without objects or focusing on hand-object interactions in static postures or relatively simple and short interactions. To address these limitations, the authors introduce the ParaHome system designed to capture and parameterize dynamic 3D movements of humans and objects within a common home environment. The system consists of a multi-view setup with 70 synchronized RGB cameras, as well as wearable motion capture devices equipped with an IMU-based body suit and hand motion capture gloves. By leveraging this system, they collect a novel large-scale dataset of human-object interaction. The dataset offers key advancements over existing datasets in three main aspects: (1) capturing 3D body and dexterous hand manipulation motion alongside 3D object movement within a contextual home environment during natural activities; (2) encompassing human interaction with multiple objects in various episodic scenarios with corresponding descriptions in texts; (3) including articulated objects with multiple parts expressed with parameterized articulations. The participants perform sequences of actions composed of small atomic actions involving the manipulation of one or two objects. A total of  are performed by participants, including cooking-related actions and small actions that can occur in a room environment. Each participant performs  consisting of non-cooking actions and cooking-related actions placed in semi-arbitrary order with corresponding verbal instruction for each action. The dataset was captured from  (15 females and 15 males) and contains  interacting with . Each scenario performed by the participants consists of , divided into two sessions of captures due to storage limits. The duration of each session ranges from . In total,  were captured, resulting in a total of . The authors introduce new research tasks aimed at building a generative model for learning and synthesizing human-object interactions in a real-world room setting using this dataset. This work addresses the limitations of existing datasets and provides valuable data for advancing the understanding and modeling of human-object interactions in natural and casual settings.