MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions

2025-10-09 · Kaen Kogashi, Anoop Cherian, Meng-Yu Jennifer Kuo arxiv

Real-world scenes often feature multiple humans interacting with multiple objects in ways that are causal, goal-oriented, or cooperative. Yet existing 3D human-object interaction (HOI) benchmarks consider only a fraction of these complex interactions. To close this gap, we present MMHOI -- a large-scale, Multi-human Multi-object Interaction dataset consisting of images from 12 everyday scenarios. MMHOI offers complete 3D shape and pose annotations for every person and object, along with labels for 78 action categories and 14 interaction-specific body parts, providing a comprehensive testbed for next-generation HOI research. Building on MMHOI, we present MMHOI-Net, an end-to-end transformer-based neural network for jointly estimating human-object 3D geometries, their interactions, and associated actions. A key innovation in our framework is a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance the interaction prediction. Experiments on MMHOI and the recently proposed CORE4D datasets demonstrate that our approach achieves state-of-the-art performance in multi-HOI modeling, excelling in both accuracy and reconstruction quality. The MMHOI dataset is publicly available at https://zenodo.org/records/17711786.

📄 PDF Abstract BibTeX arXiv:2510.07828

Code (0)

등록된 구현이 없습니다.

Tasks

Action Recognition

Pi-HOC: Pairwise 3D Human-Object Contact Estimation

2026-04-14 · Sravan Chittupalli, Ayush Jain, Dong Huang arxiv

Resolving real-world human-object interactions in images is a many-to-many challenge, in which disentangling fine-grained concurrent physical contact is particularly difficult. Existing semantic contact estimation method…

Multi-Scale Incremental Modeling for Enhanced Human Motion Prediction in Human-Robot Collaboration

2024-12-16 · Juncheng Zou

Accurate human motion prediction is crucial for safe human-robot collaboration but remains challenging due to the complexity of modeling intricate and variable human movements. This paper presents Parallel Multi-scale In…

Human motion prediction motion prediction Prediction

GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos

2026-04-17 · Deepak Kumar, Abhishek Pratap Singh, Puneet Kumar, Xiaobai Li 외 arxiv

Understanding affective dynamics in real-world social systems is fundamental to modeling and analyzing human-human interactions in complex environments. Group affect emerges from intertwined human-human interactions, con…

Beyond Winning and Losing: Modeling Human Motivations and Behaviors Using Inverse Reinforcement Learning

2018-07-01 · Baoxiang Wang, Tongfang Sun, Xianjun Sam Zheng

In recent years, reinforcement learning (RL) methods have been applied to model gameplay with great success, achieving super-human performance in various environments, such as Atari, Go, and Poker. However, those studies…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling

2024-05-21 · Xingzhou Lou, Junge Zhang, Jian Xie, Lifeng Liu 외

Human preference alignment is critical in building powerful and reliable large language models (LLMs). However, current methods either ignore the multi-dimensionality of human preferences (e.g. helpfulness and harmlessne…