Hypergraph-based Motion Generation with Multi-modal Interaction Relational Reasoning

2024-09-18 · Keshu Wu, Yang Zhou, Haotian Shi, Dominique Lord, Bin Ran, Xinyue Ye

The intricate nature of real-world driving environments, characterized by dynamic and diverse interactions among multiple vehicles and their possible future states, presents considerable challenges in accurately predicting the motion states of vehicles and handling the uncertainty inherent in the predictions. Addressing these challenges requires comprehensive modeling and reasoning to capture the implicit relations among vehicles and the corresponding diverse behaviors. This research introduces an integrated framework for autonomous vehicles (AVs) motion prediction to address these complexities, utilizing a novel Relational Hypergraph Interaction-informed Neural mOtion generator (RHINO). RHINO leverages hypergraph-based relational reasoning by integrating a multi-scale hypergraph neural network to model group-wise interactions among multiple vehicles and their multi-modal driving behaviors, thereby enhancing motion prediction accuracy and reliability. Experimental validation using real-world datasets demonstrates the superior performance of this framework in improving predictive accuracy and fostering socially aware automated driving in dynamic traffic scenarios.

📄 PDF Abstract BibTeX arXiv:2409.11676

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous Vehicles Implicit RelationsMotion Generationmotion predictionRelational Reasoning

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection

2026-02-18 · Rong Fu, Ziming Wang, Shuo Yin, Haiyun Wei 외 arxiv

Emotional expression underpins natural communication and effective human-computer interaction. We present Emotion Collider (EC-Net), a hyperbolic hypergraph framework for multimodal emotion and sentiment modeling. EC-Net…

Contrastive Learning

HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation

2024-11-27 · CVPR 2025 1 · Trong-Thuan Nguyen, Pha Nguyen, Jackson Cothren, Alper Yilmaz 외

Multimodal LLMs have advanced vision-language tasks but still struggle with understanding video scenes. To bridge this gap, Video Scene Graph Generation (VidSGG) has emerged to capture multi-object relationships across v…

Graph Generation Question Answering Scene Graph Generation Video Captioning+2

Hyper-STTN: Social Group-aware Spatial-Temporal Transformer Network for Human Trajectory Prediction with Hypergraph Reasoning

2024-01-12 · Weizheng Wang, Chaowei Wang, Baijian Yang, Guohua Chen 외

Predicting crowded intents and trajectories is crucial in varouls real-world applications, including service robots and autonomous vehicles. Understanding environmental dynamics is challenging, not only due to the comple…

Autonomous Vehicles Trajectory Prediction

Hyper-FEOD: Sparse Hypergraph-Enhanced Frame-Event Object Detection with Fine-Grained MoE

2026-04-13 · Wei Bao, Yuehan Wang, Tianhang Zhou, Siqi Li 외 arxiv

The integration of frame-based RGB cameras with event streams constitutes a promising paradigm for robust object detection under challenging dynamic conditions. Nevertheless, effectively modeling intricate multi-modal in…

Robust Object Detection

Disentangled Dual-Branch Graph Learning for Conversational Emotion Recognition

2026-04-03 · Chengling Guo, Yuntao Shou, Tao Meng, Wei Ai 외 arxiv

Multimodal emotion recognition in conversations aims to infer utterance-level emotions by jointly modeling textual, acoustic, and visual cues within context. Despite recent progress, key challenges remain, including redu…

Multimodal Emotion Recognition Graph Neural Network Graph Learning