Reasoning Mamba: Hypergraph-Guided Region Relation Calculating for Weakly Supervised Affordance Grounding
This paper pays attention to Weakly Supervised Affordance Grounding (WSAG) task that aims to train model to identify affordance regions using human-object interaction images and egocentric images without the need for costly pixel-level annotations. Most existing methods usually consider the affordance regions to be isolated and directly employ class activation maps to conduct localization, ignoring the relationships with other object components and weakening the performance. For example, a cup's handle is combined with its body to achieve the pouring ability. Obviously, capturing the region relationships is beneficial for improving the localization accuracy of affordance regions. To this end, we first explore exploiting hypergraph to discover these relations and propose a Reasoning Mamba (R-Mamba) framework. We first extract feature embeddings from exo-centric and egocentric images to construct the hypergraphs consisting of multiple vertices and hyperedges, which capture the in-context local region relationships between different visual components. Subsequently, we design a Hypergraph-guided State Space (HSS) block to reorganize these local relationships from the global perspective. By this mechanism, the model could leverage the captured relationships to improve the localization accuracy of affordance regions. Extensive experiments and visualization analyses demonstrate the superiority of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Human-Object Interaction DetectionMambaRelationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MaGNet: A Mamba Dual-Hypergraph Network for Stock Prediction via Temporal-Causal and Global Relational Learning
Stock trend prediction is crucial for profitable trading strategies and portfolio management yet remains challenging due to market volatility, complex temporal dynamics and multifaceted inter-stock relationships. Existin…
Stock Trend PredictionRelational ReasoningGaLa: Hypergraph-Guided Visual Language Models for Procedural Planning
Implicit spatial relations and deep semantic structures encoded in object attributes are crucial for procedural planning in embodied AI systems. However, existing approaches often over rely on the reasoning capabilities …
Contrastive LearningDistillH-Mamba: A Hypergraph-Mamba-Based Knowledge Distillation Model for Efficient Impact Fall Detection
Falls among the elderly represent a significant public health concern due to their prevalence, consequences, and societal burden. While deep learning has improved fall detection, accurately identifying impact moments (wh…
Knowledge DistillationHyperGraphPro: Progress-Aware Reinforcement Learning for Structure-Guided Hypergraph RAG
Graph Retrieval-Augmented Generation (GraphRAG) has emerged as a promising paradigm that organizes external knowledge into structured graphs of entities and relations, enabling large language models (LLMs) to perform com…
Multi-hop Question AnsweringReinforcement LearningSemantic SimilarityKnowledge GraphsPersonality-guided Public-Private Domain Disentangled Hypergraph-Former Network for Multimodal Depression Detection
Depression represents a global mental health challenge requiring efficient and reliable automated detection methods. Current Transformer- or Graph Neural Networks (GNNs)-based multimodal depression detection methods face…
Representation LearningContrastive Learning