Reproducible Multimodal Affordance Prediction
Affordance prediction is the identification of potential actions an agent can perform on a target object from multimodal inputs. Affordance prediction methods are difficult to evaluate and compare due to heterogeneous problem formulations, inconsistent dataset annotations, incomplete reporting of experimental protocols, and limited information about deployment conditions. These limitations challenge fair benchmarking and performance comparison. To promote transparency, we propose the Affordance Sheet, a documentation detailing task formulation with its input modalities, model architectures and training information, datasets, and experimental protocols. Affordance Sheets enable reproducible benchmarking and reliable evaluation of affordance models for real-world scenarios, including generalisation to novel conditions and human safety.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Egocentric Instruction-oriented Affordance Prediction via Large Multimodal Model
Affordance is crucial for intelligent robots in the context of object manipulation. In this paper, we argue that affordance should be task-/instruction-dependent, which is overlooked by many previous works. That is, diff…
Segmenting Object Affordances: Reproducibility and Sensitivity to Scale
Visual affordance segmentation identifies image regions of an object an agent can interact with. Existing methods re-use and adapt learning-based architectures for semantic segmentation to the affordance segmentation tas…
ObjectSegmentationSemantic SegmentationSensitivityRoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
Robotic manipulation and navigation are fundamental capabilities of embodied intelligence, enabling effective robot interactions with the physical world. Achieving these capabilities requires a cohesive understanding of …
Affordance RecognitionScene UnderstandingObject RecognitionQuestion AnsweringSeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model
3D affordance segmentation aims to link human instructions to touchable regions of 3D objects for embodied manipulations. Existing efforts typically adhere to single-object, single-affordance paradigms, where each afford…
Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language Model+2Text-driven object affordance for guiding grasp-type recognition in multimodal robot teaching
This study investigates how text-driven object affordance, which provides prior knowledge about grasp types for each object, affects image-based grasp-type recognition in robot teaching. The researchers created labeled d…
Mixed RealityObjectVocal Bursts Type Prediction