Affordance Extraction with an External Knowledge Database for Text-Based Simulated Environments
Text-based simulated environments have proven to be a valid testbed for machine learning approaches. The process of affordance extraction can be used to generate possible actions for interaction within such an environment. In this paper the capabilities and challenges for utilizing external knowledge databases (in particular ConceptNet) in the process of affordance extraction are studied. An algorithm for automated affordance extraction is introduced and evaluated on the Interactive Fiction (IF) platforms TextWorld and Jericho. For this purpose, the collected affordances are translated into text commands for IF agents. To probe the quality of the automated evaluation process, an additional human baseline study is conducted. The paper illustrates that, despite some challenges, external databases can in principle be used for affordance extraction. The paper concludes with recommendations for further modification and improvement of the process.
Code (0)
등록된 구현이 없습니다.
Tasks
validSimilar Papers 제목 키워드 기반
What can you do with a rock? Affordance extraction via word embeddings
Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance detection is particularly helpful in domains with large action spaces, allowing the agent to prune its search space…
Affordance DetectionReinforcement LearningReinforcement Learning (RL)Word EmbeddingsLearning 2D Invariant Affordance Knowledge for 3D Affordance Grounding
3D Object Affordance Grounding aims to predict the functional regions on a 3D object and has laid the foundation for a wide range of applications in robotics. Recent advances tackle this problem via learning a mapping be…
Human-Object Interaction DetectionObjectTRACE: Textual Reasoning for Affordance Coordinate Extraction
Vision-Language Models (VLMs) struggle to translate high-level instructions into the precise spatial affordances required for robotic manipulation. While visual Chain-of-Thought (CoT) methods exist, they are often comput…
Spatial ReasoningA large scale multi-view RGBD visual affordance learning dataset
The physical and textural attributes of objects have been widely studied for recognition, detection and segmentation tasks in computer vision.~A number of datasets, such as large scale ImageNet, have been proposed for fe…
Affordance RecognitionSegmentationOakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction
Learning how humans manipulate objects requires machines to acquire knowledge from two perspectives: one for understanding object affordances and the other for learning human's interactions based on the affordances. Even…
Grasp GenerationObjectPose Estimation