SCOD: Active Object Detection for Embodied Agents using Sensory Commutativity of Action Sequences
We introduce SCOD (Sensory Commutativity Object Detection), an active method for movable and immovable object detection. SCOD exploits the commutative properties of action sequences, in the scenario of an embodied agent equipped with first-person sensors and a continuous motor space with multiple degrees of freedom. SCOD is based on playing an action sequence in two different orders from the same starting point and comparing the two final observations obtained after each sequence. Our experiments on 3D realistic robotic setups (iGibson) demonstrate the accuracy of SCOD and its generalization to unseen environments and objects. We also successfully apply SCOD on a real robot to further illustrate its generalization properties. With SCOD, we aim at providing a novel way of approaching the problem of object discovery in the context of a naive embodied agent. We provide code and a supplementary video.
Code (0)
등록된 구현이 없습니다.
Tasks
Active Object DetectionObjectobject-detectionObject DetectionObject DiscoverySimilar Papers 제목 키워드 기반
Embodied Visual Recognition
Passive visual systems typically fail to recognize objects in the amodal setting where they are heavily occluded. In contrast, humans and other embodied agents have the ability to move in the environment, and actively co…
ObjectObject LocalizationSemantic SegmentationEmbodied Amodal Recognition: Learning to Move to Perceive Objects
Passive visual systems typically fail to recognize objects in the amodal setting where they are heavily occluded. In contrast, humans and other embodied agents have the ability to move in the environment and actively con…
ObjectObject LocalizationSemantic SegmentationEmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents
Multimodal Large Language Models (MLLMs) have shown significant advancements, providing a promising future for embodied agents. Existing benchmarks for evaluating MLLMs primarily utilize static images or videos, limiting…
AttributeQuestion AnsweringPInVerify: An Offline Embodied Benchmark for Active Instance Verification
Embodied agents have made strong progress in navigating to target objects, but reaching the goal vicinity does not guarantee that the agent has found the correct instance: subtle attribute differences (e.g., "white flora…
VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning
Salient object detection (SOD) and camouflaged object detection (COD) are related yet distinct binary mapping tasks. These tasks involve multiple modalities, sharing commonalities and unique cues. Existing research often…
DecoderModel Optimizationobject-detectionObject Detection+3