Visual search and recognition for robot task execution and monitoring
Visual search of relevant targets in the environment is a crucial robot skill. We propose a preliminary framework for the execution monitor of a robot task, taking care of the robot attitude to visually searching the environment for targets involved in the task. Visual search is also relevant to recover from a failure. The framework exploits deep reinforcement learning to acquire a "common sense" scene structure and it takes advantage of a deep convolutional network to detect objects and relevant relations holding between them. The framework builds on these methods to introduce a vision-based execution monitoring, which uses classical planning as a backbone for task execution. Experiments show that with the proposed vision-based execution monitor the robot can complete simple tasks and can recover from failures in autonomy.
Code (0)
등록된 구현이 없습니다.
Tasks
Common Sense ReasoningDeep Reinforcement LearningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Vision-based deep execution monitoring
Execution monitor of high-level robot actions can be effectively improved by visual monitoring the state of the world in terms of preconditions and postconditions that hold before and after the execution of an action. Fu…
Deep Reinforcement LearningReinforcement LearningReal-Time Fruit Recognition and Grasping Estimation for Autonomous Apple Harvesting
In this research, a fully neural network based visual perception framework for autonomous apple harvesting is proposed. The proposed framework includes a multi-function neural network for fruit recognition and a Pointnet…
Instance SegmentationRobotic GraspingSemantic SegmentationDeep execution monitor for robot assistive tasks
We consider a novel approach to high-level robot task execution for a robot assistive task. In this work we explore the problem of learning to predict the next subtask by introducing a deep model for both sequencing goal…
Running hardware-aware neural architecture search on embedded devices under 512MB of RAM
This document proposes a novel approach to hardware-aware neural architecture search (HW NAS) that considers the resources available on the computing platform running it, enabling its execution on various embedded device…
Neural Architecture SearchSignVLA: Real-Time Sign Language-Guided Robotic Manipulation via Attention LSTM and Vision-Language-Action Models
Vision-Language-Action (VLA) models enable robots to execute manipulation tasks from natural-language instructions grounded in visual observations. However, existing VLA interfaces primarily rely on speech or text input,…