A Closed-Loop Multi-perspective Visual Servoing Approach with Reinforcement Learning
Traditional visual servoing methods suffer from serving between scenes from multiple perspectives, which humans can complete with visual signals alone. In this paper, we investigated how multi-perspective visual servoing could be solved under robot-specific constraints, including self-collision, singularity problems. We presented a novel learning-based multi-perspective visual servoing framework, which iteratively estimates robot actions from latent space representations of visual states using reinforcement learning. Furthermore, our approaches were trained and validated in a Gazebo simulation environment with connection to OpenAI/Gym. Through simulation experiments, we showed that our method can successfully learn an optimal control policy given initial images from different perspectives, and it outperformed the Direct Visual Servoing algorithm with mean success rate of 97.0%.
Code (0)
등록된 구현이 없습니다.
Tasks
OpenAI Gymreinforcement-learningSimilar Papers 제목 키워드 기반
Control of the Final-Phase of Closed-Loop Visual Grasping using Image-Based Visual Servoing
This paper considers the final approach phase of visual-closed-loop grasping where the RGB-D camera is no longer able to provide valid depth information. Many current robotic grasping controllers are not closed-loop and …
ObjectRobotic GraspingvalidMarkerless 6D Pose Estimation and Position-Based Visual Servoing for Endoscopic Continuum Manipulators
Continuum manipulators in flexible endoscopic surgical systems offer high dexterity for minimally invasive procedures; however, accurate pose estimation and closed-loop control remain challenging due to hysteresis, compl…
6D Pose EstimationPerception-Control Coupled Visual Servoing for Textureless Objects Using Keypoint-Based EKF
Visual servoing is fundamental to robotic applications, enabling precise positioning and control. However, applying it to textureless objects remains a challenge due to the absence of reliable visual features. Moreover, …
Keypoint DetectionTerminal Constraint Model Predictive Control for Image-Based Visual Servoing of UAVs with Kalman Filter-Based Moment Loss Compensation
Image-Based Visual Servoing (IBVS) provides an efficient vision-guided control paradigm for unmanned aerial vehicles (UAVs) by directly regulating image-space errors. However, conventional IBVS controllers are vulnerable…
Self-supervised Wide Baseline Visual Servoing via 3D Equivariance
One of the challenging input settings for visual servoing is when the initial and goal camera views are far apart. Such settings are difficult because the wide baseline can cause drastic changes in object appearance and …
Object