Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulation

2025-05-07 · Abdulaziz Almuzairee, Rohan Patil, Dwait Bhatt, Henrik I. Christensen

Vision is well-known for its use in manipulation, especially using visual servoing. To make it robust, multiple cameras are needed to expand the field of view. That is computationally challenging. Merging multiple views and using Q-learning allows the design of more effective representations and optimization of sample efficiency. Such a solution might be expensive to deploy. To mitigate this, we introduce a Merge And Disentanglement (MAD) algorithm that efficiently merges views to increase sample efficiency while augmenting with single-view features to allow lightweight deployment and ensure robust policies. We demonstrate the efficiency and robustness of our approach using Meta-World and ManiSkill3. For project website and code, see https://aalmuzairee.github.io/mad

📄 PDF Abstract BibTeX arXiv:2505.04619

Code (0)

등록된 구현이 없습니다.

Tasks

DisentanglementLightweight DeploymentQ-Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Deep Adversarial Reinforcement Learning for Object Disentangling

2020-03-08 · Melvin Laux, Oleg Arenz, Jan Peters, Joni Pajarinen

Deep learning in combination with improved training techniques and high computational power has led to recent advances in the field of reinforcement learning (RL) and to successful robotic RL applications such as in-hand…

Object reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning

2020-02-21 · Yuanyi Zhong, Alexander Schwing, Jian Peng

In many vision-based reinforcement learning (RL) problems, the agent controls a movable object in its visual field, e.g., the player's avatar in video games and the robotic arm in visual grasping and manipulation. Levera…

Atari Games Object Prediction reinforcement-learning+3

Sim2Real View Invariant Visual Servoing by Recurrent Control

2017-12-20 · Fereshteh Sadeghi, Alexander Toshev, Eric Jang, Sergey Levine

Humans are remarkably proficient at controlling their limbs and tools from a wide range of viewpoints and angles, even in the presence of optical distortions. In robotics, this ability is referred to as visual servoing: …

Reinforcement Learning

Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control

2018-06-01 · CVPR 2018 6 · Fereshteh Sadeghi, Alexander Toshev, Eric Jang, Sergey Levine

Humans are remarkably proficient at controlling their limbs and tools from a wide range of viewpoints. In robotics, this ability is referred to as visual servoing: moving a tool or end-point to a desired location using p…

Reinforcement Learning Robot Manipulation

Robotic Vision and Multi-View Synergy: Action and activity recognition in assisted living scenarios

2024-09-01 · 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob) 2024 9 · Mohammad Hossein Bamorovat Abadi, Mohamad Reza Shahabian Alashti, Patrick Holthaus, Catherine Menon 외

The significance of Human-Robot Interaction (HRI) is increasingly evident when integrating robotics within human-centric settings. A crucial component of effective HRI is Human Activity Recognition (HAR), which is instru…

Activity Recognition Human Activity Recognition