Optimizing Deep Reinforcement Learning for Adaptive Robotic Arm Control

2024-06-12 · Jonaid Shianifar, Michael Schukat, Karl Mason

In this paper, we explore the optimization of hyperparameters for the Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms using the Tree-structured Parzen Estimator (TPE) in the context of robotic arm control with seven Degrees of Freedom (DOF). Our results demonstrate a significant enhancement in algorithm performance, TPE improves the success rate of SAC by 10.48 percentage points and PPO by 34.28 percentage points, where models trained for 50K episodes. Furthermore, TPE enables PPO to converge to a reward within 95% of the maximum reward 76% faster than without TPE, which translates to about 40K fewer episodes of training required for optimal performance. Also, this improvement for SAC is 80% faster than without TPE. This study underscores the impact of advanced hyperparameter optimization on the efficiency and success of deep reinforcement learning algorithms in complex robotic tasks.

📄 PDF Abstract BibTeX arXiv:2407.02503

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningHyperparameter Optimization reinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Dilated Convolution 설명 없음

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Average Pooling 설명 없음

SAC 설명 없음

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space

2021-10-19 · Maximilian Ulmer, Elie Aljalbout, Sascha Schwarz, Sami Haddadin

Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in r…

Contact-rich Manipulation Decision Making reinforcement-learning Reinforcement Learning+1

TARC: Time-Adaptive Robotic Control

2025-10-27 · Arnav Sukhija, Lenart Treven, Jin Cheng, Florian Dörfler 외 arxiv

Most robotic systems rely on fixed-frequency discrete-time controllers, creating a trade-off between the efficiency of low-frequency control and the responsiveness of high-frequency feedback. As a result, systems typical…

Reinforcement Learning

Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control

2025-01-29 · Matsive Ali, Sandesh Giri, Sen Liu, Qin Yang

With the rapid development of deep reinforcement learning technology, it gradually demonstrates excellent potential and is becoming the most promising solution in the robotics. However, in the smart manufacturing domain,…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Transfer Learning+1

Quantum Computing and Neuromorphic Computing for Safe, Reliable, and explainable Multi-Agent Reinforcement Learning: Optimal Control in Autonomous Robotics

2024-07-29 · Mazyar Taghavi

This paper investigates the utilization of Quantum Computing and Neuromorphic Computing for Safe, Reliable, and Explainable Multi_Agent Reinforcement Learning (MARL) in the context of optimal control in autonomous roboti…

Multi-agent Reinforcement Learning

A Novel Sample-efficient Deep Reinforcement Learning with Episodic Policy Transfer for PID-Based Control in Cardiac Catheterization Robots

2021-10-28 · Olatunji Mumini Omisore, Toluwanimi Akinyemi, Wenke Duan, Wenjing Du 외

Robotic catheterization is typically used for percutaneous coronary intervention procedures nowadays and it involves steering flexible endovascular tools to open up occlusion in the coronaries. In this study, a sample-ef…

Deep Reinforcement Learning