Reliable Policy Iteration: Performance Robustness Across Architecture and Environment Perturbations
In a recent work, we proposed Reliable Policy Iteration (RPI), that restores policy iteration's monotonicity-of-value-estimates property to the function approximation setting. Here, we assess the robustness of RPI's empirical performance on two classical control tasks -- CartPole and Inverted Pendulum -- under changes to neural network and environmental parameters. Relative to DQN, Double DQN, DDPG, TD3, and PPO, RPI reaches near-optimal performance early and sustains this policy as training proceeds. Because deep RL methods are often hampered by sample inefficiency, training instability, and hyperparameter sensitivity, our results highlight RPI's promise as a more reliable alternative.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On the Convergence of Modified Policy Iteration in Risk Sensitive Exponential Cost Markov Decision Processes
Modified policy iteration (MPI) is a dynamic programming algorithm that combines elements of policy iteration and value iteration. The convergence of MPI has been well studied in the context of discounted and average-cos…
Computational EfficiencyRobust Regularized Policy Iteration under Transition Uncertainty
Offline reinforcement learning (RL) enables data-efficient and safe policy learning without online exploration, but its performance often degrades under distribution shift. The learned policy may visit out-of-distributio…
Reinforcement LearningOffline RLAction Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimization
Reinforcement Learning (RL) has achieved remarkable success in sequential decision tasks. However, recent studies have revealed the vulnerability of RL policies to different perturbations, raising concerns about their ef…
Reinforcement LearningTowards Efficient Risk-Sensitive Policy Gradient: An Iteration Complexity Analysis
Reinforcement Learning (RL) has shown exceptional performance across various applications, enabling autonomous agents to learn optimal policies through interaction with their environments. However, traditional RL framewo…
Policy Gradient MethodsReinforcement Learning (RL)Robot NavigationLearning Robust Options
Robust reinforcement learning aims to produce policies that have strong guarantees even in the face of environments/transition models whose parameters have strong uncertainty. Existing work uses value-based methods and t…
Reinforcement Learning