Perturbation-based exploration methods in deep reinforcement learning
Recent research on structured exploration placed emphasis on identifying novel states in the state space and incentivizing the agent to revisit them through intrinsic reward bonuses. In this study, we question whether the performance boost demonstrated through these methods is indeed due to the discovery of structure in exploratory schedule of the agent or is the benefit largely attributed to the perturbations in the policy and reward space manifested in pursuit of structured exploration. In this study we investigate the effect of perturbations in policy and reward spaces on the exploratory behavior of the agent. We proceed to show that simple acts of perturbing the policy just before the softmax layer and introduction of sporadic reward bonuses into the domain can greatly enhance exploration in several domains of the arcade learning environment. In light of these findings, we recommend benchmarking any enhancements to structured exploration research against the backdrop of noisy exploration.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesBenchmarkingDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration
Reinforcement learning with verifiable rewards, particularly Group Relative Policy Optimization (GRPO), has significantly advanced the reasoning capabilities of Large Language Models (LLMs). However, in complex tasks, GR…
Reinforcement LearningWhatever Does Not Kill Deep Reinforcement Learning, Makes It Stronger
Recent developments have established the vulnerability of deep Reinforcement Learning (RL) to policy manipulation attacks via adversarial perturbations. In this paper, we investigate the robustness and resilience of deep…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Exploration by Random Reward Perturbation
We introduce Random Reward Perturbation (RRP), a novel exploration strategy for reinforcement learning (RL). Our theoretical analyses demonstrate that adding zero-mean noise to environmental rewards effectively enhances …
DiversityReinforcement Learning (RL)State-Aware Variational Thompson Sampling for Deep Q-Networks
Thompson sampling is a well-known approach for balancing exploration and exploitation in reinforcement learning. It requires the posterior distribution of value-action functions to be maintained; this is generally intrac…
Thompson SamplingNADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning
Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent exploration. To achieve these, a stochasti…
continuous-controlContinuous ControlDeep Reinforcement LearningMuJoCo+3