Deep Reinforcement Learning Behavioral Mode Switching Using Optimal Control Based on a Latent Space Objective
In this work, we use optimal control to change the behavior of a deep reinforcement learning policy by optimizing directly in the policy's latent space. We hypothesize that distinct behavioral patterns, termed behavioral modes, can be identified within certain regions of a deep reinforcement learning policy's latent space, meaning that specific actions or strategies are preferred within these regions. We identify these behavioral modes using latent space dimension-reduction with \ac*{pacmap}. Using the actions generated by the optimal control procedure, we move the system from one behavioral mode to another. We subsequently utilize these actions as a filter for interpreting the neural network policy. The results show that this approach can impose desired behavioral modes in the policy, demonstrated by showing how a failed episode can be made successful and vice versa using the lunar lander reinforcement learning environment.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningDimensionality Reductionreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching
We propose a comprehensive framework for policy gradient methods tailored to continuous time reinforcement learning. This is based on the connection between stochastic control problems and randomised problems, enabling a…
Policy Gradient MethodsTo Switch or Not to Switch? Balanced Policy Switching in Offline Reinforcement Learning
Reinforcement learning (RL) -- finding the optimal behaviour (also referred to as policy) maximizing the collected long-term cumulative reward -- is among the most influential approaches in machine learning with a large …
Offline RLReinforcement Learning (RL)A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation
The exploration-exploitation dilemma has been a central challenge in reinforcement learning (RL) with complex model classes. In this paper, we propose a new algorithm, Monotonic Q-Learning with Upper Confidence Bound (MQ…
Q-LearningReinforcement Learning (RL)Understanding Mode Switching in Human-AI Collaboration: Behavioral Insights and Predictive Modeling
Human-AI collaboration is typically offered in one of two of user control levels: guidance, where the AI provides suggestions and the human makes the final decision, and delegation, where the AI acts autonomously within …
Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective
In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity-akin to chameleons adapting their coloration to environmental cues-that can be exposed through token-conditional generatio…
Reinforcement LearningQuestion Answering