Policy Space Identification in Configurable Environments
We study the problem of identifying the policy space of a learning agent, having access to a set of demonstrations generated by its optimal policy. We introduce an approach based on statistical testing to identify the set of policy parameters the agent can control, within a larger parametric policy space. After presenting two identification rules (combinatorial and simplified), applicable under different assumptions on the policy space, we provide a probabilistic analysis of the simplified one in the case of linear policies belonging to the exponential family. To improve the performance of our identification rules, we frame the problem in the recently introduced framework of the Configurable Markov Decision Processes, exploiting the opportunity of configuring the environment to induce the agent revealing which parameters it can control. Finally, we provide an empirical evaluation, on both discrete and continuous domains, to prove the effectiveness of our identification rules.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Synthetic Monitoring Environments for Reinforcement Learning
Reinforcement Learning (RL) lacks benchmarks that enable precise, white-box diagnostics of agent behavior. Current environments often entangle complexity factors and lack ground-truth optimality metrics, making it diffic…
Reinforcement LearningContinuous ControlTransfer Learning for Performance Modeling of Configurable Systems: A Causal Analysis
Modern systems (e.g., deep neural networks, big data analytics, and compilers) are highly configurable, which means they expose different performance behavior under different configurations. The fundamental challenge is …
Transfer LearningUnicorn: Reasoning about Configurable System Performance through the lens of Causality
Modern computer systems are highly configurable, with the total variability space sometimes larger than the number of atoms in the universe. Understanding and reasoning about the performance behavior of highly configurab…
BIG-bench Machine LearningCausal InferenceManagementNear-optimal Policy Identification in Active Reinforcement Learning
Many real-world reinforcement learning tasks require control of complex dynamical systems that involve both costly data acquisition processes and large state spaces. In cases where the transition dynamics can be readily …
Bayesian Optimizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Comparison of Policy Search in Joint Space and Cartesian Space for Refinement of Skills
Imitation learning is a way to teach robots skills that are demonstrated by humans. Transfering skills between these different kinematic structures seems to be straightforward in Cartesian space. Because of the correspon…
Imitation Learning