Learning Policies through Quantile Regression
Policy gradient based reinforcement learning algorithms coupled with neural networks have shown success in learning complex policies in the model free continuous action space control setting. However, explicitly parameterized policies are limited by the scope of the chosen parametric probability distribution. We show that alternatively to the likelihood based policy gradient, a related objective can be optimized through advantage weighted quantile regression. Our approach models the policy implicitly in the network, which gives the agent the freedom to approximate any distribution in each action dimension, not limiting its capabilities to the commonly used unimodal Gaussian parameterization. This broader spectrum of policies makes our algorithm suitable for problems where Gaussian policies cannot fit the optimal policy. Moreover, our results on the MuJoCo physics simulator benchmarks are comparable or superior to state-of-the-art on-policy methods.
Code (0)
등록된 구현이 없습니다.
Tasks
MuJoCoquantile regressionregressionReinforcement LearningSimilar Papers 제목 키워드 기반
Implicit Quantile Networks for Distributional Reinforcement Learning
In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regressio…
Atari GamesDistributional Reinforcement Learningquantile regressionregression+3Generative Adversarial Regression (GAR): Learning Conditional Risk Scenarios
We propose Generative Adversarial Regression (GAR), a framework for learning conditional risk scenarios through generators aligned with downstream risk objectives. GAR builds on a regression characterization of condition…
Interpreting Unconditional Quantile Regression with Conditional Independence
This note provides additional interpretation for the counterfactual outcome distribution and corresponding unconditional quantile "effects" defined and estimated by Firpo, Fortin, and Lemieux (2009) and Chernozhukov, Fer…
counterfactualquantile regressionregressionQuantile Regression for Distributional Reward Models in RLHF
Reinforcement learning from human feedback (RLHF) has become a key method for aligning large language models (LLMs) with human preferences through the use of reward models. However, traditional reward models typically ge…
Diversityquantile regressionregressionreinforcement-learning+1Decoupling Shrinkage and Selection for the Bayesian Quantile Regression
This paper extends the idea of decoupling shrinkage and sparsity for continuous priors to Bayesian Quantile Regression (BQR). The procedure follows two steps: In the first step, we shrink the quantile regression posterio…
quantile regressionregressionvalidVariable Selection