홈 › Papers

Hindsight Credit Assignment

2019-12-05 · NeurIPS 2019 12 · Anna Harutyunyan, Will Dabney, Thomas Mesnard, Mohammad Azar, Bilal Piot, Nicolas Heess, Hado van Hasselt, Greg Wayne, Satinder Singh, Doina Precup, Remi Munos

We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit to past decisions based on the likelihood of them having led to the observed outcome. This approach uses new information in hindsight, rather than employing foresight. Somewhat surprisingly, we show that value functions can be rewritten through this lens, yielding a new family of algorithms. We study the properties of these algorithms, and empirically show that they successfully address important credit assignment challenges, through a set of illustrative tasks.

📄 PDF Abstract BibTeX arXiv:1912.02503

Code (1)

hca-neurips2019/hca 공식 구현

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Towards Causal Credit Assignment

2022-12-22 · Mátyás Schubert

Adequately assigning credit to actions for future outcomes based on their contributions is a long-standing open challenge in Reinforcement Learning. The assumptions of the most commonly used credit assignment method are …

counterfactual

Hindsight Network Credit Assignment

2020-11-24 · Kenny Young

We present Hindsight Network Credit Assignment (HNCA), a novel learning method for stochastic neural networks, which works by assigning credit to each neuron's stochastic output based on how it influences the output of i…

Towards Practical Credit Assignment for Deep Reinforcement Learning

2021-06-08 · Vyacheslav Alipov, Riley Simmons-Edler, Nikita Putintsev, Pavel Kalinin 외

Credit assignment is a fundamental problem in reinforcement learning, the problem of measuring an action's influence on future rewards. Explicit credit assignment methods have the potential to boost the performance of RL…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Variance Reduced Advantage Estimation with $δ$ Hindsight Credit Assignment

2019-11-19 · Kenny Young

Hindsight Credit Assignment (HCA) refers to a recently proposed family of methods for producing more efficient credit assignment in reinforcement learning. These methods work by explicitly estimating the probability that…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hindsight-DICE: Stable Credit Assignment for Deep Reinforcement Learning

2023-07-21 · Akash Velu, Skanda Vaidyanath, Dilip Arumugam

Oftentimes, environments for sequential decision-making problems can be quite sparse in the provision of evaluative feedback to guide reinforcement-learning agents. In the extreme case, long trajectories of behavior are …

Decision Making Deep Reinforcement Learning Off-policy evaluation Policy Gradient Methods+3