Dynamic Programming with State-Dependent Discounting

2020-10-14

This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results can be recovered. We also show that the condition cannot be significantly weakened. Our framework is general enough to handle complications such as recursive preferences and unbounded rewards. Economic and financial applications are discussed.

📄 PDF Abstract BibTeX arXiv:1908.08800

Code (0)

등록된 구현이 없습니다.

Perov's Contraction Principle and Dynamic Programming with Stochastic Discounting

2021-03-25 · Alexis Akira Toda

This paper shows the usefulness of Perov's contraction principle, which generalizes Banach's contraction principle to a vector-valued metric, for studying dynamic programming problems in which the discount factor can be …

AdaGamma: State-Dependent Discounting for Temporal Adaptation in Reinforcement Learning

2026-05-07 · Yaomin Wang, Jianting Pan, Ran Tian, Xiaoyang Li 외 arxiv

The discount factor in reinforcement learning controls both the effective planning horizon and the strength of bootstrapping, yet most deep RL methods use a single fixed value across all states. While state-dependent dis…

Reinforcement Learning

Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting

2026-05-20 · Hojin Ko, Jeonggyu Huh arxiv

Most value-based and actor--critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We s…

Reinforcement Learning

Examining average and discounted reward optimality criteria in reinforcement learning

2021-07-03 · Vektor Dewanto, Marcus Gallagher

In reinforcement learning (RL), the goal is to obtain an optimal policy, for which the optimality criterion is fundamentally important. Two major optimality criteria are average and discounted rewards. While the latter i…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Relating Reinforcement Learning to Dynamic Programming-Based Planning

2026-03-08 · Filip V. Georgiev, Kalle G. Timperi, Başak Sakçak, Steven M. LaValle arxiv

This paper bridges some of the gap between optimal planning and reinforcement learning (RL), both of which share roots in dynamic programming applied to sequential decision making or optimal control. Whereas planning typ…

Reinforcement Learning Decision Making