Policy with stochastic hysteresis
The paper develops a general methodology for analyzing policies with path-dependency (hysteresis) in stochastic models with forward looking optimizing agents. Our main application is a macro-climate model with a path-dependent climate externality. We derive in closed form the dynamics of the optimal Pigouvian tax, that is, its drift and diffusion coefficients. The dynamics of the present marginal damages is given by the recently developed functional It\^o formula. The dynamics of the conditional expectation process of the future marginal damages is given by a new total derivative formula that we prove. The total derivative formula represents the evolution of the conditional expectation process as a sum of the expected dynamics of hysteresis with respect to time, a form of a time derivative, and the expected dynamics of hysteresis with the shocks to the trajectory of the stochastic process, a form of a stochastic derivative. We then generalize the results. First, we propose a general class of hysteresis functionals that permits significant tractability. Second, we characterize in closed form the dynamics of the stochastic hysteresis elasticity that represents the change in the whole optimal policy process with an introduction of small hysteresis effects. Third, we determine the optimal policy process.
Code (0)
등록된 구현이 없습니다.
Tasks
FormMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transient hysteresis and inherent stochasticity in gene regulatory networks
Cell fate determination, the process through which cells commit to differentiated states is commonly mediated by gene regulatory motifs with mutually exclusive expression states. The classical deterministic picture for c…
Hysteresis-Based RL: Robustifying Reinforcement Learning-based Control Policies via Hybrid Control
Reinforcement learning (RL) is a promising approach for deriving control policies for complex systems. As we show in two control problems, the derived policies from using the Proximal Policy Optimization (PPO) and Deep Q…
reinforcement-learningReinforcement Learning (RL)Minimal Computational Preconditions for Subjective Perspective in Artificial Agents
This study operationalizes subjective perspective in artificial agents by grounding it in a minimal, phenomenologically motivated internal structure. The perspective is implemented as a slowly evolving global latent stat…
Wealth Effect on Portfolio Allocation in Incomplete Markets
We develop a novel five-component decomposition of optimal dynamic portfolio choice, which reveals the simultaneous impacts from market incompleteness and wealth-dependent utilities. Under the HARA utility and a nonrando…
Reference-Augmented Learning for Precise Tracking Policy of Tendon-Driven Continuum Robots
Tendon-Driven Continuum Robots (TDCRs) pose significant control challenges due to their highly nonlinear, path-dependent dynamics and non-Markovian characteristics. Traditional Jacobian-based controllers often struggle w…