Learn to Change the World: Multi-level Reinforcement Learning with Model-Changing Actions

2025-10-16 · Ziqing Lu, Babak Hassibi, Lifeng Lai, Weiyu Xu arxiv

Reinforcement learning usually assumes a given or sometimes even fixed environment in which an agent seeks an optimal policy to maximize its long-term discounted reward. In contrast, we consider agents that are not limited to passive adaptations: they instead have model-changing actions that actively modify the RL model of world dynamics itself. Reconfiguring the underlying transition processes can potentially increase the agents' rewards. Motivated by this setting, we introduce the multi-layer configurable time-varying Markov decision process (MCTVMDP). In an MCTVMDP, the lower-level MDP has a non-stationary transition function that is configurable through upper-level model-changing actions. The agent's objective consists of two parts: Optimize the configuration policies in the upper-level MDP and optimize the primitive action policies in the lower-level MDP to jointly improve its expected long-term reward.

📄 PDF Abstract BibTeX arXiv:2510.15056

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

2022-06-25 · Zhixuan Liang, Jiannong Cao, Shan Jiang, Divya Saxena 외

Many real-world applications can be formulated as multi-agent cooperation problems, such as network packet routing and coordination of autonomous vehicles. The emergence of deep reinforcement learning (DRL) provides a pr…

Autonomous Vehicles Decision Making Deep Reinforcement Learning Hierarchical Reinforcement Learning+3

ViLaCD-R1: A Vision-Language Framework for Semantic Change Detection in Remote Sensing

2025-12-29 · Xingwei Ma, Shiyang Feng, Bo Zhang, Bin Wang arxiv

Remote sensing change detection (RSCD), a complex multi-image inference task, traditionally uses pixel-based operators or encoder-decoder networks that inadequately capture high-level semantics and are vulnerable to non-…

Reinforcement Learning Change Detection

Strategic Coordination for Evolving Multi-agent Systems: A Hierarchical Reinforcement and Collective Learning Approach

2025-09-22 · Chuhao Qin, Evangelos Pournaras arxiv

Decentralized combinatorial optimization in evolving multi-agent systems poses significant challenges, requiring agents to balance long-term decision-making, short-term optimized collective outcomes, while preserving aut…

Multi-agent Reinforcement Learning

Lane Change Decision-Making through Deep Reinforcement Learning

2021-12-24 · Mukesh Ghimire, Malobika Roy Choudhury, Guna Sekhar Sai Harsha Lagudu

Due to the complexity and volatility of the traffic environment, decision-making in autonomous driving is a significantly hard problem. In this project, we use a Deep Q-Network, along with rule-based constraints to make …

Autonomous Driving Decision Making Deep Reinforcement Learning reinforcement-learning+2

Data-Efficient Hierarchical Reinforcement Learning

2018-05-21 · NeurIPS 2018 12 · Ofir Nachum, Shixiang Gu, Honglak Lee, Sergey Levine

Hierarchical reinforcement learning (HRL) is a promising approach to extend traditional reinforcement learning (RL) methods to solve more complex tasks. Yet, the majority of current HRL methods require careful task-speci…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)