COMAP: Co-Evolving World Models and Agent Policies for LLM Agents

2026-06-01 · Youwei Liu, Jian Wang, Hanlin Wang, Wenjie Li arxiv

Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution. However, existing textual world models are typically fixed after training, preventing them from adapting to the on-policy state-action distributions induced by an evolving agent. Meanwhile, agent-improvement methods often rely on external rewards or verifiers, limiting their applicability in realistic interactive environments. In this paper, we propose COMAP, a novel framework that co-evolves textual world models and agent policies through closed-loop interaction. At each decision step, the world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback and refining its action accordingly. The resulting on-policy trajectories are then used to update the world model via self-distillation, allowing it to better match the agent's evolving interaction distribution. Across embodied task planning, Web navigation, and tool-use benchmarks, COMAP consistently outperforms competitive baselines, e.g., +16.75% relative improvement with Qwen3-4B. Further analyses show that the co-evolutionary loop improves the world model's prediction accuracy over time and leads to more effective long-horizon decision-making. Our code is available at: https://github.com/loyiv/CoMAP.

📄 PDF Abstract BibTeX arXiv:2606.02372

Code (0)

등록된 구현이 없습니다.

CoMaPOI: A Collaborative Multi-Agent Framework for Next POI Prediction Bridging the Gap Between Trajectory and Language

2025-05-28 · Lin Zhong, Lingzhi Wang, Xu Yang, Qing Liao

Large Language Models (LLMs) offer new opportunities for the next Point-Of-Interest (POI) prediction task, leveraging their capabilities in semantic understanding of POI trajectories. However, previous LLM-based methods,…

Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence

2026-04-20 · Guanting Dong, Junting Lu, Junjie Huang, Wanjun Zhong 외 arxiv

Large language models are increasingly expected to serve as general-purpose agents that interact with external, stateful tool environments. The Model Context Protocol (MCP) and broader agent skills offer a unified interf…

Reinforcement Learning

CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

2026-09-16 · Jiaxuan Jiang, Liyuan He, Zhixuan Fang hf

Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing stra…

Reinforcement Learning

Joint Learning of Experiential Rules and Policies for Large Language Model Agents

2026-06-25 · Shicheng Ye, Chao Yu arxiv

For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience. Existing work has typically separated two uses of such experience: keeping it outside…

Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning

2021-06-23 · Stephen Kelly, Tatiana Voegerl, Wolfgang Banzhaf, Cedric Gondro

A fundamental aspect of behaviour is the ability to encode salient features of experience in memory and use these memories, in combination with current sensory information, to predict the best action for each situation s…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)