From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning

2026-05-22 · Ranxu zhang, zeyang li, Jiacheng Huang, Rui Zhang, Xiaozhou Xu, sun zhe, Yanyong Zhang, Chao Wang arxiv

Agentic reinforcement learning (Agentic RL) has achieved strong progress in tasks with clear success signals. However, many real-world agent applications require user-conditioned behavior: the same query may call for different planning strategies and tool-use decisions across users. This setting raises key challenges: generic rewards cannot capture heterogeneous user preferences, observed behaviors are entangled with conformity effects, and flat memories cannot support personalized skill retrieval. To this end, we propose a unified personalized Agentic RL framework that embeds personalization into training-time optimization. At its core is \emph{Personalized Anchor Reward-Decoupled Policy Optimization} (\textbf{PARPO}), which decouples generic task-quality rewards from personalized preference rewards and uses user-specific anchors to stabilize learning under heterogeneous reward scales. We further introduce a two-stage preference-disentangled reward model and \emph{Preference-Aligned Skill Evolution Graph Memory} (\textbf{PSGM}) for personalized supervision and preference-aligned skill retrieval. Together, they form a closed loop of preference identification, policy optimization, and structured skill accumulation. Experiments on ETAPP, ETAPP-Hard, and SJAgent show that our framework consistently outperforms strong memory and RL baselines. Code and data are included in the supplementary materials.

📄 PDF Abstract BibTeX arXiv:2605.23382

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

RecNet: Self-Evolving Preference Propagation for Agentic Recommender Systems

2026-01-29 · Bingqian Li, Xiaolei Wang, Junyi Li, Weitao Li 외 arxiv

Agentic recommender systems leverage Large Language Models (LLMs) to model complex user behaviors and support personalized decision-making. However, existing methods primarily model preference changes based on explicit u…

Multi-agent Reinforcement Learning

PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory

2025-12-07 · Bowen Jiang, Yuan Yuan, Maohao Shen, Zhuoqun Hao 외 arxiv

Personalization is one of the next milestones in advancing AI capability and alignment. We introduce PersonaMem-v2, the state-of-the-art dataset for LLM personalization that simulates 1,000 realistic user-chatbot interac…

Agentic Recommender System with Hierarchical Belief-State Memory

2026-05-14 · Xiang Shen, Yuhang Zhou, Yifan Wu, Zhuokai Zhao 외 arxiv

Memory-augmented LLM agents have advanced personalized recommendation, yet existing approaches universally adopt flat memory representations that conflate ephemeral signals with stable preferences, and none provides a co…

MemRerank: Preference Memory for Personalized Product Reranking

2026-03-31 · Zhiyuan Peng, Xuyang Wu, Huaixiao Tou, Yi Fang 외 arxiv

LLM-based shopping agents increasingly rely on long purchase histories and multi-turn interactions for personalization, yet naively appending raw history to prompts is often ineffective due to noise, length, and relevanc…

Reinforcement Learning

Synthetic Interaction Data for Scalable Personalization in Large Language Models

2026-02-12 · Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo 외 arxiv

Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking…

Synthetic Data Generation Reinforcement Learning