Exploring Computational User Models for Agent Policy Summarization
AI agents are being developed to support high stakes decision-making processes from driving cars to prescribing drugs, making it increasingly important for human users to understand their behavior. Policy summarization methods aim to convey strengths and weaknesses of such agents by demonstrating their behavior in a subset of informative states. Some policy summarization methods extract a summary that optimizes the ability to reconstruct the agent's policy under the assumption that users will deploy inverse reinforcement learning. In this paper, we explore the use of different models for extracting summaries. We introduce an imitation learning-based approach to policy summarization; we demonstrate through computational simulations that a mismatch between the model used to extract a summary and the model used to reconstruct the policy results in worse reconstruction quality; and we demonstrate through a human-subject study that people use different models to reconstruct policies in different contexts, and that matching the summary extraction model to these can improve performance. Together, our results suggest that it is important to carefully consider user models in policy summarization.
Code (1)
Tasks
Decision MakingImitation LearningReinforcement LearningSimilar Papers 제목 키워드 기반
Explaining Decentralized Multi-Agent Reinforcement Learning Policies
Multi-Agent Reinforcement Learning (MARL) has gained significant interest in recent years, enabling sequential decision-making across multiple agents in various domains. However, most existing explanation methods focus o…
Multi-agent Reinforcement LearningComputational EfficiencyCompare and Select: Video Summarization with Multi-Agent Reinforcement Learning
Video summarization aims at generating concise video summaries from the lengthy videos, to achieve better user watching experience. Due to the subjectivity, purely supervised methods for video summarization may bring the…
Decision MakingMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+3Toward Policy Explanations for Multi-Agent Reinforcement Learning
Advances in multi-agent reinforcement learning (MARL) enable sequential decision making for a range of exciting multi-agent applications such as cooperative AI and autonomous driving. Explaining agent decisions is crucia…
Autonomous DrivingDecision MakingHuman Agent CollaborationMulti-agent Reinforcement Learning+4"I Don't Think So": Summarizing Policy Disagreements for Agent Comparison
With Artificial Intelligence on the rise, human interaction with autonomous agents becomes more frequent. Effective human-agent collaboration requires users to understand the agent's behavior, as failing to do so may cau…
Human Agent CollaborationSySLLM: Generating Synthesized Policy Summaries for Reinforcement Learning Agents Using Large Language Models
Policies generated by Reinforcement Learning (RL) algorithms can be difficult to describe to users, as they result from the interplay between complex reward structures and neural network-based representations. This combi…
Reinforcement Learning (RL)World Knowledge