Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates
In 2021 the Johns Hopkins University Applied Physics Laboratory held an internal challenge to develop artificially intelligent (AI) agents that could excel at the collaborative card game Hanabi. Agents were evaluated on their ability to play with human players whom the agents had never previously encountered. This study details the development of the agent that won the challenge by achieving a human-play average score of 16.5, outperforming the current state-of-the-art for human-bot Hanabi scores. The winning agent's development consisted of observing and accurately modeling the author's decision making in Hanabi, then training with a behavioral clone of the author. Notably, the agent discovered a human-complementary play style by first mimicking human decision making, then exploring variations to the human-like strategy that led to higher simulated human-bot scores. This work examines in detail the design and implementation of this human compatible Hanabi teammate, as well as the existence and implications of human-complementary strategies and how they may be explored for more successful applications of AI in human machine teams.
Code (1)
Tasks
Decision Makingreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
In Pursuit of Predictive Models of Human Preferences Toward AI Teammates
We seek measurable properties of AI agents that make them better or worse teammates from the subjective perspective of human collaborators. Our experiments use the cooperative card game Hanabi -- a common benchmark for A…
DiversityImplicitly Aligning Humans and Autonomous Agents through Shared Task Abstractions
In collaborative tasks, autonomous agents fall short of humans in their capability to quickly adapt to new and unfamiliar teammates. We posit that a limiting factor for zero-shot coordination is the lack of shared task a…
Hierarchical Reinforcement LearningMulti-Armed Bandits with Fairness Constraints for Distributing Resources to Human Teammates
How should a robot that collaborates with multiple people decide upon the distribution of resources (e.g. social attention, or parts needed for an assembly)? People are uniquely attuned to how resources are distributed. …
FairnessMulti-Armed BanditsReinforcement LearningCollaborative Causal Sensemaking: Closing the Complementarity Gap in Human-AI Decision Support
LLM-based agents are increasingly deployed for expert decision support, yet human-AI teams in high-stakes settings do not yet reliably outperform the best individual. We argue this complementarity gap reflects a fundamen…
Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming
Information sharing is key in building team cognition and enables coordination and cooperation. High-performing human teams also benefit from acting strategically with hierarchical levels of iterated communication and ra…
Decision MakingMulti-agent Reinforcement Learning