Advice Conformance Verification by Reinforcement Learning agents for Human-in-the-Loop

2022-10-07 · Mudit Verma, Ayush Kharkwal, Subbarao Kambhampati

Human-in-the-loop (HiL) reinforcement learning is gaining traction in domains with large action and state spaces, and sparse rewards by allowing the agent to take advice from HiL. Beyond advice accommodation, a sequential decision-making agent must be able to express the extent to which it was able to utilize the human advice. Subsequently, the agent should provide a means for the HiL to inspect parts of advice that it had to reject in favor of the overall environment objective. We introduce the problem of Advice-Conformance Verification which requires reinforcement learning (RL) agents to provide assurances to the human in the loop regarding how much of their advice is being conformed to. We then propose a Tree-based lingua-franca to support this communication, called a Preference Tree. We study two cases of good and bad advice scenarios in MuJoCo's Humanoid environment. Through our experiments, we show that our method can provide an interpretable means of solving the Advice-Conformance Verification problem by conveying whether or not the agent is using the human's advice. Finally, we present a human-user study with 20 participants that validates our method.

📄 PDF Abstract BibTeX arXiv:2210.03455

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making

Teachable Reinforcement Learning via Advice Distillation

2022-03-19 · NeurIPS 2021 12 · Olivia Watkins, Trevor Darrell, Pieter Abbeel, Jacob Andreas 외

Training automated agents to complete complex tasks in interactive environments is challenging: reinforcement learning requires careful hand-engineering of reward functions, imitation learning requires specialized infras…

Decision Making Imitation Learning reinforcement-learning Reinforcement Learning+1

Human Engagement Providing Evaluative and Informative Advice for Interactive Reinforcement Learning

2020-09-21 · Adam Bignold, Francisco Cruz, Richard Dazeley, Peter Vamplew 외

Interactive reinforcement learning proposes the use of externally-sourced information in order to speed up the learning process. When interacting with a learner agent, humans may provide either evaluative or informative …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Improving Deep Reinforcement Learning in Minecraft with Action Advice

2019-08-02 · Spencer Frazier, Mark Riedl

Training deep reinforcement learning agents complex behaviors in 3D virtual environments requires significant computational resources. This is especially true in environments with high degrees of aliasing, where many sta…

BIG-bench Machine Learning Deep Reinforcement Learning Minecraft reinforcement-learning+2

Influencing Reinforcement Learning through Natural Language Guidance

2021-04-04 · Tasmia Tasrin, Md Sultan Al Nahian, Habarakadage Perera, Brent Harrison

Interactive reinforcement learning agents use human feedback or instruction to help them learn in complex environments. Often, this feedback comes in the form of a discrete signal that is either positive or negative. Whi…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents

2026-04-01 · Alibek Kaliyev, Artem Maryanskyy arxiv

Agents that synthesize their own tools ship a second artifact alongside each answer: a software library that future tasks reuse, compose, and depend on. Task completion (TC) certifies the answer; it does not certify the …