Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrations
Learning from Demonstration (LfD) is a popular approach for robots to acquire new skills, but most LfD methods suffer from imperfections in human demonstrations. Prior work typically treats these suboptimalities as random noise. In this paper we study non-optimal behaviors in non-expert demonstrations and show that they are systematic, forming what we call demonstration sidetracks. Using a public space study with 40 participants performing a long-horizon robot task, we recreated the setup in simulation and annotated all demonstrations. We identify four types of sidetracks (Exploration, Mistake, Alignment, Pause) and one control pattern (one-dimension control). Sidetracks appear frequently across participants, and their temporal and spatial distribution is tied to task context. We also find that users' control patterns depend on the control interface. These insights point to the need for better models of suboptimal demonstrations to improve LfD algorithms and bridge the gap between lab training and real-world deployment. All demonstrations, infrastructure, and annotations are available at https://github.com/AABL-Lab/Human-Demonstration-Sidetracks.
Code (1)
Similar Papers 제목 키워드 기반
Towards Sample-efficient Apprenticeship Learning from Suboptimal Demonstration
Learning from Demonstration (LfD) seeks to democratize robotics by enabling non-roboticist end-users to teach robots to perform novel tasks by providing demonstrations. However, as demonstrators are typically non-experts…
Inverse Reinforcement Learning via Matching of Optimality Profiles
The goal of inverse reinforcement learning (IRL) is to infer a reward function that explains the behavior of an agent performing a task. The assumption that most approaches make is that the demonstrated behavior is near-…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality
Most existing imitation learning approaches assume the demonstrations are drawn from experts who are optimal, but relaxing this assumption enables us to use a wider range of data. Standard imitation learning may learn a …
Imitation LearningSkill Disentanglement for Imitation Learning from Suboptimal Demonstrations
Imitation learning has achieved great success in many sequential decision-making tasks, in which a neural agent is learned by imitating collected human demonstrations. However, existing algorithms typically require a lar…
Decision MakingDisentanglementImitation LearningSequential Decision MakingThe Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models
Models of human behavior for prediction and collaboration tend to fall into two categories: ones that learn from large amounts of data via imitation learning, and ones that assume human behavior to be noisily-optimal for…
Bayesian InferenceImitation Learning