A Multimodal Handover Failure Detection Dataset and Baselines
An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.
Code (1)
Tasks
Action SegmentationObjectTemporal Action SegmentationVideo ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bandit Policies for Reliable Cellular Network Handovers in Extreme Mobility
The demand for seamless Internet access under extreme user mobility, such as on high-speed trains and vehicles, has become a norm rather than an exception. However, the 4G/5G mobile network is not always reliable to meet…
Thompson SamplingReasons People Want Explanations After Unrecoverable Pre-Handover Failures
Most research on human-robot handovers focuses on the development of comfortable and efficient HRI; few have studied handover failures. If a failure occurs in the beginning of the interaction, it prevents the whole hando…
Machine Learning-Driven Anomaly Detection for 5G O-RAN Performance Metrics
The ever-increasing reliance of critical services on network infrastructure coupled with the increased operational complexity of beyond-5G/6G networks necessitate the need for proactive and automated network fault manage…
Anomaly DetectionTemporal Tactile Encoding and Compliance for Intent-Aware Robot-to-Human Bimanual Handover
Reliable robot-to-human handover requires the robot to infer when the person is ready to receive the object, and release it safely, comfortably, and at the right time. This is challenging because visual observations alon…
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
We introduce a new simulation benchmark "HandoverSim" for human-to-robot object handovers. To simulate the giver's motion, we leverage a recent motion capture dataset of hand grasping of objects. We create training and e…