Watch-Bot: Unsupervised Learning for Reminding Humans of Forgotten Actions
We present a robotic system that watches a human using a Kinect v2 RGB-D sensor, detects what he forgot to do while performing an activity, and if necessary reminds the person using a laser pointer to point out the related object. Our simple setup can be easily deployed on any assistive robot. Our approach is based on a learning algorithm trained in a purely unsupervised setting, which does not require any human annotations. This makes our approach scalable and applicable to variant scenarios. Our model learns the action/object co-occurrence and action temporal relations in the activity, and uses the learned rich relationships to infer the forgotten action and the related object. We show that our approach not only improves the unsupervised action segmentation and action cluster assignment performance, but also effectively detects the forgotten actions on a challenging human activity RGB-D video dataset. In robotic experiments, we show that our robot is able to remind people of forgotten actions successfully.
Code (0)
등록된 구현이 없습니다.
Tasks
Action SegmentationObjectUnsupervised Action SegmentationSimilar Papers 제목 키워드 기반
Watch-n-Patch: Unsupervised Learning of Actions and Relations
There is a large variation in the activities that humans perform in their everyday lives. We consider modeling these composite human activities which comprises multiple basic level actions in a completely unsupervised se…
Action SegmentationClusteringUnsupervised Action SegmentationWatch-n-Patch: Unsupervised Understanding of Actions and Relations
We focus on modeling human activities comprising multiple actions in a completely unsupervised setting. Our model learns the high-level action co-occurrence and temporal relations between the actions in the activity vide…
Action SegmentationUnsupervised Action SegmentationTargeted Forgetting of Image Subgroups in CLIP Models
Foundation models (FMs) such as CLIP have demonstrated impressive zero-shot performance across various tasks by leveraging large-scale, unsupervised pre-training. However, they often inherit harmful or unwanted knowl…
Knowledge DistillationUnsupervised Pre-trainingMeta-Imitation Learning by Watching Video Demonstrations
Meta-Imitation Learning is a promising technique for the robot to learn a new task from observing one or a few human demonstrations. However, it usually requires a significant number of demonstrations both from humans an…
Imitation LearningSample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videos
Current advanced policy learning methodologies have demonstrated the ability to develop expert-level strategies when provided enough information. However, their requirements, including task-specific rewards, expert-label…