Utilizing Eye Gaze to Enhance the Generalization of Imitation Networks to Unseen Environments
Vision-based autonomous driving through imitation learning mimics the behaviors of human drivers by training on pairs of data of raw driver-view images and actions. However, there are other cues, e.g. gaze behavior, available from human drivers that have yet to be exploited. Previous research has shown that novice human learners can benefit from observing experts' gaze patterns. We show here that deep neural networks can also benefit from this. We demonstrate different approaches to integrating gaze information into imitation networks. Our results show that the integration of gaze information improves the generalization performance of networks to unseen environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingImitation LearningSimilar Papers 제목 키워드 기반
Gaze Training by Modulated Dropout Improves Imitation Learning
Imitation learning by behavioral cloning is a prevalent method that has achieved some success in vision-based autonomous driving. The basic idea behind behavioral cloning is to have the neural network learn from observin…
Autonomous DrivingImitation LearningRotation-Constrained Cross-View Feature Fusion for Multi-View Appearance-based Gaze Estimation
Appearance-based gaze estimation has been actively studied in recent years. However, its generalization performance for unseen head poses is still a significant limitation for existing methods. This work proposes a gener…
Domain GeneralizationGaze EstimationGazeCLIP: Gaze-Guided CLIP with Adaptive-Enhanced Fine-Grained Language Prompt for Deepfake Attribution and Detection
Current deepfake attribution or deepfake detection works tend to exhibit poor generalization to novel generative methods due to the limited exploration in visual modalities alone. They tend to assess the attribution or d…
DeepFake DetectionEfficiently Guiding Imitation Learning Agents with Human Gaze
Human gaze is known to be an intention-revealing signal in human demonstrations of tasks. In this work, we use gaze cues from human demonstrators to enhance the performance of agents trained via three popular imitation l…
Atari GamesImitation LearningReinforcement LearningEnhancing Reusability of Learned Skills for Robot Manipulation via Gaze and Bottleneck
Autonomous agents capable of diverse object manipulations should be able to acquire a wide range of manipulation skills with high reusability. Although advances in deep learning have made it increasingly feasible to repl…
Imitation LearningObjectRobot Manipulation