Where are they looking?
Humans have the remarkable ability to follow the gaze of other people to identify what they are looking at. Following eye gaze, or gaze-following, is an important ability that allows us to understand what other people are thinking, the actions they are performing, and even predict what they might do next. Despite the importance of this topic, this problem has only been studied in limited scenarios within the computer vision community. In this paper, we propose a deep neural network-based approach for gaze-following and a new benchmark dataset for thorough evaluation. Given an image and the location of a head, our approach follows the gaze of the person and identifies the object being looked at. After training, the network is able to discover how to extract head pose and gaze orientation, and to select objects in the scene that are in the predicted line of sight and likely to be looked at (such as televisions, balls and food). The quantitative evaluation shows that our approach produces reliable results, even when viewing only the back of the head. While our method outperforms several baseline approaches, we are still far from reaching human performance at this task. Overall, we believe that this is a challenging and important task that deserves more attention from the community.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaze Target EstimationSimilar Papers 제목 키워드 기반
Telling the What while Pointing to the Where: Multimodal Queries for Image Retrieval
Most existing image retrieval systems use text queries as a way for the user to express what they are looking for. However, fine-grained image retrieval often requires the ability to also express where in the image the c…
Image RetrievalRetrievalWhere and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks
This paper addresses a new problem - jointly inferring human attention, intentions, and tasks from videos. Given an RGB-D video where a human performs a task, we answer three questions simultaneously: 1) where the human …
To know or not to know? How looking at payoffs signals selfish behavior
In daily life, subjects often face a social dilemma in two stages. In Stage 1, they recognize the social dilemma structure of the decision problem at hand (a tension between personal interest and collective interest); in…
Gaze-contingent decoding of human navigation intention on an autonomous wheelchair platform
We have pioneered the Where-You-Look-Is Where-You-Go approach to controlling mobility platforms by decoding how the user looks at the environment to understand where they want to navigate their mobility device. However, …
Motor ImageryNavigateObjectWordNet and beyond: the case of lexical access
For humans the main functions of a dictionary is to store information concerning words and to reveal it when needed. While readers are interested in the meaning of words, writers look for answers concerning usage, spelli…
LEMMA