Human Attention Detection Using AM-FM Representations
Human activity detection from digital videos presents many challenges to the computer vision and image processing communities. Recently, many methods have been developed to detect human activities with varying degree of success. Yet, the general human activity detection problem remains very challenging, especially when the methods need to work 'in the wild' (e.g., without having precise control over the imaging geometry). The thesis explores phase-based solutions for (i) detecting faces, (ii) back of the heads, (iii) joint detection of faces and back of the heads, and (iv) whether the head is looking to the left or the right, using standard video cameras without any control on the imaging geometry. The proposed phase-based approach is based on the development of simple and robust methods that rely on the use of Amplitude Modulation- Frequency Modulation (AM-FM) models. The approach is validated using video frames extracted from the Advancing Out-of-school Learning in Mathematics and Engineering (AOLME) project. The dataset consisted of 13,265 images from ten students looking at the camera, and 6,122 images from five students looking away from the camera. For the students facing the camera, the method was able to correctly classify 97.1% of them looking to the left and 95.9% of them looking to the right. For the students facing the back of the camera, the method was able to correctly classify 87.6% of them looking to the left and 93.3% of them looking to the right. The results indicate that AM-FM based methods hold great promise for analyzing human activity videos.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionActivity DetectionSimilar Papers 제목 키워드 기반
AEyeDE: An Attention-Based Attribution Framework for AI-Generated Text Detection
Detecting AI-generated text is becoming increasingly challenging as modern language models approach human-level fluency and can evade detectors that rely on surface statistics or likelihood-based signals. We propose \tex…
Text DetectionDisentangled Interaction Representation for One-Stage Human-Object Interaction Detection
Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction; h…
DecoderHuman-Object Interaction DetectionPose EstimationUltrasound Image Representation Learning by Modeling Sonographer Visual Attention
Image representations are commonly learned from class labels, which are a simplistic approximation of human image understanding. In this paper we demonstrate that transferable representations of images can be learned wit…
regressionRepresentation LearningSaliency PredictionTransfer LearningTowards Zero-shot Human-Object Interaction Detection via Vision-Language Integration
Human-object interaction (HOI) detection aims to locate human-object pairs and identify their interaction categories in images. Most existing methods primarily focus on supervised learning, which relies on extensive manu…
DecoderHuman-Object Interaction DetectionLanguage ModelingLanguage Modelling+2Visual Relationship Detection with Visual-Linguistic Knowledge from Multimodal Representations
Visual relationship detection aims to reason over relationships among salient objects in images, which has drawn increasing attention over the past few years. Inspired by human reasoning mechanisms, it is believed that e…
Objectobject-detectionObject DetectionRelational Reasoning+2