FIFA: Fine-grained Inter-frame Attention for Driver's Video Gaze Estimation
Gaze direction serves as a pivotal indicator for assessing the level of driver attention. While image-based gaze estimation has been extensively researched, there has been a recent shift towards capturing gaze direction from video sequences. This approach encounters notable challenges, including the comprehension of the dynamic pupil evolution across frames and the extraction of head pose information from a relatively static background. To surmount these challenges, we introduce a dual-stream deep learning framework that explicitly models the displacement changes of the pupil through a fine-grained inter-frame attention mechanism and generates weights to adjust gaze embeddings. This technique transforms the face into a set of distinct patches and employs cross-attention to ascertain the correlation between pixel displacements in various patches and adjacent frames, thereby tracking spatial dynamics within the sequence. Our method is validated using two publicly available driver gaze datasets, and the results indicate that it achieves state-of-the-art performance or is on par with the best outcomes while reducing the parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaze EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fake-in-Facext: Towards Fine-Grained Explainable DeepFake Analysis
The advancement of Multimodal Large Language Models (MLLMs) has bridged the gap between vision and language tasks, enabling the implementation of Explainable DeepFake Analysis (XDFA). However, current methods suffer from…
Multi-Task LearningFIFA: Fast Inference Approximation for Action Segmentation
We introduce FIFA, a fast approximate inference method for action segmentation and alignment. Unlike previous approaches, FIFA does not rely on expensive dynamic programming for inference. Instead, it uses an approximate…
Action SegmentationSegmentationWeakly Supervised Action Segmentation (Transcript)From Players to Champions: A Generalizable Machine Learning Approach for Match Outcome Prediction with Insights from the FIFA World Cup
Accurate prediction of FIFA World Cup match outcomes holds significant value for analysts, coaches, bettors, and fans. This paper presents a machine learning framework specifically designed to forecast match winners in F…
Dimensionality ReductionHyperparameter OptimizationSports AnalyticsFIFA ranking: Evaluation and path forward
In this work we study the ranking algorithm used by F\'ed\'eration Internationale de Football Association (FIFA); we analyze the parameters it currently uses, show the formal probabilistic model from which it can be deri…
FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation
Video Multimodal Large Language Models (VideoMLLMs) have achieved remarkable progress in both Video-to-Text and Text-to-Video tasks. However, they often suffer fro hallucinations, generating content that contradicts the …
DescriptiveText GenerationVideo Generation