Personalized Federated Learning for Egocentric Video Gaze Estimation with Comprehensive Parameter Frezzing
Egocentric video gaze estimation requires models to capture individual gaze patterns while adapting to diverse user data. Our approach leverages a transformer-based architecture, integrating it into a PFL framework where only the most significant parameters, those exhibiting the highest rate of change during training, are selected and frozen for personalization in client models. Through extensive experimentation on the EGTEA Gaze+ and Ego4D datasets, we demonstrate that FedCPF significantly outperforms previously reported federated learning methods, achieving superior recall, precision, and F1-score. These results confirm the effectiveness of our comprehensive parameters freezing strategy in enhancing model personalization, making FedCPF a promising approach for tasks requiring both adaptability and accuracy in federated learning settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningGaze EstimationPersonalized Federated LearningSimilar Papers 제목 키워드 기반
In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting
The emergence of advanced multimodal large language models (MLLMs) has significantly enhanced AI assistants' ability to process complex information across modalities. Recently, egocentric videos, by directly capturing us…
Video Question AnsweringGaze EstimationIn the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation
In this paper, we present the first transformer-based model to address the challenging problem of egocentric gaze estimation. We observe that the connection between the global scene context and local visual information i…
Gaze EstimationEgoHRV: Continuous Heart Rate Variability Estimation from Egocentric Systems for Autonomic Response and Skill Assessment
Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used nonin…
Gaze-Enabled Egocentric Video Summarization via Constrained Submodular Maximization
With the proliferation of wearable cameras, the number of videos of users documenting their personal lives using such devices is rapidly increasing. Since such videos may span hours, there is an important need for …
Combinatorial OptimizationCommon Sense ReasoningVideo SummarizationARGaze: Autoregressive Transformers for Online Egocentric Gaze Estimation
Online egocentric gaze estimation predicts where a camera wearer is looking from first-person video using only past and current frames, a task essential for augmented reality and assistive technologies. Unlike third-pers…
Gaze Estimation