Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation
In this paper, at first, the impact of ImageNet pre-training on fine-grained Facial Emotion Recognition (FER) is investigated which shows that when enough augmentations on images are applied, training from scratch provides better result than fine-tuning on ImageNet pre-training. Next, we propose a method to improve fine-grained and in-the-wild FER, called Hybrid Multi-Task Learning (HMTL). HMTL uses Self-Supervised Learning (SSL) as an auxiliary task during classical Supervised Learning (SL) in the form of Multi-Task Learning (MTL). Leveraging SSL during training can gain additional information from images for the primary fine-grained SL task. We investigate how proposed HMTL can be used in the FER domain by designing two customized version of common pre-text task techniques, puzzling and in-painting. We achieve state-of-the-art results on the AffectNet benchmark via two types of HMTL, without utilizing pre-training on additional data. Experimental results on the common SSL pre-training and proposed HMTL demonstrate the difference and superiority of our work. However, HMTL is not only limited to FER domain. Experiments on two types of fine-grained facial tasks, i.e., head pose estimation and gender recognition, reveals the potential of using HMTL to improve fine-grained facial representation.
Code (0)
등록된 구현이 없습니다.
Tasks
Emotion RecognitionFacial Emotion RecognitionFacial Expression Recognition (FER)Head Pose EstimationMulti-Task LearningPose EstimationSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adaptive Transfer Learning on Graph Neural Networks
Graph neural networks (GNNs) is widely used to learn a powerful representation of graph-structured data. Recent work demonstrates that transferring knowledge from self-supervised tasks to downstream tasks could further i…
Meta-LearningMulti-Task LearningTransfer LearningAutomatic Self-supervised Learning for Social Recommendations
In recent years, researchers have attempted to exploit social relations to improve the performance in recommendation systems. Generally, most existing social recommendation methods heavily depends on substantial domain k…
Meta-LearningRecommendation SystemsRepresentation LearningSelf-Supervised LearningIntrinsically Motivated Self-supervised Learning in Reinforcement Learning
In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency. However, abu…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning
In recent years, graph neural networks (GNNs) have been widely adopted in the representation learning of graph-structured data and provided state-of-the-art performance in various applications such as link prediction, no…
Auxiliary LearningGraph Neural NetworkLink PredictionMeta-Learning+3Self-Supervised Generalisation with Meta Auxiliary Learning
Learning with auxiliary tasks can improve the ability of a primary task to generalise. However, this comes at the cost of manually labelling auxiliary data. We propose a new method which automatically learns appropriate …
Auxiliary LearningMeta-LearningMulti-Task Learning