Facial Expression Recognition and Image Description Generation in Vietnamese
This paper discusses a facial expression recognition model and a description generation model to build descriptive sentences for images and facial expressions of people in images. Our study shows that YOLOv5 achieves better results than a traditional CNN for all emotions on the KDEF dataset. In particular, the accuracies of the CNN and YOLOv5 models for emotion recognition are 0.853 and 0.938, respectively. A model for generating descriptions for images based on a merged architecture is proposed using VGG16 with the descriptions encoded over an LSTM model. YOLOv5 is also used to recognize dominant colors of objects in the images and correct the color words in the descriptions generated if it is necessary. If the description contains words referring to a person, we recognize the emotion of the person in the image. Finally, we combine the results of all models to create sentences that describe the visual content and the human emotions in the images. Experimental results on the Flickr8k dataset in Vietnamese achieve BLEU-1, BLEU-2, BLEU-3, BLEU-4 scores of 0.628; 0.425; 0.280; and 0.174, respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
DescriptiveEmotion RecognitionFacial Expression RecognitionFacial Expression Recognition (FER)Image DescriptionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Survey of the Trends in Facial and Expression Recognition Databases and Methods
Automated facial identification and facial expression recognition have been topics of active research over the past few decades. Facial and expression recognition find applications in human-computer interfaces, subject t…
Facial Expression RecognitionFacial Expression Recognition (FER)Facial Expression Recognition Using Human to Animated-Character Expression Translation
Facial expression recognition is a challenging task due to two major problems: the presence of inter-subject variations in facial expression recognition dataset and impure expressions posed by human subjects. In this pap…
Facial Expression RecognitionFacial Expression Recognition (FER)Generative Adversarial NetworkTranslationSynFER: Towards Boosting Facial Expression Recognition with Synthetic Data
Facial expression datasets remain limited in scale due to privacy concerns, the subjectivity of annotations, and the labor-intensive nature of data collection. This limitation poses a significant challenge for developing…
Facial Expression RecognitionPseudo LabelRepresentation LearningA Recursive Framework for Expression Recognition: From Web Images to Deep Models to Game Dataset
In this paper, we propose a recursive framework to recognize facial expressions from images in real scenes. Unlike traditional approaches that typically focus on developing and refining algorithms for improving recogniti…
Dataset GenerationFacial Expression RecognitionFacial Expression Recognition (FER)Prompting Visual-Language Models for Dynamic Facial Expression Recognition
This paper presents a novel visual-language model called DFER-CLIP, which is based on the CLIP model and designed for in-the-wild Dynamic Facial Expression Recognition (DFER). Specifically, the proposed DFER-CLIP consist…
Dynamic Facial Expression RecognitionFacial Expression RecognitionLanguage Modelling