EEG-based Emotional Video Classification via Learning Connectivity Structure
Electroencephalography (EEG) is a useful way to implicitly monitor the users perceptual state during multimedia consumption. One of the primary challenges for the practical use of EEG-based monitoring is to achieve a satisfactory level of accuracy in EEG classification. Connectivity between different brain regions is an important property for the classification of EEG. However, how to define the connectivity structure for a given task is still an open problem, because there is no ground truth about how the connectivity structure should be in order to maximize the classification performance. In this paper, we propose an end-to-end neural network model for EEG-based emotional video classification, which can extract an appropriate multi-layer graph structure and signal features directly from a set of raw EEG signals and perform classification using them. Experimental results demonstrate that our method yields improved performance in comparison to the existing approaches where manually defined connectivity structures and signal features are used. Furthermore, we show that the graph structure extraction process is reliable in terms of consistency, and the learned graph structures make much sense in the viewpoint of emotional perception occurring in the brain.
Code (1)
Tasks
ClassificationEEGElectroencephalogram (EEG)General ClassificationVideo ClassificationSimilar Papers 제목 키워드 기반
Emotional EEG Classification using Connectivity Features and Convolutional Neural Networks
Convolutional neural networks (CNNs) are widely used to recognize the user's state through electroencephalography (EEG) signals. In the previous studies, the EEG signals are usually fed into the CNNs in the form of high-…
ClassificationEEGElectroencephalogram (EEG)General Classification+1Emotional EEG Classification using Upscaled Connectivity Matrices
In recent studies of emotional EEG classification, connectivity matrices have been successfully employed as input to convolutional neural networks (CNNs), which can effectively consider inter-regional interaction pattern…
ClassificationEEGA Multi-task Neural Approach for Emotion Attribution, Classification and Summarization
Emotional content is a crucial ingredient in user-generated videos. However, the sparsity of emotional expressions in the videos poses an obstacle to visual emotion analysis. In this paper, we propose a new neural approa…
ClassificationEmotion RecognitionGeneral ClassificationSPECTRUM: Semantic Processing and Emotion-informed video-Captioning Through Retrieval and Understanding Modalities
Capturing a video's meaning and critical concepts by analyzing the subtle details is a fundamental yet challenging task in video captioning. Identifying the dominant emotional tone in a video significantly enhances the p…
AttributeDescriptiveRetrievalText Retrieval+2SS-VAERR: Self-Supervised Apparent Emotional Reaction Recognition from Video
This work focuses on the apparent emotional reaction recognition (AERR) from the video-only input, conducted in a self-supervised fashion. The network is first pre-trained on different self-supervised pretext tasks and l…
Self-Supervised Learning