BrainDecoder: Style-Based Visual Decoding of EEG Signals
Decoding neural representations of visual stimuli from electroencephalography (EEG) offers valuable insights into brain activity and cognition. Recent advancements in deep learning have significantly enhanced the field of visual decoding of EEG, primarily focusing on reconstructing the semantic content of visual stimuli. In this paper, we present a novel visual decoding pipeline that, in addition to recovering the content, emphasizes the reconstruction of the style, such as color and texture, of images viewed by the subject. Unlike previous methods, this ``style-based'' approach learns in the CLIP spaces of image and text separately, facilitating a more nuanced extraction of information from EEG signals. We also use captions for text alignment simpler than previously employed, which we find work better. Both quantitative and qualitative evaluations show that our method better preserves the style of visual stimuli and extracts more fine-grained semantic information from neural signals. Notably, it achieves significant improvements in quantitative results and sets a new state-of-the-art on the popular Brain2Image dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
EEGMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Joint fMRI Decoding and Encoding with Latent Embedding Alignment
The connection between brain activity and corresponding visual stimuli is crucial in comprehending the human brain. While deep generative models have exhibited advancement in recovering brain recordings by generating ima…
Image GenerationTowards Dynamic Neural Communication and Speech Neuroprosthesis Based on Viseme Decoding
Decoding text, speech, or images from human neural signals holds promising potential both as neuroprosthesis for patients and as innovative communication tools for general users. Although neural signals contain various i…
Face ReconstructionNeuro-3D: Towards 3D Visual Decoding from EEG Signals
Human's perception of the visual world is shaped by the stereo processing of 3D information. Understanding how the brain perceives and processes 3D visual stimuli in the real world has been a longstanding endeavor in neu…
EEGSTEER: Unified Style Transfer with Expert Reinforcement
While text style transfer has many applications across natural language processing, the core premise of transferring from a single source style is unrealistic in a real-world setting. In this work, we focus on arbitrary …
Style TransferText Style TransferExploring The Visual Feature Space for Multimodal Neural Decoding
The intrication of brain signals drives research that leverages multimodal AI to align brain modalities with visual and textual data for explainable descriptions. However, most existing studies are limited to coarse inte…
Brain DecodingQuestion Answering