Decoding Linguistic Representations of Human Brain
Language, as an information medium created by advanced organisms, has always been a concern of neuroscience regarding how it is represented in the brain. Decoding linguistic representations in the evoked brain has shown groundbreaking achievements, thanks to the rapid improvement of neuroimaging, medical technology, life sciences and artificial intelligence. In this work, we present a taxonomy of brain-to-language decoding of both textual and speech formats. This work integrates two types of research: neuroscience focusing on language understanding and deep learning-based brain decoding. Generating discernible language information from brain activity could not only help those with limited articulation, especially amyotrophic lateral sclerosis (ALS) patients but also open up a new way for the next generation's brain-computer interface (BCI). This article will help brain scientists and deep-learning researchers to gain a bird's eye view of fine-grained language perception, and thus facilitate their further investigation and research of neural process and language decoding.
Code (0)
등록된 구현이 없습니다.
Tasks
Brain Computer InterfaceBrain DecodingSimilar Papers 제목 키워드 기반
Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features
Decoding human visual neural representations is a challenging task with great scientific significance in revealing vision-processing mechanisms and developing brain-like intelligent machines. Most existing methods are di…
Does the brain represent words? An evaluation of brain decoding studies of language understanding
Language decoding studies have identified word representations which can be used to predict brain activity in response to novel words and sentences (Anderson et al., 2016; Pereira et al., 2018). The unspoken assumption o…
Brain DecodingSentenceFine-tuned vs. Prompt-tuned Supervised Representations: Which Better Account for Brain Language Representations?
To decipher the algorithm underlying the human brain's language representation, previous work probed brain responses to language input with pre-trained artificial neural network (ANN) models fine-tuned on NLU tasks. Howe…
ChunkingMulti-Task LearningDecoding Predictive Inference in Visual Language Processing via Spatiotemporal Neural Coherence
Human language processing relies on the brain's capacity for predictive inference. We present a machine learning framework for decoding neural (EEG) responses to dynamic visual language stimuli in Deaf signers. Using coh…
Linking artificial and human neural representations of language
What information from an act of sentence understanding is robustly represented in the human brain? We investigate this question by comparing sentence encoding models on a brain decoding task, where the sentence that an e…
Brain DecodingNatural Language UnderstandingSentence