Query Augmentation by Decoding Semantics from Brain Signals
Query augmentation is a crucial technique for refining semantically imprecise queries. Traditionally, query augmentation relies on extracting information from initially retrieved, potentially relevant documents. If the quality of the initially retrieved documents is low, then the effectiveness of query augmentation would be limited as well. We propose Brain-Aug, which enhances a query by incorporating semantic information decoded from brain signals. BrainAug generates the continuation of the original query with a prompt constructed with brain signal information and a ranking-oriented inference approach. Experimental results on fMRI (functional magnetic resonance imaging) datasets show that Brain-Aug produces semantically more accurate queries, leading to improved document ranking performance. Such improvement brought by brain signals is particularly notable for ambiguous queries.
Code (1)
Tasks
Document RankingSimilar Papers 제목 키워드 기반
Self-Aug: Query and Entropy Adaptive Decoding for Large Vision-Language Models
Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal capabilities, but they inherit the tendency to hallucinate from their underlying language models. While visual contrastive decoding has been pr…
An amplitudes-perturbation data augmentation method in convolutional neural networks for EEG decoding
Brain-Computer Interface (BCI) system provides a pathway between humans and the outside world by analyzing brain signals which contain potential neural information. Electroencephalography (EEG) is one of most commonly us…
Brain Computer InterfaceData AugmentationDeep LearningEEG+2Joint fMRI Decoding and Encoding with Latent Embedding Alignment
The connection between brain activity and corresponding visual stimuli is crucial in comprehending the human brain. While deep generative models have exhibited advancement in recovering brain recordings by generating ima…
Image GenerationOne Brain, Omni Modalities: Towards Unified Non-Invasive Brain Decoding with Large Language Models
Deciphering brain function through non-invasive recordings requires synthesizing complementary high-frequency electromagnetic (EEG/MEG) and low-frequency metabolic (fMRI) signals. However, despite their shared neural ori…
Brain DecodingTowards Neural Co-Processors for the Brain: Combining Decoding and Encoding in Brain-Computer Interfaces
The field of brain-computer interfaces is poised to advance from the traditional goal of controlling prosthetic devices using brain signals to combining neural decoding and encoding within a single neuroprosthetic device…