Decoding multimodal behavior using time differences of MEG events
Multimodal behavior involves multiple processing stations distributed across distant brain regions, but our understanding of how such distributed processing is coordinated in the brain is limited. Here we take a decoding approach to this problem, aiming to quantify how temporal aspects of brain-wide neural activity may be used to infer specific multimodal behaviors. Using high temporal resolution measurements by MEG, we detect bursts of activity from hundreds of locations across the surface of the brain at millisecond resolution. We then compare decoding using three characteristics of neural activity bursts, decoding with event counts, with latencies and with time differences between pairs of events. Training decoders in this regime is particularly challenging because the number of samples is smaller by orders of magnitude than the input dimensionality. We develop a new decoding approach for this regime that combines non-parametric modelling with aggressive feature selection. Surprisingly, we find that decoding using time-differences, based on thousands of region pairs, is significantly more accurate than using other activity characteristics, reaching 90% accuracy consistently across subjects. These results suggest that relevant information about multimodal brain function is provided by subtle time differences across remote brain areas.
Code (0)
등록된 구현이 없습니다.
Tasks
feature selectionSimilar Papers 제목 키워드 기반
Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
Real-time video commentary generation provides textual descriptions of ongoing events in videos. It supports accessibility and engagement in domains such as sports, esports, and livestreaming. Commentary generation invol…
MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
Vision-language models (VLMs) achieve strong performance on multimodal tasks but suffer from high inference latency due to large model sizes and long multimodal contexts. Speculative decoding has recently emerged as an e…
BCI-Based Assessment of Ocular Response Time Using Dynamic Time Warping Leveraging an RDWT-Driven Deep Neural Framework
Mild traumatic brain injury (mTBI) is a prevalent condition that remains difficult to diagnose in its early stages. Oculomotor dysfunction is a well-established marker of mTBI, motivating the development of portable tool…
Automatic localization and decoding of honeybee markers using deep convolutional neural networks
The honeybee is a fascinating model animal to investigate how collective behavior emerges from (inter-)actions of thousands of individuals. Bees may acquire unique memories throughout their lives. These experiences affec…
Multimodal Collaboration Networks for Geospatial Vehicle Detection in Dense, Occluded, and Large-Scale Events
In large-scale disaster events, the planning of optimal rescue routes depends on the object detection ability at the disaster scene, with one of the main challenges being the presence of dense and occluded objects. Exist…
4kobject-detectionObject Detectionvehicle detection