Dynamic Neural Communication: Convergence of Computer Vision and Brain-Computer Interface
Interpreting human neural signals to decode static speech intentions such as text or images and dynamic speech intentions such as audio or video is showing great potential as an innovative communication tool. Human communication accompanies various features, such as articulatory movements, facial expressions, and internal speech, all of which are reflected in neural signals. However, most studies only generate short or fragmented outputs, while providing informative communication by leveraging various features from neural signals remains challenging. In this study, we introduce a dynamic neural communication method that leverages current computer vision and brain-computer interface technologies. Our approach captures the user's intentions from neural signals and decodes visemes in short time steps to produce dynamic visual outputs. The results demonstrate the potential to rapidly capture and reconstruct lip movements during natural speech attempts from human neural signals, enabling dynamic neural communication through the convergence of computer vision and brain--computer interface.
Code (0)
등록된 구현이 없습니다.
Tasks
Brain Computer InterfaceSimilar Papers 제목 키워드 기반
DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning
Gradient quantization is an emerging technique in reducing communication costs in distributed learning. Existing gradient quantization algorithms often rely on engineering heuristics or empirical observations, lacking a …
QuantizationFederated Dynamical Low-Rank Training with Global Loss Convergence Guarantees
In this work, we propose a federated dynamical low-rank training (FeDLRT) scheme to reduce client compute and communication costs - two significant performance bottlenecks in horizontal federated learning. Our method bui…
Federated LearningWhen Wireless Communications Meet Computer Vision in Beyond 5G
This article articulates the emerging paradigm, sitting at the confluence of computer vision and wireless communication, to enable beyond-5G/6G mission-critical applications (autonomous/remote-controlled vehicles, visuo-…
Image ReconstructionLanguage modulates vision: Evidence from neural networks and human brain-lesion models
Comparing information structures in between deep neural networks (DNNs) and the human brain has become a key method for exploring their similarities and differences. Recent research has shown better alignment of vision-l…
Imagined Speech and Visual Imagery as Intuitive Paradigms for Brain-Computer Interfaces
Brain-computer interfaces (BCIs) have shown promise in enabling communication for individuals with motor impairments. Recent advancements like brain-to-speech technology aim to reconstruct speech from neural activity. Ho…
Brain Computer InterfaceEEGFunctional Connectivity