Cross-modal codification of images with auditory stimuli: a language for the visually impaired
In this study we describe a methodology to realize visual images cognition in the broader sense, by a cross-modal stimulation through the auditory channel. An original algorithm of conversion from bi-dimensional images to sounds has been established and tested on several subjects. Our results show that subjects where able to discriminate with a precision of 95\% different sounds corresponding to different test geometric shapes. Moreover, after brief learning sessions on simple images, subjects where able to recognize among a group of 16 complex and never-trained images a single target by hearing its acoustical counterpart. Rate of recognition was found to depend on image characteristics, in 90% of the cases, subjects did better than choosing at random. This study contribute to the understanding of cross-modal perception and help for the realization of systems that use acoustical signals to help visually impaired persons to recognize objects and improve navigation
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
EEG2Mel: Reconstructing Sound from Brain Responses to Music
Information retrieval from brain responses to auditory and visual stimuli has shown success through classification of song names and image classes presented to participants while recording EEG signals. Information retrie…
EEGElectroencephalogram (EEG)Information RetrievalRetrievalIndependent Feature Enhanced Crossmodal Fusion for Match-Mismatch Classification of Speech Stimulus and EEG Response
It is crucial for auditory attention decoding to classify matched and mismatched speech stimuli with corresponding EEG responses by exploring their relationship. However, existing methods often adopt two independent netw…
EEGEeg DecodingCoherent Language Reconstruction from Brain Recordings with Flexible Multi-Modal Input Stimuli
Decoding thoughts from brain activity offers valuable insights into human cognition and enables promising applications in brain-computer interaction. While prior studies have explored language reconstruction from fMRI da…
validCross-Sensory Brain Passage Retrieval: Scaling Beyond Visual to Audio
Query formulation from internal information needs remains fundamentally challenging across all Information Retrieval paradigms due to cognitive complexity and physical impairments. Brain Passage Retrieval (BPR) addresses…
Information RetrievalPassage Retrievalnaplib-python: Neural Acoustic Data Processing and Analysis Tools in Python
Recently, the computational neuroscience community has pushed for more transparent and reproducible methods across the field. In the interest of unifying the domain of auditory neuroscience, naplib-python provides an int…