A Novel mapping for visual to auditory sensory substitution
visual information can be converted into audio stream via sensory substitution devices in order to give visually impaired people the chance of perception of their surrounding easily and simultaneous to performing everyday tasks. In this study, visual environmental features namely, coordinate, type of objects and their size are assigned to audio features related to music tones such as frequency, time duration and note permutations. Results demonstrated that this new method has more training time efficiency in comparison with our previous method named VBTones which sinusoidal tones were applied. Moreover, results in blind object recognition for real objects was achieved 88.05 on average.
Code (0)
등록된 구현이 없습니다.
Tasks
Object RecognitionSimilar Papers 제목 키워드 기반
Autoencoding sensory substitution
Tens of millions of people live blind, and their number is ever increasing. Visual-to-auditory sensory substitution (SS) encompasses a family of cheap, generic solutions to assist the visually impaired by conveying visua…
Cross-Sensory Brain Passage Retrieval: Scaling Beyond Visual to Audio
Query formulation from internal information needs remains fundamentally challenging across all Information Retrieval paradigms due to cognitive complexity and physical impairments. Brain Passage Retrieval (BPR) addresses…
Information RetrievalPassage RetrievalAutomated mapping of virtual environments with visual predictive coding
Humans construct internal cognitive maps of their environment directly from sensory inputs without access to a system of explicit coordinates or distance measurements. While machine learning algorithms like SLAM utilize …
Quantifiers in a Multimodal World: Hallucinating Vision with Language and Sound
Inspired by the literature on multisensory integration, we develop a computational model to ground quantifiers in perception. The model learns to pick, out of nine quantifiers ({`}few{'}, {`}many{'}, {`}all{'}, etc.), th…
An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits
Audio-visual approaches involving visual inputs have laid the foundation for recent progress in speech separation. However, the optimization of the concurrent usage of auditory and visual inputs is still an active resear…
Speech Separation