paper-with-me

Papers

Closing the loop on multisensory interactions: A neural architecture for multisensory causal inference and recalibration

2018-02-19 · Jonathan Tong, German I. Parisi, Stefan Wermter, Brigitte Röder

When the brain receives input from multiple sensory systems, it is faced with the question of whether it is appropriate to process the inputs in combination, as if they originated from the same event, or separately, as if they originated from distinct events. Furthermore, it must also have a mechanism through which it can keep sensory inputs calibrated to maintain the accuracy of its internal representations. We have developed a neural network architecture capable of i) approximating optimal multisensory spatial integration, based on Bayesian causal inference, and ii) recalibrating the spatial encoding of sensory systems. The architecture is based on features of the dorsal processing hierarchy, including the spatial tuning properties of unisensory neurons and the convergence of different sensory inputs onto multisensory neurons. Furthermore, we propose that these unisensory and multisensory neurons play dual roles in i) encoding spatial location as separate or integrated estimates and ii) accumulating evidence for the independence or relatedness of multisensory stimuli. We further propose that top-down feedback connections spanning the dorsal pathway play key a role in recalibrating spatial encoding at the level of early unisensory cortices. Our proposed architecture provides possible explanations for a number of human electrophysiological and neuroimaging results and generates testable predictions linking neurophysiology with behaviour.

📄 PDF Abstract BibTeX arXiv:1802.06591

Code (0)

등록된 구현이 없습니다.

Tasks

Causal Inference

Similar Papers 제목 키워드 기반

Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning

2025-11-18 · Rickmer Krohn, Vignesh Prasad, Gabriele Tiboni, Georgia Chalvatzaki arxiv

Effective contact-rich manipulation requires robots to synergistically leverage vision, force, and proprioception. However, Reinforcement Learning agents struggle to learn in such multisensory settings, especially amidst…

Reinforcement LearningRobot Manipulation

Foundations of Multisensory Artificial Intelligence

2024-04-29 · Paul Pu Liang

Building multisensory AI systems that learn from multiple sensory inputs such as text, speech, video, real-world sensors, wearable devices, and medical data holds great promise for impact in many scientific areas with pr…

Prognosis

A Vision for Multisensory Intelligence: Sensing, Science, and Synergy

2026-01-08 · Paul Pu Liang arxiv

Our experience of the world is multisensory, spanning a synthesis of language, sight, sound, touch, taste, and smell. Yet, artificial intelligence has primarily advanced in digital modalities like text, vision, and audio…

The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects

2023-06-01 · CVPR 2023 1 · Ruohan Gao, Yiming Dou, Hao Li, Tanmay Agarwal 외

We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for multisensory object-centric learning, centered around object recognition, reconstruction, and manipulation with sight, sound, and touch. We also …

BenchmarkingObjectObject Recognition

MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World

2024-01-16 · CVPR 2024 1 · Yining Hong, Zishuo Zheng, Peihao Chen, Yian Wang 외

Human beings possess the capability to multiply a melange of multisensory cues while actively exploring and interacting with the 3D world. Current multi-modal large language models, however, passively absorb sensory data…

Language ModelingLanguage ModellingLarge Language Model