Bidirectional Interaction between Visual and Motor Generative Models using Predictive Coding and Active Inference
In this work, we build upon the Active Inference (AIF) and Predictive Coding (PC) frameworks to propose a neural architecture comprising a generative model for sensory prediction, and a distinct generative model for motor trajectories. We highlight how sequences of sensory predictions can act as rails guiding learning, control and online adaptation of motor trajectories. We furthermore inquire the effects of bidirectional interactions between the motor and the visual modules. The architecture is tested on the control of a simulated robotic arm learning to reproduce handwritten letters.
Code (1)
Similar Papers 제목 키워드 기반
Bidirectional Tutoring for Developmental Motor Learning in Robots: Co-Developed Interaction Dynamics Support Stable Learning
Infants are well known to develop their motor skills through dense interaction with caregivers. Although such social interaction is crucial for human development, motor-skill learning in robots is often treated as a unid…
Bidirectionality From Cargo Thermal Fluctuations in Motor-Mediated Transport
Molecular motor proteins serve as an essential component of intracellular transport by generating forces to haul cargoes along cytoskeletal filaments. Two species of motors that are directed oppositely (e.g. kinesin, dyn…
The Mirrornet : Learning Audio Synthesizer Controls Inspired by Sensorimotor Interaction
Experiments to understand the sensorimotor neural interactions in the human cortical speech system support the existence of a bidirectional flow of interactions between the auditory and motor regions. Their key function …
Autonomous VehiclesSpongeBob: Sync-Aware Harmonious Audio-Visual Generative Editing
Visual and acoustic events in the physical world are inherently coupled, yet existing video editing methods typically adopt decoupled pipelines, lacking bidirectional modality interaction. This results in two key limitat…
PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation
We introduce PerpetualWonder, a hybrid generative simulator that enables long-horizon, action-conditioned 4D scene generation from a single image. Current works fail at this task because their physical state is decoupled…
Scene Generation