Backprop-Free Reinforcement Learning with Active Neural Generative Coding
In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input. This awareness largely depends on how the human agent interacts with the environment. In this work, we propose active neural generative coding, a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. Specifically, we develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. We demonstrate on several control problems, in the online learning setting, that our proposed modeling framework performs competitively with deep Q-learning models. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
Code (1)
Tasks
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems
In this article, we propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC), designing an agent built completely from powerful pred…
reinforcement-learningReinforcement Learning (RL)Applications of the Free Energy Principle to Machine Learning and Neuroscience
In this PhD thesis, we explore and apply methods inspired by the free energy principle to two important areas in machine learning and neuroscience. The free energy principle is a general mathematical theory of the necess…
Bayesian InferenceBIG-bench Machine LearningDeep Reinforcement LearningMeta-Representational Predictive Coding: Biomimetic Self-Supervised Learning
Self-supervised learning has become an increasingly important paradigm in the domain of machine intelligence. Furthermore, evidence for self-supervised adaptation, such as contrastive formulations, has emerged in recent …
FormSelf-Supervised LearningReinforce Security: A Model-Free Approach Towards Secure Wiretap Coding
The use of deep learning-based techniques for approximating secure encoding functions has attracted considerable interest in wireless communications due to impressive results obtained for general coding and decoding task…
Mutual Information Estimationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learning
Training neural networks with reinforcement learning (RL) typically relies on backpropagation (BP), necessitating storage of activations from the forward pass for subsequent backward updates. Furthermore, backpropagating…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)