How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation

2014-07-29 · Yoshua Bengio

We propose to exploit {\em reconstruction} as a layer-local training signal for deep learning. Reconstructions can be propagated in a form of target propagation playing a role similar to back-propagation but helping to reduce the reliance on derivatives in order to perform credit assignment across many levels of possibly strong non-linearities (which is difficult for back-propagation). A regularized auto-encoder tends produce a reconstruction that is a more likely version of its input, i.e., a small move in the direction of higher likelihood. By generalizing gradients, target propagation may also allow to train deep networks with discrete hidden units. If the auto-encoder takes both a representation of input and target (or of any side information) in input, then its reconstruction of input representation provides a target towards a representation that is more likely, conditioned on all the side information. A deep auto-encoder decoding path generalizes gradient propagation in a learned way that can could thus handle not just infinitesimal changes but larger, discrete changes, hopefully allowing credit assignment through a long chain of non-linear operations. In addition to each layer being a good auto-encoder, the encoder also learns to please the upper layers by transforming the data into a space where it is easier to model by them, flattening manifolds and disentangling factors. The motivations and theoretical justifications for this approach are laid down in this paper, along with conjectures that will have to be verified either mathematically or experimentally, including a hypothesis stating that such auto-encoder mediated target propagation could play in brains the role of credit assignment through many non-linear, noisy and discrete transformations.

📄 PDF Abstract BibTeX arXiv:1407.7906

Code (0)

등록된 구현이 없습니다.

Difference Target Propagation

2014-12-23 · Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, Yoshua Bengio

Back-propagation has been the workhorse of recent successes of deep learning but it relies on infinitesimal effects (partial derivatives) in order to perform credit assignment. This could become a serious issue as one co…

Implementing engrams from a machine learning perspective: the relevance of a latent space

2024-07-23 · J Marco de Lucas

In our previous work, we proposed that engrams in the brain could be biologically implemented as autoencoders over recurrent neural networks. These autoencoders would comprise basic excitatory/inhibitory motifs, with cre…

Nucleolus Credit Assignment for Effective Coalitions in Multi-agent Reinforcement Learning

2025-03-01 · Yugu Li, Zehong Cao, Jianglin Qiao, Siyi Hu

In cooperative multi-agent reinforcement learning (MARL), agents typically form a single grand coalition based on credit assignment to tackle a composite task, often resulting in suboptimal performance. This paper propos…

Multi-agent Reinforcement Learning Q-Learning Starcraft

Variance Reduced Advantage Estimation with $δ$ Hindsight Credit Assignment

2019-11-19 · Kenny Young

Hindsight Credit Assignment (HCA) refers to a recently proposed family of methods for producing more efficient credit assignment in reinforcement learning. These methods work by explicitly estimating the probability that…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Credit Assignment via Neural Manifold Noise Correlation

2026-01-06 · Byungwoo Kang, Maceo Richards, Bernardo Sabatini arxiv

Credit assignment--how changes in individual neurons and synapses affect a network's output--is central to learning in brains and machines. Noise correlation, which estimates gradients by correlating perturbations of act…