Unlocked Backpropagation using Wave Scattering
Both the backpropagation algorithm in machine learning and the maximum principle in optimal control theory are posed as a two-point boundary problem, resulting in a "forward-backward" lock. We derive a reformulation of the maximum principle in optimal control theory as a hyperbolic initial value problem by introducing an additional "optimization time" dimension. We introduce counter-propagating wave variables with finite propagation speed and recast the optimization problem in terms of scattering relationships between them. This relaxation of the original problem can be interpreted as a physical system that equilibrates and changes its physical properties in order to minimize reflections. We discretize this continuum theory to derive a family of fully unlocked algorithms suitable for training neural networks. Different parameter dynamics, including gradient descent, can be derived by demanding dissipation and minimization of reflections at parameter ports. These results also imply that any physical substrate that supports the scattering and dissipation of waves can be interpreted as solving an optimization problem.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Recursive Born Approach to Nonlinear Inverse Scattering
The Iterative Born Approximation (IBA) is a well-known method for describing waves scattered by semi-transparent objects. In this paper, we present a novel nonlinear inverse scattering method that combines IBA with an ed…
Transparent objectsScattering Transformer: A Training-Free Transformer Architecture for Heart Murmur Detection
In an attempt to address the need for skilled clinicians in heart sound interpretation, recent research efforts on automating cardiac auscultation have explored deep learning approaches. The majority of these approaches …
Physics-Informed Machine Learning for the Inverse Design of Wave Scattering Clusters
Clusters of wave-scattering oscillators offer the ability to passively control wave energy in elastic continua. However, designing such clusters to achieve a desired wave energy pattern is a highly nontrivial task. While…
Physics-informed machine learningPerformance evaluation of wavelet scattering network in image texture classification in various color spaces
Texture plays an important role in many image analysis applications. In this paper, we give a performance evaluation of color texture classification by performing wavelet scattering network in various color spaces. Exper…
ClassificationGeneral ClassificationTexture ClassificationParametric Scattering Networks
The wavelet scattering transform creates geometric invariants and deformation stability. In multiple signal domains, it has been shown to yield more discriminative representations compared to other non-learned representa…
Image ClassificationSmall Data Image Classification