Derivation of the Backpropagation Algorithm Based on Derivative Amplification Coefficients
The backpropagation algorithm for neural networks is widely felt hard to understand, despite the existence of some well-written explanations and/or derivations. This paper provides a new derivation of this algorithm based on the concept of derivative amplification coefficients. First proposed by this author for fully connected cascade networks, this concept is found to well carry over to conventional feedforward neural networks and it paves the way for the use of mathematical induction in establishing a key result that enables backpropagation for derivative amplification coefficients. Then we establish the connection between derivative amplification coefficients and error coefficients (commonly referred to as errors in the literature), and show that the same backpropagation procedure can be used for error coefficients. The entire derivation is thus rigorous, simple, and elegant.
Code (0)
등록된 구현이 없습니다.
Tasks
Mathematical InductionSimilar Papers 제목 키워드 기반
Equivalence of Equilibrium Propagation and Recurrent Backpropagation
Recurrent Backpropagation and Equilibrium Propagation are supervised learning algorithms for fixed point recurrent neural networks which differ in their second phase. In the first phase, both algorithms converge to a fix…
Privacy Amplification of Iterative Algorithms via Contraction Coefficients
We investigate the framework of privacy amplification by iteration, recently proposed by Feldman et al., from an information-theoretic lens. We demonstrate that differential privacy guarantees of iterative mappings can b…
Backpropagation generalized for output derivatives
Backpropagation algorithm is the cornerstone for neural network analysis. Paper extends it for training any derivatives of neural network's output with respect to its input. By the dint of it feedforward networks can be …
On reconstructing high derivatives of noisy time-series with confidence intervals
Reconstructing high derivatives of noisy measurements is an important step in many control, identification and diagnosis problems. In this paper, a heuristic is proposed to address this challenging issue. The framework i…
Time SeriesFunction changes in the Backpropagation equation is equivalent to an implicit learning rate
The backpropagation algorithm is the de-facto standard for credit assignment in artificial neural networks due to its empirical results. Since its conception, variants of the backpropagation algorithm have emerged. More …