Adjoint Differentiation for generic matrix functions
We derive a formula for the adjoint $\overline{A}$ of a square-matrix operation of the form $C=f(A)$, where $f$ is holomorphic in the neighborhood of each eigenvalue. We then apply the formula to derive closed-form expressions in particular cases of interest such as the case when we have a spectral decomposition $A=UDU^{-1}$, the spectrum cut-off $C=A_+$ and the Nearest Correlation Matrix routine. Finally, we explain how to simplify the computation of adjoints for regularized linear regression coefficients.
Code (0)
등록된 구현이 없습니다.
Tasks
FormregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Matrix Calculus (for Machine Learning and Beyond)
This course, intended for undergraduates familiar with elementary calculus and linear algebra, introduces the extension of differential calculus to functions on more general vector spaces, such as functions that take as …
Combinatory Adjoints and Differentiation
We develop a compositional approach for automatic and symbolic differentiation based on categorical constructions in functional analysis where derivatives are linear functions on abstract vectors rather than being limite…
Automatic Adjoint Differentiation for special functions involving expectations
We explain how to compute gradients of functions of the form $G = \frac{1}{2} \sum_{i=1}^{m} (E y_i - C_i)^2$, which often appear in the calibration of stochastic models, using Automatic Adjoint Differentiation and paral…
Matrix-Free Jacobian Chaining
The efficient computation of Jacobians represents a fundamental challenge in computational science and engineering. Large-scale modular numerical simulation programs can be regarded as sequences of evaluations of in our …
Scalable Gradients and Variational Inference for Stochastic Differential Equations
We derive reverse-mode (or adjoint) automatic differentiation for solutions of stochastic differential equations (SDEs), allowing time-efficient and constant-memory computation of pathwise gradients, a continuous-time an…
Time SeriesTime Series AnalysisVariational Inference