Combinatory Adjoints and Differentiation
We develop a compositional approach for automatic and symbolic differentiation based on categorical constructions in functional analysis where derivatives are linear functions on abstract vectors rather than being limited to scalars, vectors, matrices or tensors represented as multi-dimensional arrays. We show that both symbolic and automatic differentiation can be performed using a differential calculus for generating linear functions representing Fr\'echet derivatives based on rules for primitive, constant, linear and bilinear functions as well as their sequential and parallel composition. Linear functions are represented in a combinatory domain-specific language. Finally, we provide a calculus for symbolically computing the adjoint of a derivative without using matrices, which are too inefficient to use on high-dimensional spaces. The resulting symbolic representation of a derivative retains the data-parallel operations from the input program. The combination of combinatory differentiation and computing formal adjoints turns out to be behaviorally equivalent to reverse-mode automatic differentiation. In particular, it provides opportunities for optimizations where matrices are too inefficient to represent linear functions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Adjoint Differentiation for generic matrix functions
We derive a formula for the adjoint $\overline{A}$ of a square-matrix operation of the form $C=f(A)$, where $f$ is holomorphic in the neighborhood of each eigenvalue. We then apply the formula to derive closed-form expre…
FormregressionMatrix-Free Jacobian Chaining
The efficient computation of Jacobians represents a fundamental challenge in computational science and engineering. Large-scale modular numerical simulation programs can be regarded as sequences of evaluations of in our …
G\'en\'erer une grammaire d'arbres adjoints pour l'arabe \`a partir d'une m\'eta-grammaire (Generate a tree adjoining grammar for arabic from a meta-grammar)
La raret{\'e} des ressources num{\'e}riques pour la langue arabe, telles que les grammaires et corpus, rend son traitement plus difficile que les autres langues naturelles. A ce jour il n{'}existe pas une grammaire forme…
Finite Element-Based Material Learning via Automatic Differentiation: Learning constitutive neural network models from full-field deformation data
The identification of constitutive neural network models from heterogeneous full-field deformation data provides a robust alternative to traditional calibration methods based on homogeneous stress-strain experiments, par…
Computational EfficiencyAn HDP Model for Inducing Combinatory Categorial Grammars
We introduce a novel nonparametric Bayesian model for the induction of Combinatory Categorial Grammars from POS-tagged text. It achieves state of the art performance on a number of languages, and induces linguistically p…
modelPOS