Reverse Faà di Bruno's Formula for Cartesian Reverse Differential Categories
Reverse differentiation is an essential operation for automatic differentiation. Cartesian reverse differential categories axiomatize reverse differentiation in a categorical framework, where one of the primary axioms is the reverse chain rule, which is the formula that expresses the reverse derivative of a composition. Here, we present the reverse differential analogue of Faa di Bruno's Formula, which gives a higher-order reverse chain rule in a Cartesian reverse differential category. To properly do so, we also define partial reverse derivatives and higher-order reverse derivatives in a Cartesian reverse differential category.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On the Mathematics of Diffusion Models
This paper gives direct derivations of the differential equations and likelihood formulas of diffusion models assuming only knowledge of Gaussian distributions. A VAE analysis derives both forward and backward stochastic…
A Differential-form Pullback Programming Language for Higher-order Reverse-mode Automatic Differentiation
Building on the observation that reverse-mode automatic differentiation (AD) -- a generalisation of backpropagation -- can naturally be expressed as pullbacks of differential 1-forms, we design a simple higher-order prog…
FormFunctorial String Diagrams for Reverse-Mode Automatic Differentiation
We enhance the calculus of string diagrams for monoidal categories with hierarchical features in order to capture closed monoidal (and cartesian closed) structure. Using this new syntax we formulate an automatic differen…
Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits
We introduce Reverse Derivative Ascent: a categorical analogue of gradient based methods for machine learning. Our algorithm is defined at the level of so-called reverse differential categories. It can be used to learn t…
BIG-bench Machine LearningConvergence of the denoising diffusion probabilistic models for general noise schedules
This work presents a theoretical analysis of the original formulation of denoising diffusion probabilistic models (DDPMs), introduced by Ho, Jain, and Abbeel in Advances in Neural Information Processing Systems, 33 (2020…
DenoisingNoise Estimation