Differential Equations for Continuous-Time Deep Learning
This short, self-contained article seeks to introduce and survey continuous-time deep learning approaches that are based on neural ordinary differential equations (neural ODEs). It primarily targets readers familiar with ordinary and partial differential equations and their analysis who are curious to see their role in machine learning. Using three examples from machine learning and applied mathematics, we will see how neural ODEs can provide new insights into deep learning and a foundation for more efficient algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningSurveySimilar Papers 제목 키워드 기반
Continuous Depth Recurrent Neural Differential Equations
Recurrent neural networks (RNNs) have brought a lot of advancements in sequence labeling tasks and sequence data. However, their effectiveness is limited when the observations in the sequence are irregularly sampled, whe…
Comprehensive Review of Neural Differential Equations for Time Series Analysis
Time series modeling and analysis have become critical in various domains. Conventional methods such as RNNs and Transformers, while effective for discrete-time and regularly sampled data, face significant challenges in …
Time SeriesTime Series AnalysisSet-Valued Risk Measures as Backward Stochastic Difference Inclusions and Equations
Scalar dynamic risk measures for univariate positions in continuous time are commonly represented as backward stochastic differential equations. In the multivariate setting, dynamic risk measures have been defined and st…
Bayesian Neural Controlled Differential Equations for Treatment Effect Estimation
Treatment effect estimation in continuous time is crucial for personalized medicine. However, existing methods for this task are limited to point estimates of the potential outcomes, whereas uncertainty estimates have be…
Bayesian InferenceDecision MakingUncertainty QuantificationModeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
In this paper, we propose a continuous-time formulation for the AdaGrad, RMSProp, and Adam optimization algorithms by modeling them as first-order integro-differential equations. We perform numerical simulations of these…