Deep Learning as the Disciplined Construction of Tame Objects
One can see deep-learning models as compositions of functions within the so-called tame geometry. In this expository note, we give an overview of some topics at the interface of tame geometry (also known as o-minimality), optimization theory, and deep learning theory and practice. To do so, we gradually introduce the concepts and tools used to build convergence guarantees for stochastic gradient descent in a general nonsmooth nonconvex, but tame, setting. This illustrates some ways in which tame geometry is a natural mathematical framework for the study of AI systems, especially within Deep Learning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DiscoverDCP: A Data-Driven Approach for Construction of Disciplined Convex Programs via Symbolic Regression
We propose DiscoverDCP, a data-driven framework that integrates symbolic regression with the rule sets of Disciplined Convex Programming (DCP) to perform system identification. By enforcing that all discovered candidate …
Uncertainty Quantification in HSI Reconstruction using Physics-Aware Diffusion Priors and Optics-Encoded Measurements
Hyperspectral image reconstruction from a compressed measurement is a highly ill-posed inverse problem. Current data-driven methods suffer from hallucination due to the lack of spectral diversity in existing hyperspectra…
Image ReconstructionBayesian InferenceDeterministic Envelopes for Tamed SGLD: Decoupling Stochastic Gradient Noise and Localizing Taming
Stochastic gradient Langevin algorithms often use tamed denominators to stabilize superlinear drifts. This paper shows that when the denominator depends on the current stochastic gradient, the transformed update can have…
Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation
Hyperspectral imaging empowers machine vision systems with the distinct capability of identifying materials through recording their spectral signatures. Recent efforts in data-driven spectral reconstruction aim at extrac…
Data AugmentationSpectral ReconstructionDMML-Net: Deep Metametric Learning for Few-Shot Geographic Object Segmentation in Remote Sensing Imagery
Geographic object segmentation is a fundamental yet challenging problem for remote sensing image interpretation. The prevalent paradigm to solve this problem is to train deep neural networks on massive labeled samples. A…
Metric LearningSegmentationSemantic Segmentation