OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case
This paper presents a novel, mathematically rigorous framework for autoencoder-type deep neural networks that combines optimal control theory and low-rank tensor methods to yield memory-efficient training and automated architecture discovery. The learning task is formulated as an optimization problem constrained by differential equations representing the encoder and decoder components of the network and the corresponding optimality conditions are derived via a Lagrangian approach. Efficient memory compression is enabled by approximating differential equation solutions on low-rank tensor manifolds using an adaptive explicit integration scheme. These concepts are combined to form OCTANE (Optimal Control for Tensor-based Autoencoder Network Emergence) -- a unified training framework that yields compact autoencoder architectures, reduces memory usage, and enables effective learning, even with limited training data. The framework's utility is illustrated with application to image denoising and deblurring tasks and recommendations regarding governing hyperparameters are provided.
Code (0)
등록된 구현이 없습니다.
Tasks
Image DenoisingSimilar Papers 제목 키워드 기반
Graph Machine Learning for Design of High-Octane Fuels
Fuels with high-knock resistance enable modern spark-ignition engines to achieve high efficiency and thus low CO2 emissions. Identification of molecules with desired autoignition properties indicated by a high research o…
Bayesian OptimizationBIG-bench Machine LearningVocal Bursts Intensity PredictionMode-Aware Non-Linear Tucker Autoencoder for Tensor-based Unsupervised Learning
High-dimensional data, particularly in the form of high-order tensors, presents a major challenge in self-supervised learning. While MLP-based autoencoders (AE) are commonly employed, their dependence on flattening opera…
Self-Supervised LearningControl theoretically explainable application of autoencoder methods to fault detection in nonlinear dynamic systems
This paper is dedicated to control theoretically explainable application of autoencoders to optimal fault detection in nonlinear dynamic systems. Autoencoder-based learning is a standard machine learning method and widel…
Anomaly DetectionFault DetectionRepresentation LearningNonlinear State Space Modeling and Control of the Impact of Patients' Modifiable Lifestyle Behaviors on the Emergence of Multiple Chronic Conditions
The emergence and progression of multiple chronic conditions (MCC) over time often form a dynamic network that depends on patient's modifiable risk factors and their interaction with non-modifiable risk factors and exist…
tvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder
Variational autoencoders (VAEs) are a powerful class of deep generative latent variable model for unsupervised representation learning on high-dimensional data. To ensure computational tractability, VAEs are often implem…
Gaussian ProcessesRepresentation LearningTime SeriesTime Series Analysis