Markov-Lipschitz Deep Learning
We propose a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation. A prior constraint, called locally isometric smoothness (LIS), is imposed across-layers and encoded into a Markov random field (MRF)-Gibbs distribution. This leads to the best possible solutions for local geometry preservation and robustness as measured by locally geometric distortion and locally bi-Lipschitz continuity. Consequently, the layer-wise vector transformations are enhanced into well-behaved, LIS-constrained metric homeomorphisms. Extensive experiments, comparisons, and ablation study demonstrate significant advantages of MLDL for manifold learning and manifold data generation. MLDL is general enough to enhance any vector transformation-based networks. The code is available at https://github.com/westlake-cairi/Markov-Lipschitz-Deep-Learning.
Code (2)
Tasks
Deep LearningDimensionality ReductionRepresentation LearningTopological Data AnalysisSimilar Papers 제목 키워드 기반
Performance Improvement Bounds for Lipschitz Configurable Markov Decision Processes
Configurable Markov Decision Processes (Conf-MDPs) have recently been introduced as an extension of the traditional Markov Decision Processes (MDPs) to model the real-world scenarios in which there is the possibility to …
Markov Chain Decoders Overcome the Heavy-Tail Limitations of Lipschitz Generative Models
Heavy-tailed distributions are prevalent in performance evaluation, network traffic, and risk modeling. This behavior poses a fundamental challenge for modern deep generative models. Standard Variational Autoencoders (VA…
Markov Chain Block Coordinate Descent
The method of block coordinate gradient descent (BCD) has been a powerful method for large-scale optimization. This paper considers the BCD method that successively updates a series of blocks selected according to a Mark…
Distributed OptimizationLipschitz Lifelong Reinforcement Learning
We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks. We introduce a novel metric between Markov Decision Processes (MDPs) and establish that close MDPs have…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Transfer LearningSolving Backward Stochastic Differential Equations with quadratic-growth drivers by Connecting the Short-term Expansions
This article proposes a new approximation scheme for quadratic-growth BSDEs in a Markovian setting by connecting a series of semi-analytic asymptotic expansions applied to short-time intervals. Although there remains a c…