Stochastic Learning Equation using Monotone Increasing Resolution of Quantization
In this paper, we propose a quantized learning equation with a monotone increasing resolution of quantization and stochastic analysis for the proposed algorithm. According to the white noise hypothesis for the quantization error with dense and uniform distribution, we can regard the quantization error as i.i.d.\ white noise. Based on this, we show that the learning equation with monotonically increasing quantization resolution converges weakly as the distribution viewpoint. The analysis of this paper shows that global optimization is possible for a domain that satisfies the Lipschitz condition instead of local convergence properties such as the Hessian constraint of the objective function.
Code (0)
등록된 구현이 없습니다.
Tasks
global-optimizationQuantizationSimilar Papers 제목 키워드 기반
Stochastic Gradient Langevin Dynamics Based on Quantization with Increasing Resolution
Stochastic learning dynamics based on Langevin or Levy stochastic differential equations (SDEs) in deep neural networks control the variance of noise by varying the size of the mini-batch or directly those of injecting n…
QuantizationContinuous-Time Portfolio Choice Under Monotone Mean-Variance Preferences-Stochastic Factor Case
We consider an incomplete market with a nontradable stochastic factor and a continuous time investment problem with an optimality criterion based on monotone mean-variance preferences. We formulate it as a stochastic dif…
Variance-Reduced Splitting Schemes for Monotone Stochastic Generalized Equations
We consider monotone inclusion problems where the operators may be expectation-valued, a class of problems that subsumes convex stochastic optimization problems as well as subclasses of stochastic variational inequality …
Stochastic OptimizationLast-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equations
Several widely-used first-order saddle-point optimization methods yield an identical continuous-time ordinary differential equation (ODE) that is identical to that of the Gradient Descent Ascent (GDA) method when derived…
Quantization Robustness of Monotone Operator Equilibrium Networks
Monotone operator equilibrium networks are implicit-layer models whose output is the unique equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence. When deployed on low-precision hardware…