Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learning

2019-03-25 · Huy N. Chau, Miklos Rasonyi

Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) is a momentum version of stochastic gradient descent with properly injected Gaussian noise to find a global minimum. In this paper, non-asymptotic convergence analysis of SGHMC is given in the context of non-convex optimization, where subsampling techniques are used over an i.i.d dataset for gradient updates. Our results complement those of [RRT17] and improve on those of [GGZ18].

📄 PDF Abstract BibTeX arXiv:1903.10328

Code (0)

등록된 구현이 없습니다.

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration

2018-09-12 · Xuefeng Gao, Mert Gürbüzbalaban, Lingjiong Zhu

Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stochastic gradient with momentum where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates towar…

Stochastic Optimization

Reflection coupling for unadjusted generalized Hamiltonian Monte Carlo in the nonconvex stochastic gradient case

2023-10-28 · Martin Chak, Pierre Monmarché

Contraction in Wasserstein 1-distance with explicit rates is established for generalized Hamiltonian Monte Carlo with stochastic gradients under possibly nonconvex conditions. The algorithms considered include splitting …

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

2018-03-29 · Zhize Li, Tianyi Zhang, Shuyu Cheng, Jun Zhu 외

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stocha…

Bayesian Inference

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks

2024-09-25 · Luxu Liang, Ariel Neufeld, Ying Zhang

In this paper, we provide a non-asymptotic analysis of the convergence of the stochastic gradient Hamiltonian Monte Carlo (SGHMC) algorithm to a target measure in Wasserstein-1 and Wasserstein-2 distance. Crucially, comp…

Stochastic Optimization

Relativistic Monte Carlo

2016-09-14 · Xiaoyu Lu, Valerio Perrone, Leonard Hasenclever, Yee Whye Teh 외

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC) algorithm that generates proposals for a Metropolis-Hastings algorithm by simulating the dynamics of a Hamiltonian system. However, HMC is sensit…