paper-with-me

Papers

Efficient Optimal Transport Algorithm by Accelerated Gradient descent

2021-04-12 · Dongsheng An, Na lei, Xianfeng GU

Optimal transport (OT) plays an essential role in various areas like machine learning and deep learning. However, computing discrete optimal transport plan for large scale problems with adequate accuracy and efficiency is still highly challenging. Recently, methods based on the Sinkhorn algorithm add an entropy regularizer to the prime problem and get a trade off between efficiency and accuracy. In this paper, we propose a novel algorithm to further improve the efficiency and accuracy based on Nesterov's smoothing technique. Basically, the non-smooth c-transform of the Kantorovich potential is approximated by the smooth Log-Sum-Exp function, which finally smooths the original non-smooth Kantorovich dual functional (energy). The smooth Kantorovich functional can be optimized by the fast proximal gradient algorithm (FISTA) efficiently. Theoretically, the computational complexity of the proposed method is given by $O(n^{\frac{5}{2}} \sqrt{\log n} /\epsilon)$, which is lower than that of the Sinkhorn algorithm. Empirically, compared with the Sinkhorn algorithm, our experimental results demonstrate that the proposed method achieves faster convergence and better accuracy with the same parameter.

📄 PDF Abstract BibTeX arXiv:2104.05802

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Fast Algorithms for Computational Optimal Transport and Wasserstein Barycenter

2019-05-23 · Wenshuo Guo, Nhat Ho, Michael. I. Jordan

We provide theoretical complexity analysis for new algorithms to compute the optimal transport (OT) distance between two discrete probability distributions, and demonstrate their favorable practical performance over stat…

Traditional and accelerated gradient descent for neural architecture search

2020-06-26 · Nicolas Garcia Trillos, Felix Morales, Javier Morales

In this paper we introduce two algorithms for neural architecture search (NASGD and NASAGD) following the theoretical work by two of the authors [5] which used the geometric structure of optimal transport to introduce th…

GPUNeural Architecture Search

An Accelerated Stochastic Algorithm for Solving the Optimal Transport Problem

2022-03-02 · Yiling Xie, Yiling Luo, Xiaoming Huo

A primal-dual accelerated stochastic gradient descent with variance reduction algorithm (PDASGD) is proposed to solve linear-constrained optimization problems. PDASGD could be applied to solve the discrete optimal transp…

A geometric alternative to Nesterov's accelerated gradient descent

2015-06-26 · Sébastien Bubeck, Yin Tat Lee, Mohit Singh

We propose a new method for unconstrained optimization of a smooth and strongly convex function, which attains the optimal rate of convergence of Nesterov's accelerated gradient descent. The new algorithm has a simple ge…

Conditional Accelerated Lazy Stochastic Gradient Descent

2017-03-16 · ICML 2017 8 · Guanghui Lan, Sebastian Pokutta, Yi Zhou, Daniel Zink

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ …