A Comparison of First-order Algorithms for Machine Learning
Using an optimization algorithm to solve a machine learning problem is one of mainstreams in the field of science. In this work, we demonstrate a comprehensive comparison of some state-of-the-art first-order optimization algorithms for convex optimization problems in machine learning. We concentrate on several smooth and non-smooth machine learning problems with a loss function plus a regularizer. The overall experimental results show the superiority of primal-dual algorithms in solving a machine learning problem from the perspectives of the ease to construct, running time and accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Comparing Machine Learning Algorithms by Union-Free Generic Depth
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies in linear and metric spaces, there is very little discussion on depth functions…
BenchmarkingDepth Functions for Partial Orders with a Descriptive Analysis of Machine Learning Algorithms
We propose a framework for descriptively analyzing sets of partial orders based on the concept of depth functions. Despite intensive studies of depth functions in linear and metric spaces, there is very little discussion…
BenchmarkingDescriptiveA Gaussian Comparison Theorem for Training Dynamics in Machine Learning
We study training algorithms with data following a Gaussian mixture model. For a specific family of such algorithms, we present a non-asymptotic result, connecting the evolution of the model to a surrogate dynamical syst…
Machine Learning Lie Structures & Applications to Physics
Classical and exceptional Lie algebras and their representations are among the most important tools in the analysis of symmetry in physical systems. In this letter we show how the computation of tensor products and branc…
BIG-bench Machine LearningAdaptive First- and Second-Order Algorithms for Large-Scale Machine Learning
In this paper, we consider both first- and second-order techniques to address continuous optimization problems arising in machine learning. In the first-order case, we propose a framework of transition from deterministic…
BIG-bench Machine LearningDeep LearningStochastic Optimization