Anderson Acceleration For Bioinformatics-Based Machine Learning

2023-02-01 · Sarwan Ali, Prakash Chourasia, Murray Patterson

Anderson acceleration (AA) is a well-known method for accelerating the convergence of iterative algorithms, with applications in various fields including deep learning and optimization. Despite its popularity in these areas, the effectiveness of AA in classical machine learning classifiers has not been thoroughly studied. Tabular data, in particular, presents a unique challenge for deep learning models, and classical machine learning models are known to perform better in these scenarios. However, the convergence analysis of these models has received limited attention. To address this gap in research, we implement a support vector machine (SVM) classifier variant that incorporates AA to speed up convergence. We evaluate the performance of our SVM with and without Anderson acceleration on several datasets from the biology domain and demonstrate that the use of AA significantly improves convergence and reduces the training loss as the number of iterations increases. Our findings provide a promising perspective on the potential of Anderson acceleration in the training of simple machine learning classifiers and underscore the importance of further research in this area. By showing the effectiveness of AA in this setting, we aim to inspire more studies that explore the applications of AA in classical machine learning.

📄 PDF Abstract BibTeX arXiv:2302.00347

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

SVM A Support Vector Machine, or SVM, is a non-parametric supervised learning model. For non-linear classification and regression, they utilise the kernel trick to map inputs…

Anderson Acceleration of Proximal Gradient Methods

2019-10-18 · ICML 2020 1 · Vien V. Mai, Mikael Johansson

Anderson acceleration is a well-established and simple technique for speeding up fixed-point computations with countless applications. Previous studies of Anderson acceleration in optimization have only been able to prov…

A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization

2018-09-07 · Zhize Li, Jian Li

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-\a…

subspace methods

Anderson acceleration for iteratively reweighted $\ell_1$ algorithm

2024-03-12 · Kexin Li

Iteratively reweighted L1 (IRL1) algorithm is a common algorithm for solving sparse optimization problems with nonconvex and nonsmooth regularization. The development of its acceleration algorithm, often employing Nester…

Anderson Acceleration for Reinforcement Learning

2018-09-25 · Matthieu Geist, Bruno Scherrer

Anderson acceleration is an old and simple method for accelerating the computation of a fixed point. However, as far as we know and quite surprisingly, it has never been applied to dynamic programming or reinforcement le…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

2021-10-17 · NeurIPS 2021 12 · Ke Sun, Yafei Wang, Yi Liu, Yingnan Zhao 외

Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)