Multinomial Logistic Regression Algorithms via Quadratic Gradient

2022-08-14 · John Chiang

Multinomial logistic regression, also known by other names such as multiclass logistic regression and softmax regression, is a fundamental classification method that generalizes binary logistic regression to multiclass problems. A recently work proposed a faster gradient called $\texttt{quadratic gradient}$ that can accelerate the binary logistic regression training, and presented an enhanced Nesterov's accelerated gradient (NAG) method for binary logistic regression. In this paper, we extend this work to multiclass logistic regression and propose an enhanced Adaptive Gradient Algorithm (Adagrad) that can accelerate the original Adagrad method. We test the enhanced NAG method and the enhanced Adagrad method on some multiclass-problem datasets. Experimental results show that both enhanced methods converge faster than their original ones respectively.

📄 PDF Abstract BibTeX arXiv:2208.06828

Code (1)

petitioner/ml.multiclasslrtraining 공식 구현

Tasks

regression

Methods 이 논문이 사용한 방법론

Test 설명 없음

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

AdaGrad AdaGrad is a stochastic optimization method that adapts the learning rate to the parameters. It performs smaller updates for parameters associated with frequently occurring…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

2022-01-26 · John Chiang

Training logistic regression over encrypted data has been a compelling approach in addressing security concerns for several years. In this paper, we introduce an efficient gradient variant, called $quadratic$ $gradient$,…

Privacy Preserving regression

PIANO: A Fast Parallel Iterative Algorithm for Multinomial and Sparse Multinomial Logistic Regression

2020-02-21 · R. Jyothi, P. Babu

Multinomial Logistic Regression is a well-studied tool for classification and has been widely used in fields like image processing, computer vision and, bioinformatics, to name a few. Under a supervised classification sc…

feature selection regression

FIRAL: An Active Learning Algorithm for Multinomial Logistic Regression

2024-09-11 · NeurIPS 2023 11 · Youguang Chen, George Biros

We investigate theory and algorithms for pool-based active learning for multiclass classification using multinomial logistic regression. Using finite sample analysis, we prove that the Fisher Information Ratio (FIR) lowe…

Active Learning regression

DS-MLR: Exploiting Double Separability for Scaling up Distributed Multinomial Logistic Regression

2016-04-16 · Parameswaran Raman, Sriram Srinivasan, Shin Matsushima, Xinhua Zhang 외

Scaling multinomial logistic regression to datasets with very large number of data points and classes is challenging. This is primarily because one needs to compute the log-partition function on every data point. This ma…

Blocking Multi-class Classification regression

Convolutional networks and learning invariant to homogeneous multiplicative scalings

2015-06-26 · Mark Tygert, Arthur Szlam, Soumith Chintala, Marc'Aurelio Ranzato 외

The conventional classification schemes -- notably multinomial logistic regression -- used in conjunction with convolutional networks (convnets) are classical in statistics, designed without consideration for the usual c…

Classification General Classification regression