Equivalence of Multicategory SVM and Simplex Cone SVM: Fast Computations and Statistical Theory
The multicategory SVM (MSVM) of Lee et al. (2004) is a natural generalization of the classical, binary support vector machines (SVM). However, its use has been limited by computational difficulties. The simplex-cone SVM (SCSVM) of Mroueh et al. (2012) is a computationally efficient multicategory classifier, but its use has been limited by a seemingly opaque interpretation. We show that MSVM and SCSVM are in fact exactly equivalent, and provide a bijection between their tuning parameters. MSVM may then be entertained as both a natural and computationally efficient multicategory extension of SVM. We further provide a Donsker theorem for finite-dimensional kernel MSVM and partially answer the open question pertaining to the very competitive performance of One-vs-Rest methods against MSVM. Furthermore, we use the derived asymptotic covariance formula to develop an inverse-variance weighted classification rule which improves on the One-vs-Rest approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Open-Ended Question AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Wasserstein k-means with sparse simplex projection
This paper presents a proposal of a faster Wasserstein $k$-means algorithm for histogram data by reducing Wasserstein distance computations and exploiting sparse simplex projection. We shrink data samples, centroids, and…
ClusteringOn Reject and Refine Options in Multicategory Classification
In many real applications of statistical learning, a decision made from misclassification can be too costly to afford; in this case, a reject option, which defers the decision until further investigation is conducted, is…
Binary ClassificationClassificationGeneral ClassificationLearning TheoryCausal Linear Topological Filters over a 2-Simplex
Topological filters via sheaves generalize the classical linear translation-invariant filter theory by attaching the filter computation locally to a simplicial topological space. This paper develops topological filters f…
TranslationMultiplicative Updates for Online Convex Optimization over Symmetric Cones
We study online convex optimization where the possible actions are trace-one elements in a symmetric cone, generalizing the extensively-studied experts setup and its quantum counterpart. Symmetric cones provide a unifyin…
Unimodal Distributions for Ordinal Regression
In many real-world prediction tasks, class labels contain information about the relative order between labels that are not captured by commonly used loss functions such as multicategory cross-entropy. Recently, the prefe…
regression