Large Scale, Large Margin Classification using Indefinite Similarity Measures

2014-05-27 · Omid Aghazadeh, Stefan Carlsson

Despite the success of the popular kernelized support vector machines, they have two major limitations: they are restricted to Positive Semi-Definite (PSD) kernels, and their training complexity scales at least quadratically with the size of the data. Many natural measures of similarity between pairs of samples are not PSD e.g. invariant kernels, and those that are implicitly or explicitly defined by latent variable models. In this paper, we investigate scalable approaches for using indefinite similarity measures in large margin frameworks. In particular we show that a normalization of similarity to a subset of the data points constitutes a representation suitable for linear classifiers. The result is a classifier which is competitive to kernelized SVM in terms of accuracy, despite having better training and test time complexities. Experimental results demonstrate that on CIFAR-10 dataset, the model equipped with similarity measures invariant to rigid and non-rigid deformations, can be made more than 5 times sparser while being more accurate than kernelized SVM using RBF kernels.

📄 PDF Abstract BibTeX arXiv:1405.6922

Code (0)

등록된 구현이 없습니다.

Tasks

Classification General Classification

Methods 이 논문이 사용한 방법론

SVM A Support Vector Machine, or SVM, is a non-parametric supervised learning model. For non-linear classification and regression, they utilise the kernel trick to map inputs…

Probabilistic classifiers with low rank indefinite kernels

2016-04-08 · Frank-Michael Schleif, Andrej Gisbrecht, Peter Tino

Indefinite similarity measures can be frequently found in bio-informatics by means of alignment scores, but are also common in other fields like shape measures in image retrieval. Lacking an underlying vector space, the …

Image Retrieval Retrieval

Towards Unbiased Random Features with Lower Variance For Stationary Indefinite Kernels

2021-04-13 · Qin Luo, Kun Fang, Jie Yang, Xiaolin Huang

Random Fourier Features (RFF) demonstrate wellappreciated performance in kernel approximation for largescale situations but restrict kernels to be stationary and positive definite. And for non-stationary kernels, the cor…

regression

Support Vector Machine Classification with Indefinite Kernels

2007-12-01 · NeurIPS 2007 12 · Ronny Luss, Alexandre d'Aspremont

In this paper, we propose a method for support vector machine classification using indefinite kernels. Instead of directly minimizing or stabilizing a nonconvex loss function, our method simultaneously finds the support …

Classification General Classification Robust classification

Data Assimilation for Sign-indefinite Priors: A generalization of Sinkhorn's algorithm

2023-08-22 · Anqi Dong, Tryphon T. Georgiou, Allen Tannenbaum

The purpose of this work is to develop a framework to calibrate signed datasets so as to be consistent with specified marginals by suitably extending the Schr\"odinger-Fortet-Sinkhorn paradigm. Specifically, we seek to r…

Trust-Region Algorithms for Training Responses: Machine Learning Methods Using Indefinite Hessian Approximations

2018-07-01 · Jennifer B. Erway, Joshua Griffin, Roummel F. Marcia, Riadh Omheni

Machine learning (ML) problems are often posed as highly nonlinear and nonconvex unconstrained optimization problems. Methods for solving ML problems based on stochastic gradient descent are easily scaled for very large …

BIG-bench Machine Learning