Analysis of SVM with Indefinite Kernels

2009-12-01 · NeurIPS 2009 12 · Yiming Ying, Colin Campbell, Mark Girolami

The recent introduction of indefinite SVM by Luss and dAspremont [15] has effectively demonstrated SVM classification with a non-positive semi-definite kernel (indefinite kernel). This paper studies the properties of the objective function introduced there. In particular, we show that the objective function is continuously differentiable and its gradient can be explicitly computed. Indeed, we further show that its gradient is Lipschitz continuous. The main idea behind our analysis is that the objective function is smoothed by the penalty term, in its saddle (min-max) representation, measuring the distance between the indefinite kernel matrix and the proxy positive semi-definite one. Our elementary result greatly facilitates the application of gradient-based algorithms. Based on our analysis, we further develop Nesterovs smooth optimization approach [16,17] for indefinite SVM which has an optimal convergence rate for smooth problems. Experiments on various benchmark datasets validate our analysis and demonstrate the efficiency of our proposed algorithms.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SVM A Support Vector Machine, or SVM, is a non-parametric supervised learning model. For non-linear classification and regression, they utilise the kernel trick to map inputs…

Learning with Operator-valued Kernels in Reproducing Kernel Krein Spaces

2020-12-01 · NeurIPS 2020 12 · Akash Saha, Balamurugan Palaniappan

Operator-valued kernels have shown promise in supervised learning problems with functional inputs and functional outputs. The crucial (and possibly restrictive) assumption of positive definiteness of operator-valued kern…

Towards Unbiased Random Features with Lower Variance For Stationary Indefinite Kernels

2021-04-13 · Qin Luo, Kun Fang, Jie Yang, Xiaolin Huang

Random Fourier Features (RFF) demonstrate wellappreciated performance in kernel approximation for largescale situations but restrict kernels to be stationary and positive definite. And for non-stationary kernels, the cor…

regression

Probabilistic classifiers with low rank indefinite kernels

2016-04-08 · Frank-Michael Schleif, Andrej Gisbrecht, Peter Tino

Indefinite similarity measures can be frequently found in bio-informatics by means of alignment scores, but are also common in other fields like shape measures in image retrieval. Lacking an underlying vector space, the …

Image Retrieval Retrieval

Support Vector Machine Classification with Indefinite Kernels

2007-12-01 · NeurIPS 2007 12 · Ronny Luss, Alexandre d'Aspremont

In this paper, we propose a method for support vector machine classification using indefinite kernels. Instead of directly minimizing or stabilizing a nonconvex loss function, our method simultaneously finds the support …

Classification General Classification Robust classification

Indefinite Kernel Logistic Regression with Concave-inexact-convex Procedure

2017-07-06 · Fanghui Liu, Xiaolin Huang, Chen Gong, Jie Yang 외

In kernel methods, the kernels are often required to be positive definite, which restricts the use of many indefinite kernels. To consider those non-positive definite kernels, in this paper, we aim to build an indefinite…