Stability-based Generalization Analysis for Mixtures of Pointwise and Pairwise Learning
Recently, some mixture algorithms of pointwise and pairwise learning (PPL) have been formulated by employing the hybrid error metric of "pointwise loss + pairwise loss" and have shown empirical effectiveness on feature selection, ranking and recommendation tasks. However, to the best of our knowledge, the learning theory foundation of PPL has not been touched in the existing works. In this paper, we try to fill this theoretical gap by investigating the generalization properties of PPL. After extending the definitions of algorithmic stability to the PPL setting, we establish the high-probability generalization bounds for uniformly stable PPL algorithms. Moreover, explicit convergence rates of stochastic gradient descent (SGD) and regularized risk minimization (RRM) for PPL are stated by developing the stability analysis technique of pairwise learning. In addition, the refined generalization bounds of PPL are obtained by replacing uniform stability with on-average stability.
Code (0)
등록된 구현이 없습니다.
Tasks
feature selectionGeneralization BoundsLearning TheorySimilar Papers 제목 키워드 기반
Randomized Pairwise Learning with Adaptive Sampling: A PAC-Bayes Analysis
We study stochastic optimization with data-adaptive sampling schemes to train pairwise learning models. Pairwise learning is ubiquitous, and it covers several popular learning tasks such as ranking, metric learning and A…
Metric LearningStochastic OptimizationUniHash: Unifying Pointwise and Pairwise Hashing Paradigms
Effective retrieval across both seen and unseen categories is crucial for modern image retrieval systems. Retrieval on seen categories ensures precise recognition of known classes, while retrieval on unseen categories pr…
Image RetrievalSharper Generalization Bounds for Pairwise Learning
Pairwise learning refers to learning tasks with loss functions depending on a pair of training examples, which includes ranking and metric learning as specific examples. Recently, there has been an increasing amount of a…
Generalization BoundsMetric LearningStability-based Generalization Analysis of Randomized Coordinate Descent for Pairwise Learning
Pairwise learning includes various machine learning tasks, with ranking and metric learning serving as the primary representatives. While randomized coordinate descent (RCD) is popular in various learning problems, there…
Generalization BoundsMetric LearningRevisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)
Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+,…
Contrastive Learning