A PAC-Bayesian Perspective on Structured Prediction with Implicit Loss Embeddings
Many practical machine learning tasks can be framed as Structured prediction problems, where several output variables are predicted and considered interdependent. Recent theoretical advances in structured prediction have focused on obtaining fast rates convergence guarantees, especially in the Implicit Loss Embedding (ILE) framework. PAC-Bayes has gained interest recently for its capacity of producing tight risk bounds for predictor distributions. This work proposes a novel PAC-Bayes perspective on the ILE Structured prediction framework. We present two generalization bounds, on the risk and excess risk, which yield insights into the behavior of ILE predictors. Two learning algorithms are derived from these bounds. The algorithms are implemented and their behavior analyzed, with source code available at \url{https://github.com/theophilec/PAC-Bayes-ILE-Structured-Prediction}.
Code (1)
Tasks
Generalization BoundsPredictionStructured PredictionSimilar Papers 제목 키워드 기반
A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings
We propose and analyze a novel theoretical and algorithmic framework for structured prediction. While so far the term has referred to discrete output spaces, here we consider more general settings, such as manifolds or s…
Structured PredictionRisk Minimization in Structured Prediction using Orbit Loss
We introduce a new surrogate loss function called orbit loss in the structured prediction framework, which has good theoretical and practical advantages. While the orbit loss is not convex, it has a simple analytical gra…
DecoderPredictionStructured PredictionConformal Prediction as Bayesian Quadrature
As machine learning-based prediction systems are increasingly used in high-stakes situations, it is important to understand how such predictive models will perform upon deployment. Distribution-free uncertainty quantific…
Conformal PredictionPredictionUncertainty QuantificationNeural-Symbolic Recommendation with Graph-Enhanced Information
The recommendation system is not only a problem of inductive statistics from data but also a cognitive task that requires reasoning ability. The most advanced graph neural networks have been widely used in recommendation…
Recommendation SystemsScalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data
We introduce a new Bayesian multi-class support vector machine by formulating a pseudo-likelihood for a multi-class hinge loss in the form of a location-scale mixture of Gaussians. We derive a variational-inference-based…
Active LearningGeneral ClassificationVariational Inference