Likelihood-ratio calibration using prior-weighted proper scoring rules

2013-07-30 · Niko Brümmer, George Doddington

Prior-weighted logistic regression has become a standard tool for calibration in speaker recognition. Logistic regression is the optimization of the expected value of the logarithmic scoring rule. We generalize this via a parametric family of proper scoring rules. Our theoretical analysis shows how different members of this family induce different relative weightings over a spectrum of applications of which the decision thresholds range from low to high. Special attention is given to the interaction between prior weighting and proper scoring rule parameters. Experiments on NIST SRE'12 suggest that for applications with low false-alarm rate requirements, scoring rules tailored to emphasize higher score thresholds may give better accuracy than logistic regression.

📄 PDF Abstract BibTeX arXiv:1307.7981

Code (0)

등록된 구현이 없습니다.

Tasks

regressionscoring ruleSpeaker Recognition

Methods 이 논문이 사용한 방법론

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

General Bayesian Updating and the Loss-Likelihood Bootstrap

2017-09-22 · Simon Lyddon, Chris Holmes, Stephen Walker

In this paper we revisit the weighted likelihood bootstrap, a method that generates samples from an approximate Bayesian posterior of a parametric model. We show that the same method can be derived, without approximation…

The PAV algorithm optimizes binary proper scoring rules

2013-04-08 · Niko Brummer, Johan du Preez

There has been much recent interest in application of the pool-adjacent-violators (PAV) algorithm for the purpose of calibrating the probabilistic outputs of automatic pattern recognition and machine learning algorithms.…

Decision-Aligned Evaluation of Uncertainty Quantification

2026-06-25 · Annika Schneider, Tommy Rochussen, Joshua Stiller, Vincent Fortuin arxiv

Uncertainty estimates in machine learning are typically evaluated using generic metrics such as the negative log-likelihood and expected calibration error, yet good performance on such metrics does not necessarily imply …

Amortized Conditional Normalized Maximum Likelihood

2020-09-28 · Aurick Zhou, Sergey Levine

While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges. In this paper, we propose the amortized conditional normalized maximum…

Bayesian Inference

Calibration of Phone Likelihoods in Automatic Speech Recognition

2016-06-14 · David A. van Leeuwen, Joost van Doremalen

In this paper we study the probabilistic properties of the posteriors in a speech recognition system that uses a deep neural network (DNN) for acoustic modeling. We do this by reducing Kaldi's DNN shared pdf-id posterior…

Automatic Speech Recognition Automatic Speech Recognition (ASR)scoring rule speech-recognition+1