Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation

2022-09-14 · COLING 2022 10 · Yanan Wu, Zhiyuan Zeng, Keqing He, Yutao Mou, Pei Wang, Weiran Xu

Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set. Previous softmax-based detection algorithms are proved to be overconfident for OOD samples. In this paper, we analyze overconfident OOD comes from distribution uncertainty due to the mismatch between the training and test distributions, which makes the model can't confidently make predictions thus probably causing abnormal softmax scores. We propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout. Our method is flexible and easily pluggable into existing softmax-based baselines and gains 33.33\% OOD F1 improvements with increasing only 0.41\% inference time compared to MSP. Further analyses show the effectiveness of Bayesian learning for OOD detection.

📄 PDF Abstract BibTeX arXiv:2209.06612

Code (1)

pris-nlp/coling2022_bayesian-for-ood 공식 구현 pytorch

Tasks

Out of Distribution (OOD) Detection

Methods 이 논문이 사용한 방법론

Test 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Fusion framework and multimodality for the Laplacian approximation of Bayesian neural networks

2023-10-12 · Magnus Malmström, Isaac Skog, Daniel Axehill, Fredrik Gustafsson

This paper considers the problem of sequential fusion of predictions from neural networks (NN) and fusion of predictions from multiple NN. This fusion strategy increases the robustness, i.e., reduces the impact of one in…

image-classification Image Classification

Bayesian E(3)-Equivariant Interatomic Potential with Iterative Restratification of Many-body Message Passing

2025-10-03 · Soohaeng Yoo Willow, Tae Hyeon Park, Gi Beom Sim, Sung Wook Moon 외 arxiv

Machine learning potentials (MLPs) have become essential for large-scale atomistic simulations, enabling ab initio-level accuracy with computational efficiency. However, current MLPs struggle with uncertainty quantificat…

Computational Efficiency Active Learning

Identity Curvature Laplace Approximation for Improved Out-of-Distribution Detection

2023-12-16 · Maksim Zhdanov, Stanislav Dereka, Sergey Kolesnikov

Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational d…

Attribute Out-of-Distribution Detection Out of Distribution (OOD) Detection

Approximating the clusters' prior distribution in Bayesian nonparametric models

2020-11-23 · pproximateinference AABI Symposium 2021 1 · Daria Bystrova, Julyan Arbel, Guillaume Kon Kam King, François Deslandes

In Bayesian nonparametrics, knowledge of the prior distribution induced on the number of clusters is key for prior specification and calibration. However, evaluating this prior is infamously difficult even for moderate s…

Uncertainty quantification for trustworthy deep learning: Methods and measures

2026-07-30 · H. Martin Gillis, Thomas Trappenberg arxiv

The deployment of deep neural networks in safety-critical domains demands reliable estimates of predictive confidence, yet conventional architectures lack principled uncertainty quantification. This survey provides a str…

Out-of-Distribution Detection