Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs
Uncertainty quantification is crucial for building reliable and trustable machine learning systems. We propose to estimate uncertainty in recurrent neural networks (RNNs) via stochastic discrete state transitions over recurrent timesteps. The uncertainty of the model can be quantified by running a prediction several times, each time sampling from the recurrent state transition distribution, leading to potentially different results if the model is uncertain. Alongside uncertainty quantification, our proposed method offers several advantages in different settings. The proposed method can (1) learn deterministic and probabilistic automata from data, (2) learn well-calibrated models on real-world classification tasks, (3) improve the performance of out-of-distribution detection, and (4) control the exploration-exploitation trade-off in reinforcement learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Out-of-Distribution Detectionreinforcement-learningReinforcement Learning (RL)Uncertainty QuantificationSimilar Papers 제목 키워드 기반
LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy
LLMs' overconfidence, particularly when hallucinating, poses a significant challenge for the deployment of the models in safety-critical settings and makes a reliable estimation of uncertainty necessary. Existing approac…
Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study
Large Language Models (LLMs) have been widely employed in programming language analysis to enhance human productivity. Yet, their reliability can be compromised by various code distribution shifts, leading to inconsisten…
Can We Trust You? On Calibration of a Probabilistic Object Detector for Autonomous Driving
Reliable uncertainty estimation is crucial for perception systems in safe autonomous driving. Recently, many methods have been proposed to model uncertainties in deep learning based object detectors. However, the estimat…
3D Object DetectionAutonomous DrivingObjectobject-detection+1A Geometric Method for Improved Uncertainty Estimation in Real-time
Machine learning classifiers are probabilistic in nature, and thus inevitably involve uncertainty. Predicting the probability of a specific input to be correct is called uncertainty (or confidence) estimation and is cruc…
ManagementParametric and Multivariate Uncertainty Calibration for Regression and Object Detection
Reliable spatial uncertainty evaluation of object detection models is of special interest and has been subject of recent work. In this work, we review the existing definitions for uncertainty calibration of probabilistic…
object-detectionObject DetectionObject Trackingregression+1