Using Subjective Logic to Estimate Uncertainty in Multi-Armed Bandit Problems
The multi-armed bandit problem is a classical decision-making problem where an agent has to learn an optimal action balancing exploration and exploitation. Properly managing this trade-off requires a correct assessment of uncertainty; in multi-armed bandits, as in other machine learning applications, it is important to distinguish between stochasticity that is inherent to the system (aleatoric uncertainty) and stochasticity that derives from the limited knowledge of the agent (epistemic uncertainty). In this paper we consider the formalism of subjective logic, a concise and expressive framework to express Dirichlet-multinomial models as subjective opinions, and we apply it to the problem of multi-armed bandits. We propose new algorithms grounded in subjective logic to tackle the multi-armed bandit problem, we compare them against classical algorithms from the literature, and we analyze the insights they provide in evaluating the dynamics of uncertainty. Our preliminary results suggest that subjective logic quantities enable useful assessment of uncertainty that may be exploited by more refined agents.
Code (1)
Tasks
Decision MakingMulti-Armed BanditsSimilar Papers 제목 키워드 기반
Subjective Logic Encodings
Many existing approaches for learning from labeled data assume the existence of gold-standard labels. According to these approaches, inter-annotator disagreement is seen as noise to be removed, either through refinement …
Hate Speech DetectionSentiment AnalysisKalman Filter Meets Subjective Logic: A Self-Assessing Kalman Filter Using Subjective Logic
Self-assessment is a key to safety and robustness in automated driving. In order to design safer and more robust automated driving functions, the goal is to self-assess the performance of each module in a whole automated…
Multifaceted Uncertainty Estimation for Label-Efficient Deep Learning
We present a novel multi-source uncertainty prediction approach that enables deep learning (DL) models to be actively trained with much less labeled data. By leveraging the second-order uncertainty representation provide…
Deep LearningBin-Conditional Conformal Prediction of Fatalities from Armed Conflict
Forecasting armed conflicts is a critical area of research with the potential to save lives and mitigate suffering. While existing forecasting models offer valuable point predictions, they often lack individual-level unc…
Conformal PredictionPredictionPrediction Intervalsquantile regressionEvidence-Aware Entropy Decomposition For Active Deep Learning
We present a novel multi-source uncertainty prediction approach that enables deep learning (DL) models to be actively trained with much less labeled data. By leveraging the second-order uncertainty representation provide…
Deep LearningDensity Estimation