Improved User Identification through Calibrated Monte-Carlo Dropout
This paper presents an enhanced approach to user identification using smartphone and wearable sensor data. Our methodology involves segmenting input data and independently analyzing subsequences with CNNs. During testing, we apply calibrated Monte-Carlo Dropout to measure prediction uncertainty. By leveraging the weights obtained from uncertainty quantification, we integrate the results through weighted averaging, thereby improving overall identification accuracy. The main motivation behind this paper is the need to calibrate the CNN for improved weighted averaging. It has been observed that incorrect predictions often receive high confidence, while correct predictions are assigned lower confidence. To tackle this issue, we have implemented the Ensemble of Near Isotonic Regression (ENIR) as an advanced calibration technique. This ensures that certainty scores more accurately reflect the true likelihood of correctness. Furthermore, our experiment shows that calibrating CNN reduces the need for Monte Carlo samples in uncertainty quantification, thereby reducing computational costs. Our thorough evaluation and comparison of different calibration methods have shown improved accuracy in user identification across multiple datasets. Our results showed notable performance improvements when compared to the latest models available. In particular, our approach achieved better results than DB2 by 1.12% and HAR by 0.3% in accuracy.
Code (1)
Tasks
Classifier calibrationUncertainty QuantificationUser IdentificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Uncertainty Quantification to Enhance Probabilistic Fusion Based User Identification Using Smartphones
User identification through smartphones and wearable sensors holds promise but faces challenges from variability in user activities and sampling windows. This paper presents a method that takes into account uncertainties…
Uncertainty QuantificationUser IdentificationRobust Minimum Distance Inference in Structural Models
This paper proposes minimum distance inference for a structural parameter of interest, which is robust to the lack of identification of other structural nuisance parameters. Some choices of the weighting matrix lead to a…
Explain then Rank: Scale Calibration of Neural Rankers Using Natural Language Explanations from LLMs
In search settings, calibrating the scores during the ranking process to quantities such as click-through rates or relevance levels enhances a system's usefulness and trustworthiness for downstream users. While previous …
Document RankingLearning-To-RankLaplacian Autoencoders for Learning Stochastic Representations
Established methods for unsupervised representation learning such as variational autoencoders produce none or poorly calibrated uncertainty estimates making it difficult to evaluate if learned representations are stable …
Bayesian InferenceOut-of-Distribution DetectionRepresentation LearningSphUnc: Hyperspherical Uncertainty Decomposition and Causal Identification via Information Geometry
Reliable decision-making in complex multi-agent systems requires calibrated predictions and interpretable uncertainty. We introduce SphUnc, a unified framework combining hyperspherical representation learning with struct…
Representation Learning