Uncertainty-Aware Search and Value Models: Mitigating Search Scaling Flaws in LLMs
Value model-guided search is effective in steering the generation but suffers from scaling flaws: Its superiority diminishes with larger sample sizes, underperforming non-search baselines. This limitation arises from reliability degradation in value models in unseen reasoning paths. To address this, we propose an uncertainty-aware search framework that includes two key components: (1) uncertainty-aware value models that incorporate uncertainty into predictions, and (2) an uncertainty-aware selection process using the proposed efficient Group Thompson Sampling algorithm. Experiments on GSM8K show that our method mitigates search scaling flaws, achieving 90.5% coverage at 16 samples compared to 85.8% for conventional value-guided search. This work establishes the first systematic integration of uncertainty quantification in LLM search paradigms.
Code (0)
등록된 구현이 없습니다.
Tasks
GSM8KThompson SamplingUncertainty QuantificationSimilar Papers 제목 키워드 기반
Uncertainty Aware AI ML: Why and How
This paper argues the need for research to realize uncertainty-aware artificial intelligence and machine learning (AI\&ML) systems for decision support by describing a number of motivating scenarios. Furthermore, the pap…
BIG-bench Machine LearningHierarchical Uncertainty-Aware Graph Neural Network
Recent research on graph neural networks (GNNs) has explored mechanisms for capturing local uncertainty and exploiting graph hierarchies to mitigate data sparsity and leverage structural properties. However, the synergis…
DiversityGraph Neural NetworkRepresentation LearningUncertainty-aware Distributional Offline Reinforcement Learning
Offline reinforcement learning (RL) presents distinct challenges as it relies solely on observational data. A central concern in this context is ensuring the safety of the learned policy by quantifying uncertainties asso…
Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)Pretrain-then-Adapt: Uncertainty-Aware Test-Time Adaptation for Text-based Person Search
Text-based person search faces inherent limitations due to data scarcity, driven by stringent privacy constraints and the high cost of manual annotation. To mitigate this, existing methods usually rely on a Pretrain-then…
Test-time AdaptationImage RetrievalPerson SearchUncertainty-Aware Post-Hoc Calibration: Mitigating Confidently Incorrect Predictions Beyond Calibration Metrics
Despite extensive research on neural network calibration, existing methods typically apply global transformations that treat all predictions uniformly, overlooking the heterogeneous reliability of individual predictions.…
Semantic Similarity