Detecting Underspecification with Local Ensembles
We present local ensembles, a method for detecting underspecification -- when many possible predictors are consistent with the training data and model class -- at test time in a pre-trained model. Our method uses local second-order information to approximate the variance of predictions across an ensemble of models from the same class. We compute this approximation by estimating the norm of the component of a test point's gradient that aligns with the low-curvature directions of the Hessian, and provide a tractable method for estimating this quantity. Experimentally, we show that our method is capable of detecting when a pre-trained model is underspecified on test data, with applications to out-of-distribution detection, detecting spurious correlates, and active learning.
Code (2)
Tasks
Active LearningOut-of-Distribution DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Who is the richest club in the championship? Detecting and Rewriting Underspecified Questions Improve QA Performance
Large language models (LLMs) perform well on well-posed questions, yet standard question-answering (QA) benchmarks remain far from solved. We argue that this gap is partly due to underspecified questions - queries whose …
Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
Reward models play a key role in aligning language model applications towards human preferences. However, this setup creates an incentive for the language model to exploit errors in the reward model to achieve high estim…
Language ModelingLanguage ModellingRerankingConsistent Explanations in the Face of Model Indeterminacy via Ensembling
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (nearly) equally well-performing models f…
Underspecification Presents Challenges for Credibility in Modern Machine Learning
ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many…
BIG-bench Machine LearningCOCORELI: Enforcing Execution Preconditions for Reliable Collaborative Instruction Following
Autonomous agents executing human instructions must operate reliably even when instructions are incomplete. While recent approaches improve detection of missing information, detection alone is insufficient: agents often …
Instruction Following