BOKBO (Best of K Bad Options): Calibrated Abstention for VLA Policies

2026-05-28 · Anya Singh, Cabrel Happi, Jai Relan, Varun Nair, Vidyut Baradwaj arxiv

Test-time scaling for vision-language-action (VLA) policies, methods such as RoboMonkey, SEAL, MG-Select, and V-GPS, samples K candidate action chunks at inference and executes the verifier-best. When all K candidates are unsafe, the system executes a violating action with no warning. We propose BOKBO, the first conformal abstention layer for K-sample VLA inference, providing finite-sample distribution-free guarantees on executed-violation rate. We provide both global and per-task (Mondrian) variants, with the per-task variant closing the conditional gap on the hardest tasks. Our analysis exposes a structural failure of policy-internal nonconformity scores under perturbation-based K-sampling: the base-policy confidence proxy and K-sample disagreement correlate at 0.98 with the action-noise hyperparameter $σ$, while correlating at the noise floor with actual safety violations. We test the failure's scope by replicating the analysis under token-level temperature sampling and find the failure is mechanism-specific and partially mitigated under policy-stochasticity-based sampling. A learned violation predictor conditioned on semantic visual features and task identity supports tight calibration: at $ε$ = 0.05 on libero_object_temp_x0.1 with OpenVLA-OFT, the conditional CRC bound holds on 86% of bootstrap splits with 78% coverage and 70% net task success. Mondrian-BOKBO raises the minimum per-task conditional hold fraction from 0.71 to 0.93. Results are stable across 5 training seeds, replicate within bootstrap noise on $π_0$-FAST, hold on libero_spatial_temp_x0.1 as a co-equal benchmark, and survive four within-suite distribution shifts. We additionally identify and correct a methodological pitfall: globally-set force thresholds well below expert-typical manipulation forces conflate unsafe behavior with normal manipulation, inflating violation rates by $5\times$.

📄 PDF Abstract BibTeX arXiv:2605.30660

Code (0)

등록된 구현이 없습니다.

Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty

2026-01-18 · Sravanthi Machcha, Sushrita Yerra, Sahil Gupta, Aishwarya Sahoo 외 arxiv

Current evaluation of large language models (LLMs) overwhelmingly prioritizes accuracy; however, in real-world and safety-critical applications, the ability to abstain when uncertain is equally vital for trustworthy depl…

Question Answering

Causal Evidence that Language Models use Confidence to Drive Behavior

2026-03-23 · Dharshan Kumaran, Nathaniel Daw, Simon Osindero, Petar Veličković 외 arxiv

Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstrates that confidence signals can be extracted from language model outputs…

Kinematic Tokenization: Optimization-Based Continuous-Time Tokens for Learnable Decision Policies in Noisy Time Series

2026-01-15 · Griffin Kearney arxiv

Transformers are designed for discrete tokens, yet many real-world signals are continuous processes observed through noisy sampling. Discrete tokenizations (raw values, patches, finite differences) can be brittle in low …

Are LLM Decisions Faithful to Verbal Confidence?

2026-01-12 · Jiawei Wang, Yanfei Zhou, Siddartha Devic, Deqing Fu arxiv

Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decisio…

Decision Making

Knowing When Not to Predict: Self Supervised Learning and Abstention for Safer DR Screening

2026-05-18 · Muskaan Chopra, Lorenz Sparrenberg, Jan H. Terheyden, Rafet Sifa arxiv

Self-supervised learning (SSL) is now a standard way to pretrain medical image models, but performance is still mostly judged by downstream accuracy. For safety-critical screening tasks such as diabetic retinopathy gradi…

Diabetic Retinopathy Grading Self-Supervised Learning