Calibrating the Subjective
I conduct Rabin's (2000) calibration exercise in the subjective expected utility realm. I show that the rejection of some risky bet by a risk-averse agent only implies the rejection of more extreme and less desirable bets and nothing more.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?
Subjective tasks in NLP have been mostly relegated to objective standards, where the gold label is decided by taking the majority vote. This obfuscates annotator disagreement and the inherent uncertainty of the label. We…
Decision MakingHate Speech DetectionNatural Language InferenceJudging with Confidence: Calibrating Autoraters to Preference Distributions
The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or ``autoraters''. However, their reliability is limited by a foundational issue: they are trai…
Reinforcement LearningCalibrating “Cheap Signals” in Peer Review without a Prior
Peer review lies at the core of the academic process, but even well-intentioned reviewers can still provide noisy ratings. While ranking papers by average ratings may reduce noise, varying noise levels and systematic bia…
Semantic Latent Space Regression of Diffusion Autoencoders for Vertebral Fracture Grading
Vertebral fractures are a consequence of osteoporosis, with significant health implications for affected patients. Unfortunately, grading their severity using CT exams is hard and subjective, motivating automated grading…
regressionCalibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation
As LLM-based judges become integral to industry applications, obtaining well-calibrated uncertainty estimates efficiently has become critical for production deployment. However, existing techniques, such as verbalized co…