paper-with-me

홈 › Papers

The Reliability Gap in Benchmark Auditing: Distribution Shift and Scale as Failure Modes of Contamination Detection

2026-06-02 · Wojciech Zarzecki, Jan Dubiński, Sebastian Cygert arxiv

Benchmark contamination, where evaluation examples appear in a model's training data, threatens the validity of LLM assessment. Statistical tools for detecting training-data membership exist, but have been validated almost exclusively in controlled academic regimes: large, homogeneous pre-training corpora and transparent, single-stage training pipelines. Whether these methods remain reliable in realistic auditing scenarios remains unclear. We identify two under-studied failure modes: distribution shift, which arises when suspect and validation sets violate the IID assumption, and scale constraints, which arise because benchmarks are orders of magnitude smaller than pre-training corpora. We systematically evaluate three leading paradigms, LLM Dataset Inference, Post-Hoc Dataset Inference, and CoDeC, across 25 models from multiple families (including Pythia, OLMo 2, and specialised cultural and medical LLMs) and scales (up to 27B). We then further extend our analysis to frontier industry models. Across 335 evaluations, only 201 yield correct outcomes. LLM Dataset Inference results in false positives under distribution shift, Post-Hoc Dataset Inference is underpowered at benchmark scale, and CoDeC provides only coarse provenance signals that are insufficient to verify individual benchmark splits. Our results reveal a systematic reliability gap between controlled validation and practical benchmark auditing, and show that statistical detection cannot yet replace transparent data provenance. We open-source our benchmark for further research.

📄 PDF Abstract BibTeX arXiv:2606.03305

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

TRACE-Seg3D: Counterfactual Context Auditing For Robust 3D Glioma Segmentation Under Institutional Shift

2026-07-08 · Nguyen Linh Dan Le, Nguyen Pham Hoang Le, Tran Dang Khoi arxiv

Medical image segmentation models can achieve strong benchmark performance while remaining sensitive to scanner, protocol, and institutional variation. These context shifts alter image appearance without changing the und…

Medical Image Segmentation

An autonomous agent for auditing and improving the reliability of clinical AI models

2025-07-08 · Lukas Kuhn, Florian Buettner arxiv

The deployment of AI models in clinical practice faces a critical challenge: models achieving expert-level performance on benchmarks can fail catastrophically when confronted with real-world variations in medical imaging…

From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift

2025-09-30 · Ahmad-Reza Ehyaei, Golnoosh Farnadi, Samira Samadi arxiv

Group-fairness metrics (e.g., equalized odds) can vary sharply across resamples and are especially brittle under distribution shift, undermining reliable audits. We propose a Wasserstein distributionally robust framework…

CoDA: Exploring Chain-of-Distribution Attacks and Post-Hoc Token-Space Repair for Medical Vision-Language Models

2026-03-19 · Xiang Chen, Fangfang Yang, Chunlei Meng, Yuxian Dong 외 arxiv

Medical vision--language models (MVLMs) are increasingly used as perceptual backbones in radiology pipelines and as the visual front end of multimodal assistants, yet their reliability under real clinical workflows remai…

Revisiting Data Auditing in Large Vision-Language Models

2025-04-25 · Hongyu Zhu, Sichu Liang, Wenwen Wang, Boheng Li 외

With the surge of large language models (LLMs), Large Vision-Language Models (VLMs)--which integrate vision encoders with LLMs for accurate visual grounding--have shown great potential in tasks like generalist agents and…

Visual Grounding