paper-with-me

홈 › Papers

Efficient Competitions and Online Learning with Strategic Forecasters

2021-02-16 · Rafael Frongillo, Robert Gomez, Anish Thilagar, Bo Waggoner

Winner-take-all competitions in forecasting and machine-learning suffer from distorted incentives. Witkowski et al. 2018 identified this problem and proposed ELF, a truthful mechanism to select a winner. We show that, from a pool of $n$ forecasters, ELF requires $\Theta(n\log n)$ events or test data points to select a near-optimal forecaster with high probability. We then show that standard online learning algorithms select an $\epsilon$-optimal forecaster using only $O(\log(n) / \epsilon^2)$ events, by way of a strong approximate-truthfulness guarantee. This bound matches the best possible even in the nonstrategic setting. We then apply these mechanisms to obtain the first no-regret guarantee for non-myopic strategic experts.

📄 PDF Abstract BibTeX arXiv:2102.08358

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

No-Regret Online Prediction with Strategic Experts

2023-05-24 · NeurIPS 2023 11

We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick $m\geq 1$ experts from a pool of $K$ experts and the overall utility is a modular…

Prediction

Evaluating Strategic Reasoning in Forecasting Agents

2026-04-28 · Tom Liptay, Dan Schwarz, Rafael Poyiadzi, Jack Wildman 외 arxiv

Forecasting benchmarks produce accuracy leaderboards but little insight into why some forecasters are more accurate than others. We introduce Bench to the Future 2 (BTF-2), 1,417 pastcasting questions with a frozen 15M-d…

Hierarchical Partitioning Forecaster

2023-05-22 · Christopher Mattern

In this work we consider a new family of algorithms for sequential prediction, Hierarchical Partitioning Forecasters (HPFs). Our goal is to provide appealing theoretical - regret guarantees on a powerful model class - an…

The impacts of various parameters on learning process and machine learning based performance prediction in online coding competitions

2021-12-29 · Hardik Patel, Purvi Koringa

Various parameters affect the performance of students in online coding competitions. Students' behavior, approach, emotions, and problem difficulty levels significantly impact their performance in online coding competiti…

BIG-bench Machine Learning

Comparing Sequential Forecasters

2021-09-30 · Yo Joong Choe, Aaditya Ramdas

Consider two forecasters, each making a single prediction for a sequence of events over time. We ask a relatively basic question: how might we compare these forecasters, either online or post-hoc, while avoiding unverifi…

valid