paper-with-me

홈 › Papers

No-Regret Online Prediction with Strategic Experts

2023-05-24 · NeurIPS 2023 11

We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick $m\geq 1$ experts from a pool of $K$ experts and the overall utility is a modular or submodular function of the chosen experts. We focus on the setting in which experts act strategically and aim to maximize their influence on the algorithm's predictions by potentially misreporting their beliefs about the events. Among others, this setting finds applications in forecasting competitions where the learner seeks not only to make predictions by aggregating different forecasters but also to rank them according to their relative performance. Our goal is to design algorithms that satisfy the following two requirements: 1) $\textit{Incentive-compatible}$: Incentivize the experts to report their beliefs truthfully, and 2) $\textit{No-regret}$: Achieve sublinear regret with respect to the true beliefs of the best fixed set of $m$ experts in hindsight. Prior works have studied this framework when $m=1$ and provided incentive-compatible no-regret algorithms for the problem. We first show that a simple reduction of our problem to the $m=1$ setting is neither efficient nor effective. Then, we provide algorithms that utilize the specific structure of the utility functions to achieve the two desired goals.

📄 PDF Abstract BibTeX arXiv:2305.15331

Code (0)

등록된 구현이 없습니다.

Tasks

Prediction

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Similar Papers 제목 키워드 기반

No-Regret and Incentive-Compatible Online Learning

2020-02-20 · ICML 2020 1 · Rupert Freeman, David M. Pennock, Chara Podimata, Jennifer Wortman Vaughan

We study online learning settings in which experts act strategically to maximize their influence on the learning algorithm's predictions by potentially misreporting their beliefs about a sequence of binary events. Our go…

scoring rule

Online Learning with LLM Experts from Limited Feedback

2026-09-05 · Wang Wei, Soumyabrata Pal, Koyel Mukherjee, Franck Dernoncourt 외 hf

We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online setting with limited feedback. We formulate it as a bandit problem with K actions that represent expert…

On the price of exact truthfulness in incentive-compatible online learning with bandit feedback: A regret lower bound for WSU-UX

2024-04-08 · Ali Mortazavi, Junhao Lin, Nishant A. Mehta

In one view of the classical game of prediction with expert advice with binary outcomes, in each round, each expert maintains an adversarially chosen belief and honestly reports this belief. We consider a recently introd…

No-regret incentive-compatible online learning under exact truthfulness with non-myopic experts

2025-02-17 · Junpei Komiyama, Nishant A. Mehta, Ali Mortazavi

We study an online forecasting setting in which, over $T$ rounds, $N$ strategic experts each report a forecast to a mechanism, the mechanism selects one forecast, and then the outcome is revealed. In any given round, eac…

Efficient Competitions and Online Learning with Strategic Forecasters

2021-02-16 · Rafael Frongillo, Robert Gomez, Anish Thilagar, Bo Waggoner

Winner-take-all competitions in forecasting and machine-learning suffer from distorted incentives. Witkowski et al. 2018 identified this problem and proposed ELF, a truthful mechanism to select a winner. We show that, fr…