paper-with-me

홈 › Papers

Joint Scoring Rules: Zero-Sum Competition Avoids Performative Prediction

2024-12-30 · Rubi Hudson

In a decision-making scenario, a principal could use conditional predictions from an expert agent to inform their choice. However, this approach would introduce a fundamental conflict of interest. An agent optimizing for predictive accuracy is incentivized to manipulate their principal towards more predictable actions, which prevents that principal from being able to deterministically select their true preference. We demonstrate that this impossibility result can be overcome through the joint evaluation of multiple agents. When agents are made to engage in zero-sum competition, their incentive to influence the action taken is eliminated, and the principal can identify and take the action they most prefer. We further prove that this zero-sum setup is unique, efficiently implementable, and applicable under stochastic choice. Experiments in a toy environment demonstrate that training on a zero-sum objective significantly enhances both predictive accuracy and principal utility, and can eliminate previously learned manipulative behavior.

📄 PDF Abstract BibTeX arXiv:2412.20732

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making

Similar Papers 제목 키워드 기반

Scoring Rules for Performative Binary Prediction

2022-07-05 · Alan Chan

We construct a model of expert prediction where predictions can influence the state of the world. Under this model, we show through theoretical and numerical results that proper scoring rules can incentivize experts to m…

Prediction

Surface Form Competition: Why the Highest Probability Answer Isn't Always Right

2021-04-16 · Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi 외

Large language models have shown promising results in zero-shot settings (Brown et al.,2020; Radford et al., 2019). For example, they can perform multiple choice tasks simply by conditioning on a question and selecting t…

FormMultiple-choicevalid

Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right

2021-11-01 · EMNLP 2021 11 · Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi 외

Large language models have shown promising results in zero-shot settings. For example, they can perform multiple choice tasks simply by conditioning on a question and selecting the answer with the highest probability. Ho…

FormMultiple-choicevalid

Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition

2025-02-20 · Priya Kasimbeg, Frank Schneider, Runa Eschenhagen, Juhan Bae 외

The goal of the AlgoPerf: Training Algorithms competition is to evaluate practical speed-ups in neural network training achieved solely by improving the underlying training algorithms. In the external tuning ruleset, sub…

Optimizing Rescoring Rules with Interpretable Representations of Long-Term Information

2021-04-28 · Aaron Fisher

Analyzing temporal data (e.g., wearable device data) requires a decision about how to combine information from the recent and distant past. In the context of classifying sleep status from actigraphy, Webster's rescoring …

Binary Classification