Instance-Adaptive Hypothesis Tests with Heterogeneous Agents
We study hypothesis testing over a heterogeneous population of strategic agents with private information. Any single test applied uniformly across the population yields statistical error that is sub-optimal relative to the performance of an oracle given access to the private information. We show how it is possible to design menus of statistical contracts that pair type-optimal tests with payoff structures, inducing agents to self-select according to their private information. This separating menu elicits agent types and enables the principal to match the oracle performance even without a priori knowledge of the agent type. Our main result fully characterizes the collection of all separating menus that are instance-adaptive, matching oracle performance for an arbitrary population of heterogeneous agents. We identify designs where information elicitation is essentially costless, requiring negligible additional expense relative to a single-test benchmark, while improving statistical performance. Our work establishes a connection between proper scoring rules and menu design, showing how the structure of the hypothesis test constrains the elicitable information. Numerical examples illustrate the geometry of separating menus and the improvements they deliver in error trade-offs. Overall, our results connect statistical decision theory with mechanism design, demonstrating how heterogeneity and strategic participation can be harnessed to improve efficiency in hypothesis testing.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Pair copula constructions of point-optimal sign-based tests for predictive linear and nonlinear regressions
We propose pair copula constructed point-optimal sign tests in the context of linear and nonlinear predictive regressions with endogenous, persistent regressors, and disturbances exhibiting serial (nonlinear) dependence.…
validHeterogeneous Dense Subhypergraph Detection
We study the problem of testing the existence of a heterogeneous dense subhypergraph. The null hypothesis corresponds to a heterogeneous Erd\"{o}s-R\'{e}nyi uniform random hypergraph and the alternative hypothesis corres…
Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents
Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomous…
PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents
Self-evolving agents improve by repeatedly proposing changes to their own prompts, skills, or workflows and keeping those that score higher on a small held-out set. Almost all effort has gone into the proposer that gener…
Hypothesis Testing for Class-Conditional Label Noise
In this paper we provide machine learning practitioners with tools to answer the question: is there class-conditional noise in my labels? In particular, we present hypothesis tests to check whether a given dataset of ins…