A model of multiple hypothesis testing
Multiple hypothesis testing practices vary widely, without consensus on which are appropriate when. This paper provides an economic foundation for these practices designed to capture leading examples, such as regulatory approval on the basis of clinical trials. In studies of multiple treatments or sub-populations, adjustments may be appropriate depending on scale economies in the research production function, with control of classical notions of compound errors emerging in some but not all cases. In studies with multiple outcomes, indexing is appropriate and adjustments to test levels may be appropriate if the intended audience is heterogeneous. Data on actual costs in the drug approval process suggest both that some adjustment is warranted in that setting and that standard procedures may be overly conservative.
Code (0)
등록된 구현이 없습니다.
Tasks
modelSimilar Papers 제목 키워드 기반
Multiple Hypothesis Testing with Persistent Homology
Multiple hypothesis testing requires a control procedure. Simply increasing simulations or permutations to meet a Bonferroni-style threshold is prohibitively expensive. In this paper we propose a null model based approac…
Hypothesis Testing Prompting Improves Deductive Reasoning in Large Language Models
Combining different forms of prompts with pre-trained large language models has yielded remarkable results on reasoning tasks (e.g. Chain-of-Thought prompting). However, along with testing on more complex reasoning, thes…
Fact VerificationHypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models
In this paper, we present a visual analytics tool for enabling hypothesis-based evaluation of machine learning (ML) models. We describe a novel ML-testing framework that combines the traditional statistical hypothesis te…
BIG-bench Machine LearningLogical ReasoningTwo-sample testingPAPRIKA: Private Online False Discovery Rate Control
In hypothesis testing, a false discovery occurs when a hypothesis is incorrectly rejected due to noise in the sample. When adaptively testing multiple hypotheses, the probability of a false discovery increases as more te…
Two-sample testingA Generalized Graph Signal Processing Framework for Multiple Hypothesis Testing over Networks
We consider the multiple hypothesis testing (MHT) problem over the joint domain formed by a graph and a measure space. On each sample point of this joint domain, we assign a hypothesis test and a corresponding $p$-value.…