p-value peeking and estimating extrema
A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data "peeking" -- the practice of reporting only progressively extreme values of the test statistic as more data samples are collected. We develop principled mechanisms to estimate such running extrema of test statistics, which directly address the effect of peeking in some general scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Two-sample testingSimilar Papers 제목 키워드 기반
Dimensional Peeking for Low-Variance Gradients in Zeroth-Order Discrete Optimization via Simulation
Gradient-based optimization methods are commonly used to identify local optima in high-dimensional spaces. When derivatives cannot be evaluated directly, stochastic estimators can provide approximate gradients. However, …
Data-Driven Threshold Machine: Scan Statistics, Change-Point Detection, and Extreme Bandits
We present a novel distribution-free approach, the data-driven threshold machine (DTM), for a fundamental problem at the core of many learning tasks: choose a threshold for a given pre-specified level that bounds the tai…
Change Point DetectionComputational EfficiencyModeling Nonstationary Extremal Dependence via Deep Spatial Deformations
Modeling nonstationarity that often prevails in extremal dependence of spatial data can be challenging, and typically requires bespoke or complex spatial models that are difficult to estimate. Inference for stationary an…
DeepExtrema: A Deep Learning Approach for Forecasting Block Maxima in Time Series Data
Accurate forecasting of extreme values in time series is critical due to the significant impact of extreme events on human and natural systems. This paper presents DeepExtrema, a novel framework that combines a deep neur…
Time SeriesTime Series AnalysisShort time extremal response to step stimulus for a single cell {\sl E. coli}
After application of a step stimulus, in the form of a sudden change in attractant environment, the receptor activity and tumbling bias of an {\sl E. coli} cell change sharply to reach their extremal values before they g…