Hypothesis Testing for High-Dimensional Multinomials: A Selective Review
The statistical analysis of discrete data has been the subject of extensive statistical research dating back to the work of Pearson. In this survey we review some recently developed methods for testing hypotheses about high-dimensional multinomials. Traditional tests like the $\chi^2$ test and the likelihood ratio test can have poor power in the high-dimensional setting. Much of the research in this area has focused on finding tests with asymptotically Normal limits and developing (stringent) conditions under which tests have Normal limits. We argue that this perspective suffers from a significant deficiency: it can exclude many high-dimensional cases when - despite having non Normal null distributions - carefully designed tests can have high power. Finally, we illustrate that taking a minimax perspective and considering refinements of this perspective can lead naturally to powerful and practical tests.
Code (0)
등록된 구현이 없습니다.
Tasks
Two-sample testingVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Hypothesis Testing For Densities and High-Dimensional Multinomials: Sharp Local Minimax Rates
We consider the goodness-of-fit testing problem of distinguishing whether the data are drawn from a specified distribution, versus a composite alternative separated from the null in the total variation metric. In the dis…
Two-sample testingTesting High-dimensional Multinomials with Applications to Text Analysis
Motivated by applications in text mining and discrete distribution inference, we investigate the testing for equality of probability mass functions of $K$ groups of high-dimensional multinomial distributions. A test stat…
DiversityVocal Bursts Intensity PredictionA projection pursuit framework for testing general high-dimensional hypothesis
This article develops a framework for testing general hypothesis in high-dimensional models where the number of variables may far exceed the number of observations. Existing literature has considered less than a handful …
Variable SelectionVocal Bursts Intensity PredictionKernel Two-Sample Hypothesis Testing Using Kernel Set Classification
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class …
ClassificationGeneral ClassificationTwo-sample testingVocal Bursts Valence PredictionBounded P-values in Parametric Programming-based Selective Inference
Selective inference (SI) has been actively studied as a promising framework for statistical hypothesis testing for data-driven hypotheses. The basic idea of SI is to make inferences conditional on an event that a hypothe…
feature selection