Large Deviation Analysis of Score-based Hypothesis Testing
Score-based statistical models play an important role in modern machine learning, statistics, and signal processing. For hypothesis testing, a score-based hypothesis test is proposed in \cite{wu2022score}. We analyze the performance of this score-based hypothesis testing procedure and derive upper bounds on the probabilities of its Type I and II errors. We prove that the exponents of our error bounds are asymptotically (in the number of samples) tight for the case of simple null and alternative hypotheses. We calculate these error exponents explicitly in specific cases and provide numerical studies for various other scenarios of interest.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Selective Probabilistic Classifier Based on Hypothesis Testing
In this paper, we propose a simple yet effective method to deal with the violation of the Closed-World Assumption for a classifier. Previous works tend to apply a threshold either on the classification scores or the loss…
Linear Hypothesis Testing in Dense High-Dimensional Linear Models
We propose a methodology for testing linear hypothesis in high-dimensional linear models. The proposed test does not impose any restriction on the size of the model, i.e. model sparsity or the loading vector representing…
regressionTwo-sample testingvalidVocal Bursts Intensity PredictionMinP Score Tests with an Inequality Constrained Parameter Space
Score tests have the advantage of requiring estimation alone of the model restricted by the null hypothesis, which often is much simpler than models defined under the alternative hypothesis. This is typically so when the…
Binary Hypothesis Testing for Softmax Models and Leverage Score Models
Softmax distributions are widely used in machine learning, including Large Language Models (LLMs) where the attention unit uses softmax distributions. We abstract the attention unit as the softmax model, where given a ve…
Incentivizing High-Quality Human Annotations with Golden Questions
Human-annotated data plays a vital role in training large language models (LLMs), such as supervised fine-tuning and human preference alignment. However, it is not guaranteed that paid human annotators produce high-quali…