Null/No Information Rate (NIR): a statistical test to assess if a classification accuracy is significant for a given problem
In many research contexts, especially in the biomedical field, after studying and developing a classification system a natural question arises: "Is this accuracy enough high?", or better, "Can we say, with a statistically significant confidence, that our classification system is able to solve the problem"? To answer to this question, we can use the statistical test described in this paper, which is referred in some cases as NIR (No Information Rate or Null Information Rate).
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationSimilar Papers 제목 키워드 기반
Using Score Distributions to Compare Statistical Significance Tests for Information Retrieval Evaluation
Statistical significance tests can provide evidence that the observed difference in performance between two methods is not due to chance. In Information Retrieval, some studies have examined the validity and suitability …
Information RetrievalRetrievalTowards Reliable Testing for Multiple Information Retrieval System Comparisons
Null Hypothesis Significance Testing is the \textit{de facto} tool for assessing effectiveness differences between Information Retrieval systems. Researchers use statistical tests to check whether those differences will …
Information RetrievalRetrievalKernel Tests of Equivalence
We propose novel kernel-based tests for assessing the equivalence between distributions. Traditional goodness-of-fit testing is inappropriate for concluding the absence of distributional differences, because failure to r…
Alice and the Caterpillar: A more descriptive null model for assessing data mining results
We introduce novel null models for assessing the results obtained from observed binary transactional and sequence datasets, using statistical hypothesis testing. Our null models maintain more properties of the observed d…
DescriptiveChallenges in Statistically Rejecting the Perfect Competition Hypothesis Using Imperfect Competition Data
We theoretically prove why statistically rejecting the null hypothesis of perfect competition is challenging, known as a common problem in the literature. We also assess the finite sample performance of the conduct param…