Appendix - Recommended Statistical Significance Tests for NLP Tasks
Statistical significance testing plays an important role when drawing conclusions from experimental results in NLP papers. Particularly, it is a valuable tool when one would like to establish the superiority of one algorithm over another. This appendix complements the guide for testing statistical significance in NLP presented in \cite{dror2018hitchhiker} by proposing valid statistical tests for the common tasks and evaluation measures in the field.
Code (1)
Tasks
validSimilar Papers 제목 키워드 기반
The Hitchhiker's Guide to Testing Statistical Significance in Natural Language Processing
Statistical significance testing is a standard statistical tool designed to ensure that experimental results are not coincidental. In this opinion/ theoretical paper we discuss the role of statistical significance testin…
SurveyUsing Score Distributions to Compare Statistical Significance Tests for Information Retrieval Evaluation
Statistical significance tests can provide evidence that the observed difference in performance between two methods is not due to chance. In Information Retrieval, some studies have examined the validity and suitability …
Information RetrievalRetrievalContinuous Monitoring via Repeated Significance
Requiring statistical significance at multiple interim analyses to declare a statistically significant result for an AB test allows less stringent requirements for significance at each interim analysis. Repeated repeated…
CircaCompare: a method to estimate and statistically support differences in mesor, amplitude and phase, between circadian rhythms
Motivation A fundamental interest in chronobiology is to compare patterns between groups of rhythmic data. However, many existing methods are ill-equipped to derive statements concerning the statistical significance of …
Rhythmdeep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks
A lot of Machine Learning (ML) and Deep Learning (DL) research is of an empirical nature. Nevertheless, statistical significance testing (SST) is still not widely used. This endangers true progress, as seeming improvemen…