Please, Don’t Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status
This paper argues for the widest possible use of bootstrap confidence intervals for comparing NLP system performances instead of the state-of-the-art status (SOTA) and statistical significance testing. Their main benefits are to draw attention to the difference in performance between two systems and to help assessing the degree of superiority of one system over another. Two cases studies, one comparing several systems and the other based on a K-fold cross-validation procedure, illustrate these benefits.
Code (1)
Similar Papers 제목 키워드 기반
Please, Don't Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status
This paper argues for the widest possible use of bootstrap confidence intervals for comparing NLP system performances instead of the state-of-the-art status (SOTA) and statistical significance testing. Their main benefit…
Testing for Differences in Gaussian Graphical Models: Applications to Brain Connectivity
Functional brain networks are well described and estimated from data with Gaussian Graphical Models (GGMs), e.g. using sparse inverse covariance estimators. Comparing functional connectivity of subjects in two population…
Functional ConnectivityConfidence Intervals for Recursive Journal Impact Factors
We compute confidence intervals for recursive impact factors, that take into account that some citations are more prestigious than others, as well as for the associated ranks of journals, applying the methods to the popu…
A Simple and Adaptive Confidence Interval when Nuisance Parameters Satisfy an Inequality
Inequalities may appear in many models. They can be as simple as assuming a parameter is nonnegative, possibly a regression coefficient or a treatment effect. This paper focuses on the case that there is only one inequal…
regressionvalidShort and Simple Confidence Intervals when the Directions of Some Effects are Known
We provide adaptive confidence intervals on a parameter of interest in the presence of nuisance parameters when some of the nuisance parameters have known signs. The confidence intervals are adaptive in the sense that th…
valid