Using Fisher's Exact Test to Evaluate Association Measures for N-grams
To determine whether some often-used lexical association measures assign high scores to n-grams that chance could have produced as frequently as observed, we used an extension of Fisher's exact test to sequences longer than two words to analyse a corpus of four million words. The results, based on the precision-recall curve and a new index called chance-corrected average precision, show that, as expected, simple-ll is extremely effective. They also show, however, that MI3 is more efficient than the other hypothesis tests-based measures and even reaches a performance level almost equal to simple-ll for 3-grams. It is additionally observed that some measures are more efficient for 3-grams than for 2-grams, while others stagnate.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
\'Evaluation de mesures d'association pour les bigrammes et les trigrammes au moyen du test exact de Fisher (Using Fisher's Exact Test to Evaluate Association Measures for Bigrams and Trigrams)
Pour d{\'e}terminer si certaines mesures d{'}association lexicale fr{\'e}quemment employ{\'e}es en TAL attribuent des scores {\'e}lev{\'e}s {\`a} des n-grammes que le hasard aurait pu produire aussi souvent qu{'}observ{\…
es-enR. A. Fisher's Exact Test Revisited
This note provides a conceptual clarification of Ronald Aylmer Fisher's (1935) pioneering exact test in the context of the Lady Testing Tea experiment. It unveils a critical implicit assumption in Fisher's calibration: t…
Multi-Unit Directional Measures of Association: Moving Beyond Pairs of Words
This paper formulates and evaluates a series of multi-unit measures of directional association, building on the pairwise {\Delta}P measure, that are able to quantify association in sequences of varying length and type of…
SegmentationDiscussion of `Multiscale Fisher's Independence Test for Multivariate Dependence'
We discuss how MultiFIT, the Multiscale Fisher's Independence Test for Multivariate Dependence proposed by Gorsky and Ma (2022), compares to existing linear-time kernel tests based on the Hilbert-Schmidt independence cri…
A Fisher's exact test justification of the TF-IDF term-weighting scheme
Term frequency-inverse document frequency, or TF-IDF for short, is arguably the most celebrated mathematical expression in the history of information retrieval. Conceived as a simple heuristic quantifying the extent to w…
Information Retrieval