A Crucial Parameter for Rank-Frequency Relation in Natural Languages
$f \propto r^{-\alpha} \cdot (r+\gamma)^{-\beta}$ has been empirically shown more precise than a na\"ive power law $f\propto r^{-\alpha}$ to model the rank-frequency ($r$-$f$) relation of words in natural languages. This work shows that the only crucial parameter in the formulation is $\gamma$, which depicts the resistance to vocabulary growth on a corpus. A method of parameter estimation by searching an optimal $\gamma$ is proposed, where a ``zeroth word'' is introduced technically for the calculation. The formulation and parameters are further discussed with several case studies.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter estimationRelationSimilar Papers 제목 키워드 기반
A Three-Parameter Rank-Frequency Relation in Natural Languages
We present that, the rank-frequency relation in textual data follows $f \propto r^{-\alpha}(r+\gamma)^{-\beta}$, where $f$ is the token frequency and $r$ is the rank by frequency, with ($\alpha$, $\beta$, $\gamma$) as pa…
RelationRare and Zero-shot Word Sense Disambiguation using Z-Reweighting
Word sense disambiguation (WSD) is a crucial problem in the natural language processing (NLP) community. Current methods achieve decent performance by utilizing supervised learning and large pre-trained language models. …
RelationWord Sense DisambiguationA Two Parameters Equation for Word Rank-Frequency Relation
Let $f (\cdot)$ be the absolute frequency of words and $r$ be the rank of words in decreasing order of frequency, then the following function can fit the rank-frequency relation \[ f (r;s,t) = \left(\frac{r_{\tt max}}{r}…
RelationVocal Bursts Valence PredictionFAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection
Adapting models pre-trained on large-scale datasets is a proven way to reach strong performance quickly for down-stream tasks. However, the growth of state-of-the-art mod-els makes traditional full fine-tuning unsuitable…
parameter-efficient fine-tuningMulti-Task LearningFourier Low-rank and Sparse Tensor for Efficient Tensor Completion
Tensor completion is crucial in many scientific domains with missing data problems. Traditional low-rank tensor models, including CP, Tucker, and Tensor-Train, exploit low-dimensional structures to recover missing data. …
Computational Efficiency