Smoothing parameter estimation framework for IBM word alignment models
IBM models are very important word alignment models in Machine Translation. Following the Maximum Likelihood Estimation principle to estimate their parameters, the models will easily overfit the training data when the data are sparse. While smoothing is a very popular solution in Language Model, there still lacks studies on smoothing for word alignment. In this paper, we propose a framework which generalizes the notable work Moore [2004] of applying additive smoothing to word alignment models. The framework allows developers to customize the smoothing amount for each pair of word. The added amount will be scaled appropriately by a common factor which reflects how much the framework trusts the adding strategy according to the performance on data. We also carefully examine various performance criteria and propose a smoothened version of the error count, which generally gives the best result.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine Translationparameter estimationTranslationWord AlignmentSimilar Papers 제목 키워드 기반
Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR
In this paper, alternating weak triphone/BPE alignment supervision is proposed to improve end-to-end model training. Towards this end, triphone and BPE alignments are extracted using a pre-existing hybrid ASR system. The…
Unifying Input and Output Smoothing in Neural Machine Translation
Soft contextualized data augmentation is a recent method that replaces one-hot representation of words with soft posterior distributions of an external language model, smoothing the input of neural machine translation sy…
Data AugmentationLanguage ModelingLanguage ModellingMachine Translation+1PeakFocus: Bridging Peak Localization and Intensity Regression via a Unified Multi-Scale Framework for Electricity Load Forecasting
Electricity load peak forecasting (ELPF), simultaneously predicting peak timing and intensity, is a prerequisite for effective grid scheduling and risk management. However, existing methods face three limitations. First,…
Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation
As machine learning models and datasets continue to grow, developing complex models has become increasingly computationally demanding. Knowledge distillation reduces deployment cost by compressing a large, well-trained t…
Knowledge DistillationDensity EstimationTransfer Learning on Edge Connecting Probability Estimation under Graphon Model
Graphon models provide a flexible nonparametric framework for estimating latent connectivity probabilities in networks, enabling a range of downstream applications such as link prediction and data augmentation. However, …
Graph ClassificationTransfer LearningData AugmentationLink Prediction