Observing the Learning Curve of NMT Systems With Regard to Linguistic Phenomena
In this paper we present our observations and evaluations by observing the linguistic performance of the system on several steps on the training process of various English-to-German Neural Machine Translation models. The linguistic performance is measured through a semi-automatic process using a test suite. Among several linguistic observations, we find that the translation quality of some linguistic categories decreased within the recorded iterations. Additionally, we notice some drops of the translation quality of certain categories when using a larger corpus.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
Expanding the Role of Affective Phenomena in Multimodal Interaction Research
In recent decades, the field of affective computing has made substantial progress in advancing the ability of AI systems to recognize and express affective phenomena, such as affect and emotions, during human-human and h…
multimodal interactionAmbiFC: Fact-Checking Ambiguous Claims with Evidence
Automated fact-checking systems verify claims against evidence to predict their veracity. In real-world scenarios, the retrieved evidence may not unambiguously support or refute the claim and yield conflicting but valid …
Claim VerificationEvidence SelectionFact CheckingSentence+3Holmes: A Benchmark to Assess the Linguistic Competence of Language Models
We introduce Holmes, a new benchmark designed to assess language models (LMs) linguistic competence - their unconscious understanding of linguistic phenomena. Specifically, we use classifier-based probing to examine LMs'…
Part-Of-Speech TaggingHow Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs
Analysing translation quality in regards to specific linguistic phenomena has historically been difficult and time-consuming. Neural machine translation has the attractive property that it can produce scores for arbitrar…
Machine TranslationNMTTranslationTransliterationSurvey of NLU Benchmarks Diagnosing Linguistic Phenomena: Why not Standardize Diagnostics Benchmarks?
Natural Language Understanding (NLU) is a basic task in Natural Language Processing (NLP). The evaluation of NLU capabilities has become a trending research topic that attracts researchers in the last few years, resultin…
Natural Language Understanding