An Effective Method using Phrase Mechanism in Neural Machine Translation

2023-08-21 · Phuong Minh Nguyen, Le Minh Nguyen

Machine Translation is one of the essential tasks in Natural Language Processing (NLP), which has massive applications in real life as well as contributing to other tasks in the NLP research community. Recently, Transformer -based methods have attracted numerous researchers in this domain and achieved state-of-the-art results in most of the pair languages. In this paper, we report an effective method using a phrase mechanism, PhraseTransformer, to improve the strong baseline model Transformer in constructing a Neural Machine Translation (NMT) system for parallel corpora Vietnamese-Chinese. Our experiments on the MT dataset of the VLSP 2022 competition achieved the BLEU score of 35.3 on Vietnamese to Chinese and 33.2 BLEU scores on Chinese to Vietnamese data. Our code is available at https://github.com/phuongnm94/PhraseTransformer.

📄 PDF Abstract BibTeX arXiv:2308.10482

Code (1)

phuongnm94/PhraseTransformer 공식 구현 pytorch

Tasks

Machine TranslationNMTTranslation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Position-Wise Feed-Forward Layer 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Adam 설명 없음

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Learning Source Phrase Representations for Neural Machine Translation

2020-06-25 · ACL 2020 6 · Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu 외

The Transformer translation model (Vaswani et al., 2017) based on a multi-head attention mechanism can be computed effectively in parallel and has significantly pushed forward the performance of Neural Machine Translatio…

Machine Translation NMT Translation

Neural Phrase-to-Phrase Machine Translation

2018-11-06 · Jiangtao Feng, Lingpeng Kong, Po-Sen Huang, Chong Wang 외

In this paper, we propose Neural Phrase-to-Phrase Machine Translation (NP$^2$MT). Our model uses a phrase attention mechanism to discover relevant input (source) segments that are used by a decoder to generate output (ta…

Decoder Machine Translation Translation

Neural Machine Translation with External Phrase Memory

2016-06-06 · Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li 외

In this paper, we propose phraseNet, a neural machine translator with a phrase memory which stores phrase pairs in symbolic form, mined from corpus or specified by human experts. For any given source sentence, phraseNet …

Decoder Machine Translation Sentence Translation

Towards Neural Phrase-based Machine Translation

2017-06-17 · ICLR 2018 1 · Po-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou 외

In this paper, we present Neural Phrase-based Machine Translation (NPMT). Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based se…

Machine Translation NMT Translation

Chinese-to-Japanese Patent Machine Translation based on Syntactic Pre-ordering for WAT 2016

2016-12-01 · WS 2016 12 · Katsuhito Sudoh, Masaaki Nagata

This paper presents our Chinese-to-Japanese patent machine translation system for WAT 2016 (Group ID: ntt) that uses syntactic pre-ordering over Chinese dependency structures. Chinese words are reordered by a learning-to…

Chinese Word Segmentation Dependency Parsing General Classification Learning-To-Rank+3