English to Hindi Multi-modal Neural Machine Translation and Hindi Image Captioning

2019-11-01 · WS 2019 11 · Sahinur Rahman Laskar, Rohit Pratap Singh, Partha Pakray, B, Sivaji yopadhyay

With the widespread use of Machine Trans-lation (MT) techniques, attempt to minimizecommunication gap among people from di-verse linguistic backgrounds. We have par-ticipated in Workshop on Asian Transla-tion 2019 (WAT2019) multi-modal translationtask. There are three types of submissiontrack namely, multi-modal translation, Hindi-only image captioning and text-only transla-tion for English to Hindi translation. The mainchallenge is to provide a precise MT output.The multi-modal concept incorporates textualand visual features in the translation task. Inthis work, multi-modal translation track re-lies on pre-trained convolutional neural net-works (CNN) with Visual Geometry Grouphaving 19 layered (VGG19) to extract imagefeatures and attention-based Neural MachineTranslation (NMT) system for translation.The merge-model of recurrent neural network(RNN) and CNN is used for the Hindi-onlyimage captioning. The text-only translationtrack is based on the transformer model of theNMT system. The official results evaluated atWAT2019 translation task, which shows thatour multi-modal NMT system achieved Bilin-gual Evaluation Understudy (BLEU) score20.37, Rank-based Intuitive Bilingual Eval-uation Score (RIBES) 0.642838, Adequacy-Fluency Metrics (AMFM) score 0.668260 forchallenge test data and BLEU score 40.55,RIBES 0.760080, AMFM score 0.770860 forevaluation test data in English to Hindi multi-modal translation respectively.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Hindi Image Captioning Image Captioning Machine TranslationNMTTranslation

Methods 이 논문이 사용한 방법론

Test 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Hindi Visual Genome: A Dataset for Multimodal English-to-Hindi Machine Translation

2019-07-21 · Shantipriya Parida, Ondřej Bojar, Satya Ranjan Dash

Visual Genome is a dataset connecting structured image information with English language. We present ``Hindi Visual Genome'', a multimodal dataset consisting of text and images suitable for English-Hindi multimodal machi…

Machine Translation Multimodal Machine Translation Translation

Low Resource Multimodal Neural Machine Translation of English-Hindi in News Domain

2021-09-01 · MMTLRL (RANLP) 2021 9 · Loitongbam Sanayai Meetei, Thoudam Doren Singh, Sivaji Bandyopadhyay

Incorporating multiple input modalities in a machine translation (MT) system is gaining popularity among MT researchers. Unlike the publicly available dataset for Multimodal Machine Translation (MMT) tasks, where the cap…

Machine Translation Multimodal Machine Translation NMT Translation

Investigation of English to Hindi Multimodal Neural Machine Translation using Transliteration-based Phrase Pairs Augmentation

2022-10-01 · WAT 2022 10 · Sahinur Rahman Laskar, Rahul Singh, Md Faizal Karim, Riyanka Manna 외

Machine translation translates one natural language to another, a well-defined natural language processing task. Neural machine translation (NMT) is a widely accepted machine translation approach, but it requires a suffi…

Machine Translation NMT Translation Transliteration

Improved English to Hindi Multimodal Neural Machine Translation

2021-08-01 · ACL (WAT) 2021 8 · Sahinur Rahman Laskar, Abdullah Faiz Ur Rahman Khilji, Darsh Kaushik, Partha Pakray 외

Machine translation performs automatic translation from one natural language to another. Neural machine translation attains a state-of-the-art approach in machine translation, but it requires adequate training data, whic…

Data Augmentation Machine Translation NMT Translation

Multimodal Neural Machine Translation for English to Hindi

2020-12-01 · AACL (WAT) 2020 12 · Sahinur Rahman Laskar, Abdullah Faiz Ur Rahman Khilji, Partha Pakray, Sivaji Bandyopadhyay

Machine translation (MT) focuses on the automatic translation of text from one natural language to another natural language. Neural machine translation (NMT) achieves state-of-the-art results in the task of machine trans…

Machine Translation NMT Translation