Predicting Actions to Help Predict Translations
We address the task of text translation on the How2 dataset using a state of the art transformer-based multimodal approach. The question we ask ourselves is whether visual features can support the translation process, in particular, given that this is a dataset extracted from videos, we focus on the translation of actions, which we believe are poorly captured in current static image-text datasets currently used for multimodal translation. For that purpose, we extract different types of action features from the videos and carefully investigate how helpful this visual information is by testing whether it can increase translation quality when used in conjunction with (i) the original text and (ii) the original text where action-related words (or all verbs) are masked out. The latter is a simulation that helps us assess the utility of the image in cases where the text does not provide enough context about the action, or in the presence of noise in the input text.
Code (0)
등록된 구현이 없습니다.
Tasks
TranslationSimilar Papers 제목 키워드 기반
Faithful Target Attribute Prediction in Neural Machine Translation
The training data used in NMT is rarely controlled with respect to specific attributes, such as word casing or gender, which can cause errors in translations. We argue that predicting the target word and attributes simul…
AttributeData AugmentationGender PredictionMachine Translation+3Detecting Non-literal Translations by Fine-tuning Cross-lingual Pre-trained Language Models
Human-generated non-literal translations reflect the richness of human languages and are sometimes indispensable to ensure adequacy and fluency. Non-literal translations are difficult to produce even for human translator…
Language ModellingSentenceTranslationPredicting the Compositionality of Multiword Expressions Using Translations in Multiple Languages
Neural Machine Translation with Key-Value Memory-Augmented Attention
Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a nov…
DecoderMachine TranslationNMTSentence+1UAlberta at SemEval-2020 Task 2: Using Translations to Predict Cross-Lingual Entailment
We investigate the hypothesis that translations can be used to identify cross-lingual lexical entailment. We propose novel methods that leverage parallel corpora, word embeddings, and multilingual lexical resources. Our …
Lexical EntailmentTask 2Word Embeddings