(Self-Attentive) Autoencoder-based Universal Language Representation for Machine Translation
Universal language representation is the holy grail in machine translation (MT). Thanks to the new neural MT approach, it seems that there are good perspectives towards this goal. In this paper, we propose a new architecture based on combining variational autoencoders with encoder-decoders and introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we are able to train multiple encoders and decoders for each language, sharing a common universal representation. Since the final objective of this universal representation is producing close results for similar input sentences (in any language), we propose to evaluate it by encoding the same sentence in two different languages, decoding both latent representations into the same language and comparing both outputs. Preliminary results on the WMT 2017 Turkish/English task shows that the proposed architecture is capable of learning a universal language representation and simultaneously training both translation directions with state-of-the-art results.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceTranslationSimilar Papers 제목 키워드 기반
LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders
In this work, we introduce long-video masked-embedding autoencoders (LV-MAE), a self-supervised learning framework for long video representation. Our approach treats short- and long-span dependencies as two separate task…
Self-Supervised LearningText RetrievalVideo-Text RetrievalUniversal Sound Separation with Self-Supervised Audio Masked Autoencoder
Universal sound separation (USS) is a task of separating mixtures of arbitrary sound sources. Typically, universal separation models are trained from scratch in a supervised manner, using labeled data. Self-supervised le…
Self-Supervised LearningAttentiveGRUAE: An Attention-Based GRU Autoencoder for Temporal Clustering and Behavioral Characterization of Depression from Wearable Data
In this study, we present AttentiveGRUAE, a novel attention-based gated recurrent unit (GRU) autoencoder designed for temporal clustering and prediction of outcome from longitudinal wearable data. Our model jointly optim…
Binary ClassificationPoint-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence
The rapid growth of Location-based Social Networks (LBSNs) provides a great opportunity to satisfy the strong demand for personalized Point-of-Interest (POI) recommendation services. However, with the tremendous increase…
DecoderRecommendation SystemsAttentive Symmetric Autoencoder for Brain MRI Segmentation
Self-supervised learning methods based on image patch reconstruction have witnessed great success in training auto-encoders, whose pre-trained weights can be transferred to fine-tune other downstream tasks of image under…
Image SegmentationMedical Image SegmentationMRI segmentationSegmentation+2