paper-with-me

홈 › Papers

Towards Unsupervised Speech-to-Text Translation

2018-11-04 · Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass

We present a framework for building speech-to-text translation (ST) systems using only monolingual speech and text corpora, in other words, speech utterances from a source language and independent text from a target language. As opposed to traditional cascaded systems and end-to-end architectures, our system does not require any labeled data (i.e., transcribed source audio or parallel source and target text corpora) during training, making it especially applicable to language pairs with very few or even zero bilingual resources. The framework initializes the ST system with a cross-modal bilingual dictionary inferred from the monolingual corpora, that maps every source speech segment corresponding to a spoken word to its target text translation. For unseen source speech utterances, the system first performs word-by-word translation on each speech segment in the utterance. The translation is improved by leveraging a language model and a sequence denoising autoencoder to provide prior knowledge about the target language. Experimental results show that our unsupervised system achieves comparable BLEU scores to supervised end-to-end models despite the lack of supervision. We also provide an ablation analysis to examine the utility of each component in our system.

📄 PDF Abstract BibTeX arXiv:1811.01307

Code (0)

등록된 구현이 없습니다.

Tasks

DenoisingLanguage ModelingLanguage ModellingSpeech-to-TextSpeech-to-Text TranslationTranslationWord Translation

Methods 이 논문이 사용한 방법론

Denoising Autoencoder A Denoising Autoencoder is a modification on the autoencoder to prevent the network learning the identity function.…
Solana Customer Service Number +1-833-534-1729 설명 없음

Similar Papers 제목 키워드 기반

Simple and Effective Unsupervised Speech Translation

2022-10-18 · Changhan Wang, Hirofumi Inaguma, Peng-Jen Chen, Ilia Kulikov 외

The amount of labeled data to train models for speech tasks is limited for most languages, however, the data scarcity is exacerbated for speech translation which requires labeled data covering two different languages. To…

Domain AdaptationMachine Translationspeech-recognitionSpeech Recognition+6

Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces

2018-05-18 · NeurIPS 2018 12 · Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass

Recent research has shown that word embedding spaces learned from text corpora of different languages can be aligned without any parallel data supervision. Inspired by the success in unsupervised cross-lingual word embed…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Cross-Lingual Word Embeddingscross-modal alignment+6

Translatotron 3: Speech to Speech Translation with Monolingual Data

2023-05-27 · Eliya Nachmani, Alon Levkovitch, Yifan Ding, Chulayuth Asawaroengchai 외

This paper presents Translatotron 3, a novel approach to unsupervised direct speech-to-speech translation from monolingual speech-text datasets by combining masked autoencoder, unsupervised embedding mapping, and back-tr…

Speech-to-Speech TranslationTranslation

Towards speech-to-text translation without speech recognition

2017-02-13 · EACL 2017 4 · Sameer Bansal, Herman Kamper, Adam Lopez, Sharon Goldwater

We explore the problem of translating speech to text in low-resource scenarios where neither automatic speech recognition (ASR) nor machine translation (MT) are available, but we have training data in the form of audio p…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Machine Translationspeech-recognition+4

Spoken Term Discovery for Language Documentation using Translations

2017-09-01 · WS 2017 9 · Antonios Anastasopoulos, Sameer Bansal, David Chiang, Sharon Goldwater 외

Vast amounts of speech data collected for language documentation and research remain untranscribed and unsearchable, but often a small amount of speech may have text translations available. We present a method for partia…

Translation