Neural Grapheme-to-Phoneme Conversion with Pre-trained Grapheme Models
Neural network models have achieved state-of-the-art performance on grapheme-to-phoneme (G2P) conversion. However, their performance relies on large-scale pronunciation dictionaries, which may not be available for a lot of languages. Inspired by the success of the pre-trained language model BERT, this paper proposes a pre-trained grapheme model called grapheme BERT (GBERT), which is built by self-supervised training on a large, language-specific word list with only grapheme information. Furthermore, two approaches are developed to incorporate GBERT into the state-of-the-art Transformer-based G2P model, i.e., fine-tuning GBERT or fusing GBERT into the Transformer model by attention. Experimental results on the Dutch, Serbo-Croatian, Bulgarian and Korean datasets of the SIGMORPHON 2021 G2P task confirm the effectiveness of our GBERT-based G2P models under both medium-resource and low-resource data conditions.
Code (1)
Tasks
Grapheme-to-Phoneme ConversionLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
An Investigation of the Relation Between Grapheme Embeddings and Pronunciation for Tacotron-based Systems
End-to-end models, particularly Tacotron-based ones, are currently a popular solution for text-to-speech synthesis. They allow the production of high-quality synthesized speech with little to no text preprocessing. Indee…
Grapheme-to-Phoneme ConversionRelationSpeech Synthesistext-to-speech+2Improving Grapheme-to-Phoneme Conversion through In-Context Knowledge Retrieval with Large Language Models
Grapheme-to-phoneme (G2P) conversion is a crucial step in Text-to-Speech (TTS) systems, responsible for mapping grapheme to corresponding phonetic representations. However, it faces ambiguities problems where the same gr…
Grapheme-to-Phoneme ConversionRetrievaltext-to-speechText to SpeechThe SIGMORPHON 2020 Shared Task on Multilingual Grapheme-to-Phoneme Conversion
We describe the design and findings of the SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion. Participants were asked to submit systems which take in a sequence of graphemes in a given language a…
Grapheme-to-Phoneme ConversionGrapheme-to-Phoneme Conversion with a Multilingual Transformer Model
In this paper, we describe our three submissions to the SIGMORPHON 2020 shared task 1 on grapheme-to-phoneme conversion for 15 languages. We experimented with a single multilingual transformer model. We observed that the…
Grapheme-to-Phoneme ConversionResults of the Second SIGMORPHON Shared Task on Multilingual Grapheme-to-Phoneme Conversion
Grapheme-to-phoneme conversion is an important component in many speech technologies, but until recently there were no multilingual benchmarks for this task. The second iteration of the SIGMORPHON shared task on multilin…
Grapheme-to-Phoneme Conversion