Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems
In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding method on subjective dialogue evaluations with overall metrics, they did not analyze how the differences of fine-tuning datasets affect on user's detailed impression. In addition, the Transformer-based approach has only been verified for English, not for such languages with large inter-language distances as Japanese. In this study, we develop large-scale Transformer-based Japanese dialogue models and Japanese chit-chat datasets to examine the effectiveness of the Transformer-based approach for building chit-chat dialogue systems. We evaluated and analyzed the impressions of human dialogues in different fine-tuning datasets, model parameters, and the use of additional information.
Code (1)
Tasks
DecoderMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems
In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding …
DecoderIs Japanese CCGBank empirically correct? A case study of passive and causative constructions
The Japanese CCGBank serves as training and evaluation data for developing Japanese CCG parsers. However, since it is automatically generated from the Kyoto Corpus, a dependency treebank, its linguistic validity still ne…
Semantic ParsingXiaomi's Submissions for IWSLT 2020 Open Domain Translation Task
This paper describes the Xiaomi{'}s submissions to the IWSLT20 shared open domain translation task for Chinese{\textless}-{\textgreater}Japanese language pair. We explore different model ensembling strategies based on re…
Domain AdaptationKnowledge DistillationRe-RankingTranslationJCSE: Contrastive Learning of Japanese Sentence Embeddings and Its Applications
Contrastive learning is widely used for sentence representation learning. Despite this prevalence, most studies have focused exclusively on English and few concern domain adaptation for domain-specific downstream tasks, …
Contrastive LearningDomain AdaptationInformation RetrievalLanguage Modelling+7Linguistically-driven Multi-task Pre-training for Low-resource Neural Machine Translation
In the present study, we propose novel sequence-to-sequence pre-training objectives for low-resource machine translation (NMT): Japanese-specific sequence to sequence (JASS) for language pairs involving Japanese as the s…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationNMT+1