EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training
Large-scale pre-training has shown remarkable performance in building open-domain dialogue systems. However, previous works mainly focus on showing and evaluating the conversational performance of the released dialogue model, ignoring the discussion of some key factors towards a powerful human-like chatbot, especially in Chinese scenarios. In this paper, we conduct extensive experiments to investigate these under-explored factors, including data quality control, model architecture designs, training approaches, and decoding strategies. We propose EVA2.0, a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters, and will make our models and codes publicly available. Automatic and human evaluations show that EVA2.0 significantly outperforms other open-source counterparts. We also discuss the limitations of this work by presenting some failure cases and pose some future research directions on large-scale Chinese open-domain dialogue systems.
Code (2)
Tasks
ChatbotSimilar Papers 제목 키워드 기반
EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with Engli…
CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Open-Domain Conversation
Recently, the personification and empathy capabilities of dialogue systems have received extensive attention from researchers. Although it is straightforward for humans to express themselves personally and empathically, …
ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human
In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. Different from…
World KnowledgeBuilding Large Chinese Corpus for Spoken Dialogue Research in Specific Domains
Corpus is a valuable resource for information retrieval and data-driven natural language processing systems,especially for spoken dialogue research in specific domains. However,there is little non-English corpora, partic…
Information RetrievalRetrievalSentenceCrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Dataset
To advance multi-domain (cross-domain) dialogue modeling as well as alleviate the shortage of Chinese task-oriented datasets, we propose CrossWOZ, the first large-scale Chinese Cross-Domain Wizard-of-Oz task-oriented dat…
Dialogue State TrackingTask-Oriented Dialogue SystemsUser Simulation