Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System
Pre-trained language models have been recently shown to benefit task-oriented dialogue (TOD) systems. Despite their success, existing methods often formulate this task as a cascaded generation problem which can lead to error accumulation across different sub-tasks and greater data annotation overhead. In this study, we present PPTOD, a unified plug-and-play model for task-oriented dialogue. In addition, we introduce a new dialogue multi-task pre-training strategy that allows the model to learn the primary TOD task completion skills from heterogeneous dialog corpora. We extensively test our model on three benchmark TOD tasks, including end-to-end dialogue modelling, dialogue state tracking, and intent classification. Experimental results show that PPTOD achieves new state of the art on all evaluated tasks in both high-resource and low-resource scenarios. Furthermore, comparisons against previous SOTA methods show that the responses generated by PPTOD are more factually correct and semantically coherent as judged by human annotators.
Code (2)
Tasks
Dialogue State TrackingEnd-To-End Dialogue Modellingintent-classificationIntent ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Plug-and-Play Document Modules for Pre-trained Models
Large-scale pre-trained models (PTMs) have been widely used in document-oriented NLP tasks, such as question answering. However, the encoding-task coupling requirement results in the repeated encoding of the same documen…
Question AnsweringPnP-ReG: Learned Regularizing Gradient for Plug-and-Play Gradient Descent
The Plug-and-Play (PnP) framework makes it possible to integrate advanced image denoising priors into optimization algorithms, to efficiently solve a variety of image restoration tasks generally formulated as Maximum A P…
DenoisingImage DenoisingImage RestorationInvestigating the Feasibility of Patch-based Inference for Generalized Diffusion Priors in Inverse Problems for Medical Images
Plug-and-play approaches to solving inverse problems such as restoration and super-resolution have recently benefited from Diffusion-based generative priors for natural as well as medical images. However, solutions often…
Super-ResolutionDirected Beam Search: Plug-and-Play Lexically Constrained Language Generation
Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contains specific words, is a challenging prob…
Language ModelingLanguage ModellingMachine TranslationStory Generation+2Deep Plug-and-Play Prior for Multitask Channel Reconstruction in Massive MIMO Systems
Scalability is a major concern in implementing deep learning (DL) based methods in wireless communication systems. Given various channel reconstruction tasks, applying one DL model for one specific task is costly in both…
Multi-Task Learning