Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue
In task-oriented dialogues, Natural Language Generation (NLG) is the final yet crucial step to produce user-facing system utterances. The result of NLG is directly related to the perceived quality and usability of a dialogue system. While most existing systems provide semantically correct responses given goals to present, they struggle to match the variation and fluency in the human language. In this paper, we propose a novel multi-task learning framework, NLG-LM, for natural language generation. In addition to generating high-quality responses conveying the required information, it also explicitly targets for naturalness in generated responses via an unconditioned language model. This can significantly improve the learning of style and variation in human language. Empirical results show that this multi-task learning framework outperforms previous models across multiple datasets. For example, it improves the previous best BLEU score on the E2E-NLG dataset by 2.2{\%}, and on the Laptop dataset by 6.1{\%}.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMulti-Task LearningText GenerationSimilar Papers 제목 키워드 기반
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP). These benchmarks mostly focus on a range of Natural Language Understa…
Natural Language UnderstandingText GenerationTransfer LearningMVP: Multi-task Supervised Pre-training for Natural Language Generation
Pre-trained language models (PLMs) have achieved remarkable success in natural language generation (NLG) tasks. Up to now, most NLG-oriented PLMs are pre-trained in an unsupervised manner using the large-scale general co…
Text GenerationAdaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning
When a natural language generation (NLG) component is implemented in a real-world task-oriented dialogue system, it is necessary to generate not only natural utterances as learned on training data but also utterances ada…
Natural Language Understandingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+3Multi-modal embeddings using multi-task learning for emotion recognition
General embeddings like word2vec, GloVe and ELMo have shown a lot of success in natural language tasks. The embeddings are typically extracted from models that are built on general tasks such as skip-gram models and natu…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion RecognitionMulti-Task Learning+5DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders
While pretrained encoders have achieved success in various natural language understanding (NLU) tasks, there is a gap between these pretrained encoders and natural language generation (NLG). NLG tasks are often based on …
Abstractive Text SummarizationDecoderMachine TranslationNatural Language Understanding+5