Natural Language Generation at Scale: A Case Study for Open Domain Question Answering
Current approaches to Natural Language Generation (NLG) for dialog mainly focus on domain-specific, task-oriented applications (e.g. restaurant booking) using limited ontologies (up to 20 slot types), usually without considering the previous conversation context. Furthermore, these approaches require large amounts of data for each domain, and do not benefit from examples that may be available for other domains. This work explores the feasibility of applying statistical NLG to scenarios requiring larger ontologies, such as multi-domain dialog applications or open-domain question answering (QA) based on knowledge graphs. We model NLG through an Encoder-Decoder framework using a large dataset of interactions between real-world users and a conversational agent for open-domain QA. First, we investigate the impact of increasing the number of slot types on the generation quality and experiment with different partitions of the QA data with progressively larger ontologies (up to 369 slot types). Second, we perform multi-task learning experiments between open-domain QA and task-oriented dialog, and benchmark our model on a popular NLG dataset. Moreover, we experiment with using the conversational context as an additional input to improve response generation quality. Our experiments show the feasibility of learning statistical NLG models for open-domain QA with larger ontologies.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderKnowledge GraphsMulti-Task LearningOpen-Domain Question AnsweringQuestion AnsweringResponse GenerationText GenerationSimilar Papers 제목 키워드 기반
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk
Language is the principal tool for human communication, in which humor is one of the most attractive parts. Producing natural language like humans using computers, a.k.a, Natural Language Generation (NLG), has been widel…
BenchmarkingMachine TranslationText GenerationLarge-Scale Transfer Learning for Natural Language Generation
Large-scale pretrained language models define state of the art in natural language processing, achieving outstanding performance on a variety of tasks. We study how these architectures can be applied and adapted for natu…
Open-Domain DialogText GenerationTransfer LearningScaled Prompt-Tuning for Few-Shot Natural Language Generation
The increasingly Large Language Models (LLMs) demonstrate stronger language understanding and generation capabilities, while the memory demand and computation cost of fine-tuning LLMs on downstream tasks are non-negligib…
parameter-efficient fine-tuningText GenerationSay the Right Thing Right: Ethics Issues in Natural Language Generation Systems
We discuss the ethical implications of Natural Language Generation systems. We use one particular system as a case study to identify and classify issues, and we provide an ethics checklist, in the hope that future system…
EthicsText GenerationCode Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code
Few-shot learning with large-scale, pre-trained language models is a powerful way to answer questions about code, e.g., how to complete a given code example, or even generate code snippets from scratch. The success of th…
Code GenerationFew-Shot LearningLanguage ModelingLanguage Modelling