Controlled Data Generation via Insertion Operations for NLU
Use of synthetic data is rapidly emerging as a realistic alternative to manually annotating live traffic for industry-scale model building. Manual data annotation is slow, expensive and not preferred for meeting customer privacy expectations. Further, commercial natural language applications are required to support continuously evolving features as well as newly added experiences. To address these requirements, we propose a targeted synthetic data generation technique by inserting tokens into a given semantic signature. The generated data are used as additional training samples in the tasks of intent classification and named entity recognition. We evaluate on a real-world voice assistant dataset, and using only 33% of the available training set, we achieve the same accuracy as training with all available data. Further, we analyze the effects of data generation across varied real-world applications and propose heuristics that improve the task performance further.
Code (0)
등록된 구현이 없습니다.
Tasks
intent-classificationIntent Classificationnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Synthetic Data GenerationSimilar Papers 제목 키워드 기반
Insertion Transformer: Flexible Sequence Generation via Insertion Operations
We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations. Unlike typical autoregressive models which rely on a fixed, often left-to-right or…
Machine TranslationTranslationvalidEnhancing LLM Character-Level Manipulation via Divide and Conquer
Large Language Models (LLMs) have demonstrated strong generalization capabilities across a wide range of natural language processing (NLP) tasks. However, they exhibit notable weaknesses in character-level string manipul…
Code GenerationInsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model
We propose InsNet, an expressive insertion-based text generator with efficient training and flexible decoding (parallel or sequential). Unlike most existing insertion-based text generation works that require re-encoding …
Machine TranslationStory GenerationText GenerationTranslationInsertion-based Decoding with automatically Inferred Generation Order
Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal. In this work, we propose a novel decoding algorithm -- InDIGO -- which supports flexible sequ…
Code GenerationMachine TranslationTranslationBeyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
While Masked Diffusion Language Models (MDLMs) relying on token masking and unmasking have shown promise in language modeling, their computational efficiency and generation flexibility remain constrained by the masking p…
Computational Efficiency