Conditional set generation using Seq2seq models
Conditional set generation learns a mapping from an input sequence of tokens to a set. Several NLP tasks, such as entity typing and dialogue emotion tagging, are instances of set generation. Seq2Seq models, a popular choice for set generation, treat a set as a sequence and do not fully leverage its key properties, namely order-invariance and cardinality. We propose a novel algorithm for effectively sampling informative orders over the combinatorial space of label orders. We jointly model the set cardinality and output by prepending the set size and taking advantage of the autoregressive factorization used by Seq2Seq models. Our method is a model-independent data augmentation approach that endows any Seq2Seq model with the signals of order-invariance and cardinality. Training a Seq2Seq model on this augmented data (without any additional annotations) gets an average relative improvement of 20% on four benchmark datasets across various models: BART, T5, and GPT-3. Code to use SETAUG available at: https://setgen.structgen.com.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationEntity TypingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unraveling the Connections between Flow Matching and Diffusion Probabilistic Models in Training-free Conditional Generation
Training-free conditional generation aims to leverage the unconditional diffusion models to implement the conditional generation, where flow-matching (FM) and diffusion probabilistic models (DPMs) are two mature uncondit…
Unconditional Priors Matter! Improving Conditional Generation of Fine-Tuned Diffusion Models
Classifier-Free Guidance (CFG) is a fundamental technique in training conditional diffusion models. The common practice for CFG-based training is to use a single network to learn both conditional and unconditional noise …
Video GenerationTractable Transformers for Flexible Conditional Generation
Non-autoregressive (NAR) generative models are valuable because they can handle diverse conditional generation tasks in a more principled way than their autoregressive (AR) counterparts, which are constrained by sequenti…
DecoderReturn of Unconditional Generation: A Self-supervised Representation Generation Method
Unconditional generation -- the problem of modeling data distribution without relying on human-annotated labels -- is a long-standing and fundamental challenge in generative models, creating a potential of learning from …
Conditional Image GenerationImage GenerationUnconditional Image GenerationPre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders
Conditional Text Generation has drawn much attention as a topic of Natural Language Generation (NLG) which provides the possibility for humans to control the properties of generated contents. Current conditional generati…
Conditional Text GenerationDiversityText Generation