Syntax-driven Iterative Expansion Language Models for Controllable Text Generation
The dominant language modeling paradigm handles text as a sequence of discrete tokens. While that approach can capture the latent structure of the text, it is inherently constrained to sequential dynamics for text generation. We propose a new paradigm for introducing a syntactic inductive bias into neural text generation, where the dependency parse tree is used to drive the Transformer model to generate sentences iteratively. Our experiments show that this paradigm is effective at text generation, with quality between LSTMs and Transformers, and comparable diversity, requiring less than half their decoding steps, and its generation process allows direct control over the syntactic constructions of the generated text, enabling the induction of stylistic variations.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityInductive BiasLanguage ModelingLanguage ModellingText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Controllability Test for Nonlinear Datatic Systems
Controllability is a fundamental property of control systems, serving as the prerequisite for controller design. While controllability test is well established in modelic (i.e., model-driven) control systems, extending i…
DanceCrafter: Fine-Grained Text-Driven Controllable Dance Generation via Choreographic Syntax
Text-driven controllable dance generation remains under-explored, primarily due to the severe scarcity of high-quality datasets and the inherent difficulty of articulating complex choreographies. Characterizing dance is …
ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization
Large Language Models (LLMs) employ three popular training approaches: Masked Language Models (MLM), Causal Language Models (CLM), and Sequence-to-Sequence Models (seq2seq). However, each approach has its strengths and l…
InteracSPARQL: An Interactive System for SPARQL Query Refinement Using Natural Language Explanations
In recent years, querying semantic web data using SPARQL has remained challenging, especially for non-expert users, due to the language's complex syntax and the prerequisite of understanding intricate data structures. To…
ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback
LLM-based query expansion improves retrieval by enriching the original query with additional context. Yet most methods remain generation-driven, producing plausible pseudo-documents or expansions without checking how the…