WGRAMMAR: Leverage Prior Knowledge to Accelerate Structured Decoding
Structured decoding enables large language models (LLMs) to generate outputs in formats required by downstream systems, such as HTML or JSON. However, existing methods suffer from efficiency bottlenecks due to grammar compilation, state tracking, and mask creation. We observe that many real-world tasks embed strong prior knowledge about output structure. Leveraging this, we propose a decomposition of constraints into static and dynamic components -- precompiling static structures offline and instantiating dynamic arguments at runtime using grammar snippets. Instead of relying on pushdown automata, we employ a compositional set of operators to model regular formats, achieving lower transition latency. We introduce wgrammar, a lightweight decoding engine that integrates domain-aware simplification, constraint decomposition, and mask caching, achieving up to 250x speedup over existing systems. wgrammar's source code is publicly available at https://github.com/wrran/wgrammar.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ASPiRe:Adaptive Skill Priors for Reinforcement Learning
We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that leverages prior experience to accelerate reinforcement learning. Unlike existing methods that learn a single skill prior from a large and diverse dat…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Parsing the Language of Expression: Enhancing Symbolic Regression with Domain-Aware Symbolic Priors
Symbolic regression is essential for deriving interpretable expressions that elucidate complex phenomena by exposing the underlying mathematical and physical relationships in data. In this paper, we present an advanced s…
regressionSymbolic RegressionLearning Hybrid Sparsity Prior for Image Restoration: Where Deep Learning Meets Sparse Coding
State-of-the-art approaches toward image restoration can be classified into model-based and learning-based. The former - best represented by sparse coding techniques - strive to exploit intrinsic prior knowledge about th…
Image RestorationCAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction
Undersampling k-space data in MRI reduces scan time but pose challenges in image reconstruction. Considerable progress has been made in reconstructing accelerated MRI. However, restoration of high-frequency image details…
Image EnhancementImage ReconstructionMRI ReconstructionKnow Before Fix: QA-Driven Repository Knowledge Acquisition for Software Issue Resolution
LLM-based coding agents have significantly advanced automated software issue resolution, yet they remain highly prone to factual errors caused by insufficient repository understanding. Recent methods attempt to mitigate …