Schema Key Wording as an Instruction Channel in Structured Generation under Constrained Decoding
Constrained decoding is widely used to make large language models produce structured outputs that satisfy schemas such as JSON. Existing work mainly treats schemas as structural constraints, overlooking that schema-key tokens also enter the autoregressive context and may guide generation. To the best of our knowledge, we present the first systematic study of schema keys as an implicit instruction channel under constrained decoding. We formulate structured generation as a multi-channel instruction problem, where task signals can be placed in prompts, schema keys, or both. We further provide a projection-aware analysis: a CoT-style key helps only when its semantic gain exceeds the distortion induced by grammar-constrained projection, offering a theoretical explanation for model-dependent key effects. Experiments on mathematical reasoning benchmarks show that changing only schema-key wording can substantially affect accuracy while keeping the prompt, model, output structure, and decoding setup fixed. Qwen models tend to benefit more from schema-level instructions, whereas LLaMA models rely more on prompt-level guidance, and the two channels interact non-additively. Our findings show that schema design is not merely output formatting, but part of instruction specification in structured generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Mathematical ReasoningSimilar Papers 제목 키워드 기반
SBI-RAG: Enhancing Math Word Problem Solving for Students through Schema-Based Instruction and Retrieval-Augmented Generation
Many students struggle with math word problems (MWPs), often finding it difficult to identify key information and select the appropriate mathematical operations. Schema-based instruction (SBI) is an evidence-based strate…
GSM8KLanguage ModelingLanguage ModellingLarge Language Model+5KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction
In this paper, we propose KnowCoder, a Large Language Model (LLM) to conduct Universal Information Extraction (UIE) via code generation. KnowCoder aims to develop a kind of unified schema representation that LLMs can eas…
Code GenerationLanguage ModellingLarge Language ModelUIEESAinsTOD: A Unified End-to-End Schema-Aware Instruction-Tuning Framework for Task-Oriented Dialog Modeling
Existing end-to-end modeling methods for modular task-oriented dialog systems are typically tailored to specific datasets, making it challenging to adapt to new dialog scenarios. In this work, we propose ESAinsTOD, a uni…
Induce, Edit, Retrieve:Language Grounded Multimodal Schema for Instructional Video Retrieval
Schemata are structured representations of complex tasks that can aid artificial intelligence by allowing models to break down complex tasks into intermediate steps. We propose a novel system that induces schemata from w…
RetrievalVideo RetrievalAutomated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks
System Instructions in Large Language Models (LLMs) are commonly used to enforce safety policies, define agent behavior, and protect sensitive operational context in agentic AI applications. These instructions may contai…