Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation

2025-09-18 · Haoran Zhang, Yafu Li, Xuyang Hu, Dongrui Liu, Zhilin Wang, Bo Li, Yu Cheng arxiv

Large language models (LLMs) are increasingly applied in diverse real-world scenarios, each governed by bespoke behavioral and safety specifications (spec) custom-tailored by users or organizations. These spec, categorized into safety-spec and behavioral-spec, vary across scenarios and evolve with changing preferences and requirements. We formalize this challenge as specification alignment, focusing on LLMs' ability to follow dynamic, scenario-specific spec from both behavioral and safety perspectives. To address this challenge, we propose Align3, a lightweight method that employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over the specification boundaries. We further present SpecBench, a unified benchmark for measuring specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts. Experiments on 15 reasoning and 18 instruct models with several TTD methods, including Self-Refine, TPO, and MoreThink, yield three key findings: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps. These results highlight the potential of test-time deliberation as an effective strategy for reasoning over the real-world specification boundaries. Our code and resources are available at https://github.com/zzzhr97/SpecBench.

📄 PDF Abstract BibTeX arXiv:2509.14760

Code (0)

등록된 구현이 없습니다.

Local Success Does Not Compose: Benchmarking Large Language Models for Compositional Formal Verification

2025-09-27 · Xu Xu, Xin Li, Xingwei Qu, Jie Fu 외 arxiv

We introduce DafnyCOMP, a benchmark for evaluating large language models (LLMs) on compositional specification generation in Dafny. Unlike prior benchmarks that focus on single-function tasks, DafnyCOMP targets programs …

Code Generation

R2IF: Aligning Reasoning with Decisions via Composite Rewards for Interpretable LLM Function Calling

2026-04-22 · Aijia Cheng, Kailong Wang, Ling Shi, Yongxin Zhao arxiv

Function calling empowers large language models (LLMs) to interface with external tools, yet existing RL-based approaches suffer from misalignment between reasoning processes and tool-call decisions. We propose R2IF, a r…

Not My Agent, Not My Boundary? Elicitation of Personal Privacy Boundaries in AI-Delegated Information Sharing

2025-09-26 · Bingcan Guo, Eryue Xu, Zhiping Zhang, Tianshi Li arxiv

Aligning AI systems with human privacy preferences requires understanding individuals' nuanced disclosure behaviors beyond general norms. Yet eliciting such boundaries remains challenging due to the context-dependent nat…

Deliberative Alignment: Reasoning Enables Safer Language Models

2024-12-20 · Melody Y. Guan, Manas Joglekar, Eric Wallace, Saachi Jain 외

As large-scale language models increasingly impact safety-critical domains, ensuring their reliable adherence to well-defined principles remains a fundamental challenge. We introduce Deliberative Alignment, a new paradig…

Out-of-Distribution Generalization

SkillSpec: Intent-Masked Specification Reasoning for Agent Skill Correctness

2026-09-05 · Yizhuo Zhang, Bo Kang, Yi Yang, Zhiyu Duan 외 hf

Autonomous agent systems increasingly depend on reusable skill abstractions for consolidating experiential knowledge and domain expertise. These artifacts typically bundle free-form instructions with heterogeneous resour…