SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation

2026-03-17 · Long Li, Zhijian Zhou, Jiangxuan Long, Peiyang Liu, Weidi Xu, Zhe Wang, Shirui Pan, Chao Qu arxiv

Agentic Reinforcement Learning (RL) shows promise for complex tasks, but Text-to-SQL remains mostly restricted to single-turn paradigms. A primary bottleneck is the credit assignment problem. In traditional paradigms, rewards are determined solely by the final-turn feedback, which ignores the intermediate process and leads to ambiguous credit evaluation. To address this, we propose Agentic SQL, a framework featuring a universal two-tiered reward mechanism designed to provide effective trajectory-level evaluation and dense step-level signals. First, we introduce Aggregated Trajectory Reward (ATR) to resolve multi-turn credit assignment. Using an asymmetric transition matrix, ATR aggregates process-oriented scores to incentivize continuous improvement. Leveraging Lyapunov stability theory, we prove ATR acts as an energy dissipation operator, guaranteeing a cycle-free policy and monotonic convergence. Second, Column-Set Matching Reward (CSMR) provides immediate step-level rewards to mitigate sparsity. By executing queries at each turn, CSMR converts binary (0/1) feedback into dense [0, 1] signals based on partial correctness. Evaluations on BIRD show a 5% gain over binary-reward GRPO. Notably, our approach outperforms SOTA Arctic-Text2SQL-R1-7B on BIRD and Spider 2.0 using identical models, propelling Text-to-SQL toward a robust multi-turn agent paradigm.

📄 PDF Abstract BibTeX arXiv:2603.16161

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs

2026-05-15 · Haizhong Zheng, Yizhuo Di, Jiahui Wang, Shuowei Jin 외 arxiv

Reinforcement learning (RL) is increasingly used to improve the reasoning, coding, and tool-use capabilities of large language models, but agentic RL remains prohibitively expensive. Scaling RL to agentic LLMs requires s…

Reinforcement Learning

Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators

2026-06-04 · Chenming Zhu, Jingli Lin, Yilin Long, Peizhou Cao 외 arxiv

While Vision-Language Models (VLMs) have shown strong visual reasoning capabilities, their spatial reasoning abilities remain largely constrained to the observed images and text-oriented chain-of-thought. They often stru…

Spatial Reasoning Visual Reasoning

astra-langchain4j: Experiences Combining LLMs and Agent Programming

2026-01-29 · Rem Collier, Katharine Beaumont, Andrei Ciortea arxiv

Given the emergence of Generative AI over the last two years and the increasing focus on Agentic AI as a form of Multi-Agent System it is important to explore both how such technologies can impact the use of traditional …

ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas

2026-01-29 · Xiaoyu Tian, Haotian Wang, Shuaiting Chen, Hao Zhou 외 arxiv

Large language models (LLMs) are increasingly used as tool-augmented agents for multi-step decision making, yet training robust tool-using agents remains challenging. Existing methods still require manual intervention, d…

Reinforcement Learning Decision Making

Astraea: A State-Aware Scheduling Engine for LLM-Powered Agents

2025-12-16 · Hongqiu Ni, Jiabao Zhang, Guopeng Li, Zilong Wang 외 arxiv

Large Language Models (LLMs) are increasingly being deployed as intelligent agents. Their multi-stage workflows, which alternate between local computation and calls to external network services like Web APIs, introduce a…