Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL
The cross-database context-dependent Text-to-SQL (XDTS) problem has attracted considerable attention in recent years due to its wide range of potential applications. However, we identify two biases in existing datasets for XDTS: (1) a high proportion of context-independent questions and (2) a high proportion of easy SQL queries. These biases conceal the major challenges in XDTS to some extent. In this work, we present Chase, a large-scale and pragmatic Chinese dataset for XDTS. It consists of 5,459 coherent question sequences (17,940 questions with their SQL queries annotated) over 280 databases, in which only 35{\%} of questions are context-independent, and 28{\%} of SQL queries are easy. We experiment on Chase with three state-of-the-art XDTS approaches. The best approach only achieves an exact match accuracy of 40{\%} over all questions and 16{\%} over all question sequences, indicating that Chase highlights the challenging problems of XDTS. We believe that XDTS can provide fertile soil for addressing the problems.
Code (0)
등록된 구현이 없습니다.
Tasks
Text to SQLText-To-SQLSimilar Papers 제목 키워드 기반
SeSQL: Yet Another Large-scale Session-level Chinese Text-to-SQL Dataset
As the first session-level Chinese dataset, CHASE contains two separate parts, i.e., 2,003 sessions manually constructed from scratch (CHASE-C), and 3,456 sessions translated from English SParC (CHASE-T). We find the two…
SQL ParsingText to SQLText-To-SQLCATS: A Pragmatic Chinese Answer-to-Sequence Dataset with Large Scale and High Quality
There are three problems existing in the popular data-to-text datasets. First, the large-scale datasets either contain noise or lack real application scenarios. Second, the datasets close to real applications are relativ…
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset
Due to the lack of labeled data, previous research on text-to-SQL parsing mainly focuses on English. Representative English datasets include ATIS, WikiSQL, Spider, etc. This paper presents DuSQL, a larges-scale and pragm…
SQL ParsingText to SQLText-To-SQLThe Mask of Civility: Benchmarking Chinese Mock Politeness Comprehension in Large Language Models
From a pragmatic perspective, this study systematically evaluates the differences in performance among representative large language models (LLMs) in recognizing politeness, impoliteness, and mock politeness phenomena in…