What is SParC?
SParC Paper (ACL'19)
is a dataset for cross-domain S
sing in C
ontext. It is the context-dependent/multi-turn version of the Spider task
, a complex and cross-domain text-to-SQL challenge. SParC consists of 4,298 coherent question sequences (12k+ unique individual questions annotated with SQL queries annotated by 14 Yale students), obtained from user interactions with 200 complex databases over 138 domains.
and conversational CoSQL
Spider Chanllenge (EMNLP'18)
CoSQL Chanllenge (EMNLP'19)
SParC is built upon the Spider dataset. Comparing to other existing context-dependent semantic parsing/text-to-SQL datasets such as ATIS, it demonstrates:
- complex contextual dependencies (annotated by 15 Yale computer science students)
- has greater semantic diversity due to complex coverage of SQL logic patterns in the Spider dataset.
- requires generalization to new domains due to its cross-domain nature and the unseen databasest time.
The data is split into training, development, and unreleased test sets. Download a copy of the dataset (distributed under the CC BY-SA 4.0
Details of baseline models and evaluation script can be found on the following GitHub site:
SParC GitHub Page
Once you have built a model that works to your expectations on the dev set,
you can submit it to get official scores on the dev and a hidden test set. To preserve the
integrity of test results, we do not release the test set to the public. Instead, we request
you to submit your model so that we can run it on the test set for you. Here's a tutorial walking you through official evaluation of your model:
Some examples look like the following:
Have Questions or Want to Contribute ?
We expect the dataset to evolve. We would greatly appreciate it if you could donate us your non-private databases or SQL queries for the project.
We thank Tianze Shi
and the anonymous reviewers for their precious comments on this project and Melvin Gruesbeck
for designing the nice example illustrations. Also, we thank Pranav Rajpurkar
for giving us the permission to build this website based on SQuAD
Part of our SParC team at YINS