Rank | Model | Test |
---|---|---|
1 Nov 24, 2020 |
BRIDGE v2 + BERT(ensemble) (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
68.3 |
2 Nov 24, 2020 |
BRIDGE v2 + BERT (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
64.3 |
3 May 30, 2020 |
AuxNet + BART (DB content used)
Anonymous |
62.6 |
4 May 30, 2020 |
BRIDGE + BERT (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
59.9 |
5 May 20, 2020 |
GAZP + BERT (DB content used)
University of Washington & Facebook AI Research (Zhong et al., EMNLP '20) |
53.5 |
Rank | Model | Dev | Test |
---|---|---|---|
1 Nov 19, 2020 |
RAT-SQL + GraPPa + Adv (DB content used)
Anonymous |
75.5 | 70.5 |
2 Dec 25, 2020 |
RATSQL + GraPPa + GP (DB content used)
OCFT Gamma Big Data Lab (Zhao et al.,'21) |
72.8 | 69.8 |
2 Sep 08, 2020 |
RATSQL + GAP (DB content used)
University of Waterloo & AWS AI Labs (Shi et al., AAAI'21) code |
71.8 | 69.7 |
2 Aug 18, 2020 |
RATSQL + GraPPa (DB content used)
Yale & Salesforce Research (Yu et al., ICLR'21) code |
73.4 | 69.6 |
5 Nov 24, 2020 |
BRIDGE v2 + BERT(ensemble) (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
71.1 | 67.5 |
6 May 02, 2020 |
RATSQL v3 + BERT (DB content used)
Microsoft Research (Wang and Shin et al., ACL '20) code |
69.7 | 65.6 |
7 Dec. 07, 2020 |
DuoRAT + BERT (DB content used)
Anonymous |
- | 65.4 |
7 Sep. 8, 2020 |
YCSQL + BERT (DB content used)
Anonymous |
- | 65.3 |
9 Nov 24, 2020 |
BRIDGE v2 + BERT (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
70.0 | 65.0 |
10 Sep. 8, 2020 |
ShadowGNN (DB content used)
Anonymous |
- | 64.8 |
11 Sep. 8, 2020 |
GP-RATSQL + BERT (DB content used)
Anonymous |
- | 64.5 |
11 Nov. 25, 2020 |
RATSQL-HPFT + BERT (DB content used)
Anonymous |
- | 64.4 |
12 May 31, 2020 |
AuxNet + BART (DB content used)
Anonymous |
70.0 | 61.9 |
13 Dec 13, 2019 |
RATSQL v2 + BERT (DB content used)
Microsoft Research (Wang and Shin et al., ACL '20) code |
65.8 | 61.9 |
14 May 31, 2020 |
AuxNet + BART
Anonymous |
68.0 | 61.3 |
15 Feb 18, 2020 |
RYANSQL v2 + BERT
Kakao Enterprise (Choi et al., '20) |
70.6 | 60.6 |
15 Oct 19, 2020 |
SmBoP + BART
Tel-Aviv University & Allen Institute for AI (Rubin and Berant '20) |
66.0 | 60.5 |
17 Dec 18, 2019 |
IRNet++ + XLNet (DB content used)
Anonymous |
65.5 | 60.1 |
18 May 30, 2020 |
BRIDGE + BERT (DB content used)
Salesforce Research (Lin et al., EMNLP-Findings '20) code |
65.5 | 59.2 |
19 Nov 12, 2019 |
RYANSQL + BERT
Kakao Enterprise (Choi et al., '20) |
66.6 | 58.2 |
20 Dec 13, 2019 |
RATSQL v2 (DB content used)
Microsoft Research (Wang and Shin et al., ACL '20) code |
62.7 | 57.2 |
21 Dec 13, 2019 |
SLSQL + BERT + Data Annotation
National University of Singapore (Lei and Wang et al., EMNLP '20) code |
60.8 | 55.7 |
22 Dec 13, 2019 |
EditSQL+LSL + BERT
Anonymous |
57.9 | 55.2 |
23 June 24, 2019 |
IRNet v2 + BERT
Microsoft Research Asia |
63.9 | 55.0 |
24 Sep 20, 2019 |
GIRN + BERT
Anonymous |
60.2 | 54.8 |
24 May 19, 2019 |
IRNet + BERT
Microsoft Research Asia (Guo and Zhan et al., ACL '19) code |
61.9 | 54.7 |
26 Nov 4, 2019 |
GNN + Bertrand-DR
Got It R&D (Kelkar et al., '20) code |
57.9 | 54.6 |
27 Apr 8, 2020 |
CNSQL
Anonymous |
58.0 | 54.0 |
28 Sep 19, 2019 |
RATSQL
Anonymous |
60.6 | 53.7 |
29 Sep 1, 2019 |
EditSQL + BERT
Yale University & Salesforce Research (Zhang et al., EMNLP '19) code |
57.6 | 53.4 |
29 May 21, 2020 |
GAZP + BERT
University of Washington & Facebook AI Research (Zhong et al., EMNLP '20) |
- | 53.3 |
31 May 21, 2020 |
NatSQL v3
Anonymous |
- | 53.2 |
32 May 28, 2020 |
IRNET+ GNN
Anonymous |
- | 49.6 |
33 June 24, 2019 |
IRNet v2
Microsoft Research Asia |
55.4 | 48.5 |
34 Aug 30, 2019 |
Global-GNN (DB content used)
Tel-Aviv University & Allen Institute for AI (Bogin et al., EMNLP '19) code |
52.7 | 47.4 |
35 Dec 13, 2019 |
LSL
Anonymous |
56.8 | 47.0 |
35 Apr 5, 2020 |
GraphSQL
Anonymous |
52.8 | 46.9 |
37 May 19, 2019 |
IRNet
Microsoft Research Asia (Guo and Zhan et al., ACL '19) code |
53.2 | 46.7 |
37 Mar 17, 2020 |
SG-IRNet
Anonymous |
- | 46.6 |
39 Dec 13, 2019 |
NatSQL v2
Anonymous |
52.0 | 46.4 |
40 June 11, 2019 |
HSRNet
Anonymous |
51.5 | 45.6 |
41 June 12, 2019 |
CFGN
Anonymous |
48.7 | 44.1 |
42 Aug 31, 2019 |
NatSQL
Anonymous |
52.9 | 42.5 |
43 May 16, 2019 |
GNN
Tel-Aviv University & Allen Institute for AI (Bogin et al., ACL '19) code |
40.7 | 39.4 |
44 Feb 25, 2019 |
SASeq
Anonymous |
40.8 | 37.4 |
45 May 30, 2019 |
GrammarSQL
Allen Institute for AI (Lin et al., '19) |
34.8 | 33.8 |
46 Sep 1, 2019 |
EditSQL
Yale University & Salesforce Research (Zhang et al., EMNLP '19) code |
36.4 | 32.9 |
47 Dec 13, 2019 |
GuideSQL
Anonymous |
36.8 | 31.5 |
48 Sep 20, 2018 |
SyntaxSQLNet + augment
Yale University (Yu et al., EMNLP '18) code |
24.8 | 27.2 |
49 April 18, 2019 |
RCSQL
SAP Labs Korea (Lee, EMNLP'19) |
28.5 | 24.3 |
50 Sep 20, 2018 |
SyntaxSQLNet
Yale University (Yu et al., EMNLP '18) code |
18.9 | 19.7 |
51 Sep 20, 2018 |
SQLNet
Shanghai Jiao Tong University (modified by Yale) (Xu et al., '18) code |
10.9 | 12.4 |
52 Sep 20, 2018 |
TypeSQL
Yale University (Yu et al., NAACL '18) code |
8.0 | 8.2 |
53 Sep 20, 2018 |
Seq2Seq + attention
University of Edinburgh (modified by Yale) (Dong and Lapata, ACL '16) code |
1.8 | 4.8 |