Skip to content

SQL Syntax Match

Definitions

SQL Syntax Match evaluates the syntactic equivalence between generated SQL queries and a set of ground truth queries. The strict comparison can tolerate formatting changes.

Example Usage

Required data items: answer, ground_truth_answers

from continuous_eval.metrics.code import SQLSyntaxMatch
sql_syntax_match = SQLSyntaxMatch()
datum = {
"answer": "SELECT * FROM users;"",
"ground_truth_answers": [
"SELECT * from users;"
],
},
metric = SQLSyntaxMatch()
print(metric(**datum))

You can optionally initialize the metric to use optimized SQL queries using the sqlglot optimizer and optionally pass in the schema. For example:

schema={"x": {"A": "INT", "B": "INT", "C": "INT", "D": "INT", "Z": "STRING"}}
sql_syntax_match_optimized = SQLSyntaxSimilarity(optimized=True, schema=schema)

Example Output

{
"SQL_Syntax_Match": 1.0
}