SQL Syntax Match

Definitions

SQL Syntax Match evaluates the syntactic equivalence between generated SQL queries and a set of ground truth queries. The strict comparison can tolerate formatting changes.

Example Usage

Required data items: answer, ground_truth_answers

from continuous_eval.metrics.code import SQLSyntaxMatch

sql_syntax_match = SQLSyntaxMatch()

datum = {
    "answer": "SELECT * FROM users;"",
    "ground_truth_answers": [
        "SELECT  *  from  users;"
    ],
},

metric = SQLSyntaxMatch()
print(metric(**datum))

You can optionally initialize the metric to use optimized SQL queries using the sqlglot optimizer and optionally pass in the schema. For example:

schema={"x": {"A": "INT", "B": "INT", "C": "INT", "D": "INT", "Z": "STRING"}}
sql_syntax_match_optimized = SQLSyntaxSimilarity(optimized=True, schema=schema)

Example Output

{
    "SQL_Syntax_Match": 1.0
}