SQL Syntax Match

Definitions

SQL Syntax Match evaluates the syntactic equivalence between generated SQL queries and a set of ground truth queries. The strict comparison can tolerate formatting changes.

Example Usage

Required data items: answer, ground_truth_answers

from continuous_eval.metrics.code.sql import SQLSyntaxMatch

datum = {
    "answer": "SELECT * FROM users;",
    "ground_truth_answers": "SELECT  *  from  users;",
}

metric = SQLSyntaxMatch()
print(metric(**datum))

You can optionally initialize the metric to use optimized SQL queries using the sqlglot optimizer and optionally pass in the schema. For example:

schema={"x": {"A": "INT", "B": "INT", "C": "INT", "D": "INT", "Z": "STRING"}}
sql_syntax_match_optimized = SQLSyntaxMatch(optimized=True, schema=schema)

Example Output

{
    "SQL_Syntax_Match": 1.0
}