Types¶
Core public Pydantic types for evaldata.
Bases: BaseModel
Lifecycle limits for a platform's persistent connection pool.
Bases: BaseModel
DuckDB connection settings.
Bases: BaseModel
SQLite connection settings.
Bases: BaseModel
PostgreSQL connection settings.
Bases: BaseModel
Non-secret Databricks connection settings.
Bases: BaseModel
Non-secret Snowflake connection settings.
Bases: BaseModel
Non-secret BigQuery connection settings.
Bases: _PlatformRefBase
Reference to a DuckDB database.
Bases: _PlatformRefBase
Reference to a SQLite database.
Bases: _PlatformRefBase
Reference to a PostgreSQL database.
Bases: _PlatformRefBase
Reference to a Databricks SQL Warehouse.
Bases: _PlatformRefBase
Reference to a Snowflake account.
Bases: _PlatformRefBase
Reference to a BigQuery project.
Bases: BaseModel
A SQL column type with native and canonical representations.
classmethod
¶
parse(raw: str, dialect: SQLDialect) -> SqlType
Build a SqlType, canonicalising raw in dialect.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
raw
|
str
|
The native SQL type string. |
required |
dialect
|
SQLDialect
|
The SQLGlot dialect to parse |
required |
Returns:
| Type | Description |
|---|---|
SqlType
|
A |
SqlType
|
|
Bases: BaseModel
A result-set column: name, SQL type, and tri-state nullability.
Bases: RootModel[list[Column]]
An ordered sequence of typed result-set columns with unique names.
Wraps a list[Column] and serialises as a plain JSON array. Engines that report no
result-column types produce an UntypedSchema instead.
Bases: RootModel[list[str]]
An ordered sequence of unique result-column names with no type information.
Produced by engines whose driver reports no result-column types (e.g. SQLite). Carries
only names; type comparison against an UntypedSchema is inconclusive rather than refuting.
Bases: BaseModel
Expected outcome as concrete rows without column types: value comparison only.
Bases: BaseModel
Expected outcome as concrete rows plus a schema: value and type comparison.
Bases: BaseModel
Expected outcome as a gold/reference query whose executed result IS the expected answer.
The expected answer is whatever sql returns when executed, not literal rows authored
up front.
Bases: BaseModel
The result set must contain exactly this many rows.
Bases: BaseModel
The result set must contain at least these columns.
Bases: BaseModel
A named column must have the given SQL type.
Bases: BaseModel
A named column must contain no NULL values.
Bases: BaseModel
A named column's values must be distinct.
Bases: BaseModel
Expected outcome specified as a suite of expectations the result set must satisfy.
Bases: BaseModel
Rules for deciding whether two result sets are equivalent.
A non-empty match_key selects the keyed FULL OUTER JOIN comparison: rows are
aligned on the key columns and compared per remaining column, enabling
null_equality="distinct", an exact abs(actual - expected) <= float_tolerance
band, and per-column mismatch counts. An empty match_key uses the keyless bag
(multiset) comparison.
Bases: BaseModel
Per-eval-case ceiling on platform resource consumption.
Bases: BaseModel
One evaluation case: an input with an expected outcome and a platform to run against.
Bases: BaseModel
Base type for structured errors.
Bases: _SolverOutputBase
SQL produced successfully by a solver.
Bases: _SolverOutputBase
A solver failure.
Bases: Error
A typed failure from running SQL against a platform.
condition carries the driver's error class/code string (e.g. Spark's
TABLE_OR_VIEW_NOT_FOUND) when it exposes one, since not every engine reports SQLSTATE.
Bases: _ExecutionResultBase
A successful SQL execution.
Bases: _ExecutionResultBase
A failed SQL execution.
Bases: BaseModel
A column whose actual type in the result set differs from the expected type.
Bases: BaseModel
Per-column count of rows whose value in the actual result set differs from the expected value.
Bases: BaseModel
Structured difference between an actual result set and an expected result set.
Bases: BaseModel
The result of checking one Expectation against an executed result.
expected/actual carry the compared scalars (a row count, a column's type
raw); count carries the number of offending elements (NULL or duplicate
values); sample_rows carries a bounded sample of the offending rows (empty
unless the expectation failed and the kind produces one); detail is the
human-readable failure message, None when the expectation holds. Which fields
are populated depends on the expectation kind.
Bases: BaseModel
The outcome of running a Scorer against an EvalCase: a verdict plus diagnostics.
score and basis must be absent when verdict is "inconclusive" — an undecided result
carries no evidence.
Bases: BaseModel
One equivalence check's judgment on whether two queries are equivalent.
A verdict never carries a diff; a refutation surfaces as a result-set ScoreResult.diff.