The predicate DSL¶

A model plugin decides which variants are prioritized — worth a closer look — with a heuristic over its score columns. That same heuristic has to run in two very different places: as Python, over a local store's rows; and as SQL, pushed down into a warehouse so the database does the filtering. Writing it twice invites them to drift.

The predicate DSL (altar.predicates) lets a plugin declare the heuristic once as a Predicate, and Altar renders it to both — a Python callable and a SQL boolean expression. One source, two generated renderings, no hand-maintained SQL copy.

Building a predicate¶

Terms produce values; predicates compare and combine them into a boolean.

from altar.predicates import Col, Lit, Abs, Ge, Gt, Eq, And, Or, Not

# "a large effect in an accessible region":  |logfc| >= 0.5  AND  in_peak = TRUE
predicate = And(
    Ge(Abs(Col("logfc")), Lit(0.5)),
    Eq(Col("in_peak"), Lit(True)),
)

Piece	Options
Terms	`Col(name)` (a column), `Lit(value)` (a constant), `Abs(term)` (absolute value), `Greatest(terms)` and `Least(terms)` (null-ignoring maximum and minimum)
Comparisons	`Ge`, `Gt`, `Le`, `Lt`, `Eq`, `Ne` — each takes two terms
Tests	`IsNull(term)`, `In(term, values)` (membership in a tuple of literals)
Combinators	`And(preds)`, `Or(preds)`, `Not(pred)`

Rendering it¶

The same object renders both ways:

predicate.to_python()  # -> callable: row(dict) -> bool
predicate.to_sql()  # -> "(ABS(logfc) >= 0.5 AND in_peak = TRUE)"

fn = predicate.to_python()
fn({"logfc": -0.8, "in_peak": True})  # True

to_sql optionally takes a column renderer (SqlCol) so a caller can qualify names with a table alias — the warehouse store uses this to emit its push-down WHERE clause.

Three-valued (Kleene) semantics¶

The reason the two renderings stay in lock-step on missing data is that the DSL evaluates with SQL's three-valued logic. A comparison touching a None/missing value is unknown (None), not False; that unknown propagates through And / Or / Not exactly as SQL NULL does; and only a definite True counts as a match:

Ge(Col("logfc"), Lit(0.5)).evaluate({"logfc": None})  # None  (unknown — not False)
predicate.to_python()({"logfc": None})  # False (to_python matches only definite True)

That's why a nullable boolean column is compared explicitly — Eq(Col("in_peak"), Lit(True)), which renders to in_peak = TRUE — rather than tested for bare truthiness: it makes the Python and SQL renderings handle NULLs identically, so a variant is prioritized in the local store if and only if it would be in the warehouse.

Null tests, membership, and multi-column extremes¶

IsNull(term) is the one test that is never unknown: it is True for a missing or None value and False otherwise, and renders to (term IS NULL). Write its negation as Not(IsNull(term)). Use it to say what a missing value should mean instead of letting it silently fail to match:

from altar.predicates import Col, In, IsNull, Or

# "a promoter or enhancer variant, or one the region annotation has not covered"
predicate = Or(In(Col("region_type"), ("promoter", "enhancer")), IsNull(Col("region_type")))
predicate.to_sql()  # -> "((region_type IN ('promoter', 'enhancer')) OR (region_type IS NULL))"

In(term, values) follows SQL IN: it is True when the term equals one of the values, unknown when the term is null, and False otherwise. The values are literals of one type family (booleans, numbers, or strings). None is rejected, because SQL makes x IN (..., NULL) unknown rather than true for a null x; combine In with IsNull as above to match nulls. An empty tuple is False for every row and renders to FALSE, since IN () is not valid SQL.

Greatest(*terms) and Least(*terms) take the maximum or minimum over the operands that are not null, and are null only when every operand is. That is the shape of "the highest allele frequency over the ancestry groups that have one":

from altar.predicates import Col, Ge, Greatest, Lit

common = Ge(Greatest(Col("af_afr"), Col("af_nfe"), Col("af_eas")), Lit(0.01))
common.evaluate({"af_afr": None, "af_nfe": 0.02})  # True: one missing group does not hide the others

SQL dialects disagree on nulls here. DuckDB's and PostgreSQL's GREATEST skip NULL arguments, but BigQuery's returns NULL when any argument is NULL. So Greatest never hands SQL a NULL argument unless all operands are NULL: it renders each argument as a COALESCE of the operands rotated to start at a different one,

GREATEST(COALESCE(af_afr, af_nfe, af_eas), COALESCE(af_nfe, af_eas, af_afr), COALESCE(af_eas, af_afr, af_nfe))

which gives the same answer under either convention and matches the Python rendering. The SQL text grows with the square of the operand count, which is negligible for the handful of columns a predicate compares.

A float NaN is a value, not a null, in both renderings: IsNull is False for it, and Greatest and Least do not skip it. That is consistent with the rest of the DSL, which never treats NaN as missing. A source should therefore emit None (SQL NULL) for a missing value rather than NaN.

In values and Lit constants may be NumPy scalars, for example values taken from an array. They render as plain SQL literals, and In stores them as built-in Python scalars with duplicates removed in first-occurrence order. Pass an ordered collection such as a tuple or list; a set is rejected because its order, and so the SQL text, can differ between processes.

Where it's used¶

A ModelPlugin returns a Predicate from prioritize_predicate() (add a model binding), and a prioritizing AnnotationSource returns one as well. The SQLite store runs the Python rendering; the BigQuery store can push the SQL rendering down. Declare the heuristic once in the DSL so both interpretations remain aligned.