Score variants¶

run_scoring() is the common path from one already-prepared model request to persisted scalar and named-detail scores. A model binding turns the request into either container work or an in-process/API call; the engine executes that plan, decodes the binding's native result, validates its versioned schema, and avoids rewriting scores that appeared concurrently. Use prepare_scoring_batches() before container execution when the source cohort may contain cache hits: otherwise the model still computes every variant in the request even though persistence skips existing rows.

Prepare cache misses before requests¶

Cache filtering must happen across the complete candidate cohort, before executable requests are fixed. Filtering each pre-existing source batch separately can leave many undersized model runs. For example, 399 misses scattered through 1,000 candidates should be compacted into execution units of 100, 100, 100, and 99 when the configured outer limit is 100.

prepare_scoring_batches() implements that lifecycle for canonical container-model TSV inputs:

validate and deduplicate the complete chr, pos, ref, alt[, source_id] candidate stream;
set aside the variants the model's manifest declares ineligible (see variant eligibility);
derive the expected scientific identity and reject an incompatible cache before staging work;
query the score store in bounded chunks;
densely repack all misses in first-seen source order;
stage one canonical TSV per execution unit; and
construct fresh immutable ScoringRequest values only after the global miss count is known.

The candidate file is headerless, tab-separated UTF-8. It is read with altar_identity.read_variant_rows, the same reader every model runtime uses for its staged batch, so preparation and the container agree on every row: blank lines are skipped, aliases such as 1 and chr01 canonicalize to chr1, and an invalid row fails with its line number. Staged batches are written as UTF-8 too.

from altar.execution import Transfer
from altar.scoring import prepare_scoring_batches, run_scoring

preparation = await prepare_scoring_batches(
    plugin,
    request_template,  # unbatched and never executed
    candidate_variants=Transfer(uri="cohort.tsv", logical_path="candidates.tsv"),
    score_store=store,
    storage=storage,
    destination_transfer=map_output,
    max_variants_per_batch=100_000,
)

for batch in preparation.batches:

    def map_prepared_input(transfer, prepared=batch.variant_input):
        return prepared if transfer.logical_path == prepared.logical_path else map_input(transfer)

    await run_scoring(
        plugin,
        batch.request,
        model_name="ChromBPNet K562",
        score_store=store,
        backend=backend,
        storage=storage,
        input_transfer=map_prepared_input,
        output_transfer=map_output,
    )

The helper keeps the cohort and global miss relation in temporary SQLite storage rather than process memory. Its max_variants_per_batch controls an outer execution unit—one staged file and one scoring request. A model's inference batch size is an independent runtime concern: a single 399-variant execution unit may still be processed by the runtime as inference batches of 100, 100, 100, and 99. Warehouse deployments can use a relation-native anti-join and export for the same lifecycle without downloading the cohort; the returned request semantics should remain the same.

Variant eligibility¶

A model's manifest may declare which variants it can score with variant_eligibility. Enformer, Borzoi, LegNet, and Sei accept SNVs only; the other bundled models admit every variant. Preparation classifies each unique candidate and excludes ineligible ones before the cache is queried, so they are never sent to an executor. It reports them in three places:

ScoringPreparation.ineligible_count, so cache_hit_count + unscored_count + ineligible_count equals candidate_count;
ScoringPreparation.ineligible_variants, a report staged through destination_transfer at ScoringLayout.ineligible_variants_file(job_id, model_id), or None when every candidate is eligible; and
that report's columns, which are INELIGIBLE_VARIANT_COLUMNS: chr, pos, ref, alt, variant_id, variant_class, reason, in first-seen order. reason is unsupported_variant_class or allele_too_long.

Altar does not store anything for an ineligible variant. They are not cache misses, so they never cause rescoring. The check is cheap and depends only on the variant and the manifest, so each preparation excludes and reports the same variants again. A materialized result shows an ineligible variant the same way as an unscored one: it has no score from that model.

The engine enforces the same rule on output. If a container runtime or an inline scorer returns a primary or detail row for an ineligible variant, run_scoring() raises ResultValidationError. An inline result is checked as a whole before any table is written. A container result is streamed, so it is checked per batch before that batch is written; batches routed earlier stay stored. Staging a container batch by hand, without preparation, therefore fails instead of storing a score the model says it cannot produce. An inline binding that restricts eligibility filters its own run inputs with manifest.variant_eligibility.admits().

Warehouse deployments that build batches with a relation-native export must apply the same filter. Pass variant_eligibility=plugin.manifest.variant_eligibility to BigQueryScoreStore.prepare_unscored_table() or export_unscored(): the store translates it into a SQL filter over the candidate relation's ref and alt columns before the cache anti-join, so ineligible variants are neither counted nor exported. These methods don't report the excluded variants. Without the argument, an SNV-only model's export includes indels and run_scoring() rejects the whole batch.

Bindings with named details¶

Container bindings that declare named detail tables, such as Borzoi gene_effects, Enformer track_effects, and Sei sequence_class_effects, use the same helper. Pass the DetailStore that run_scoring() will write to; preparation rejects a missing store before reading the cohort. A variant is a cache hit only when all of these hold:

the ScoreStore has its primary row for the current run and genome: in the original store mode, under a scientifically compatible run identity; in compact-lineage mode, under the current run's lineage (its scientific projection, see what decides a generated lineage);
every declared detail table exists for the model ID with the same scientific result compatibility (a stored table with different meaning raises IncompatibleResultError before any batch is staged); and
every declared detail table has at least one row for that variant (DetailStore.get_missing_variants(), queried only for variants that already passed the primary check).

Everything else is a miss and is recomputed. The engine's per-key deduplication then skips primary and detail rows that already exist, so repairing a partial result never duplicates rows. ScoringPreparation.incomplete_detail_count reports how many misses had a cached primary row but lacked a detail table.

The rule is safe because run_scoring() persists every detail table before any primary row of the same run. A failure while routing details therefore leaves no primary row, and the primary row acts as the per-variant commit marker. The parent-presence check additionally catches primary rows whose details never reached the supplied store, for example after switching to a fresh detail database. Two limits follow from there being no per-variant detail completion record:

Zero-row variants are recomputed on every preparation. A variant that legitimately produces no rows for a detail table (for example, a Borzoi variant with no gene in its prediction window) cannot be told apart from one whose rows were never written, so every prepare_scoring_batches() call treats it as a miss and the model scores it again. That costs compute but never loses output; rerunning it writes nothing new.
A variant that has some rows and a primary row is assumed complete. Only the details-before-primary write order guarantees this, so a deployment that writes primary rows by another path must also write the complete detail rows first.

Score reuse remains primary-only. Detail tables carry no score lineage, so a non-empty ScoreReusePolicy is rejected for detail bindings with CapabilityNotSupported instead of pairing a reused primary score with details from another producer. See reuse scores across model updates.

Build a request¶

ScoringRequest carries facts that vary by analysis: job and model identifiers, genome label, logical file layout, and the binding-specific artifact configuration. You may build an unbatched request as a preparation template or construct one directly when its input is already known to contain exactly the variants that should execute.

Install the relevant binding before resolving its registry key (for this example, pip install altar-chrombpnet).

from altar.models import get_model_plugin
from altar.results import SqliteScoreStore
from altar.scoring import run_scoring

plugin = get_model_plugin("CHROMBPNET")
store = SqliteScoreStore("scores.sqlite")

result = await run_scoring(
    plugin,
    request,
    model_name="ChromBPNet K562",
    score_store=store,
    backend=backend,
    storage=storage,
    input_transfer=map_input,
    output_transfer=map_output,
)
print(result.added, result.skipped)

The binding validates its own artifact fields. For ChromBPNet those fields identify model files and peak artifacts. For AlphaGenome they select the hosted API configuration and variants.

Container plans¶

ContainerScoringPlan contains independently executable ContainerTaskSpec objects plus any dependent summary task. Each task declares:

image and command;
CPU, memory, and GPU request;
structured labels;
logical input and output transfers.

An execution backend resolves and runs those declarations. The model binding does not contain Kubernetes, Modal, or local-Docker code. input_transfer and output_transfer map the plan's logical transfers onto a deployment's storage URIs; they do not change the plan or introduce model/backend pairs. A deployment may use spec_mapper to apply backend-neutral execution overrides such as timeouts or resource requests before submission. It may change only resources, timeout_s, and avoid_hosts; image, command, transfers, labels, and plugin identity remain part of the binding-authored scientific plan. The mapped, JSON-serializable spec is persisted in the task ledger so recovery can relaunch it.

Inline plans¶

InlineScoringPlan contains an asynchronous scorer used for a hosted API or lightweight in-process implementation. It does not create a container task or require an execution backend.

The altar-alphagenome binding uses this form because scoring is performed through its hosted API. Supply its runtime client through runtime=; backend, storage, and transfer mappers are unnecessary. The adapter reduces the provider's per-track output to its declared scalar schema and preserves every tidy track row in the track_scores detail table. Because this binding declares details, pass a DetailStore as well as the primary ScoreStore; the engine rejects the run before making API calls if either required output sink is absent.

Decode and persist¶

For a container plan, the binding's scoring_result_codec() translates native runtime files into bounded batches containing variant_id, the manifest's scalar fields, and optional typed row-identity fields. The default codec accepts a headered TSV with the exact score names. A binding overrides it when the runtime uses different names or emits identity/transport columns; ChromBPNet maps native lfc to semantic logfc and preserves its runtime's chr/pos/ref/alt. A binding need not emit a decomposed locus: when a store declares chr/pos/ref/alt in required_row_identity(), the engine derives omitted fields from the canonical variant_id and rejects emitted fields that disagree with it.

The engine rejects missing or undeclared fields, missing store-required identity, wrong scalar types, non-finite floats, duplicate variant IDs, and manifest/column drift before persistence. It passes the exact PluginRunIdentity into every write; stores reject a model ID that already belongs to another model instance (configuration, output selection, genome, wrapper, runtime, or result ABI). The identity excludes job IDs, subjects, paths, and batch boundaries so repeated and disjoint batches can populate the same cache safely. It calls ScoreStore.get_unscored_variants() for every batch to avoid ordinary sequential re-writes. That read-then-write check is not a cross-process transaction: append-only stores may still need deployment-level single-writer coordination or compaction for concurrent runs.

Named one-to-many results use a DetailSchema in the binding manifest and a separate DetailStore. The schema declares a stable table name, version, scalar fields, and logical key fields. The engine validates every row, detects duplicates across batches, and routes it without teaching the store about AlphaGenome or another model. Large binary and array artifacts remain a separate future result shape.

See the real scoring tutorial for the storage and assembly path and run container tasks for task execution.