Skip to content

Scoring engine

altar.scoring prepares a cohort's cache misses as batches, runs a model on them, and decodes, validates and stores the results. Score a cohort shows how to use it.

Stable, except ScoringOutputError, which is provisional. Provisional names lists it.

scoring

Stable execution, decoding, validation, and routing API for model scoring runs.

ScoringOutputError is provisional and listed in __provisional__: it remains importable, and it is expected to change.

INELIGIBLE_VARIANT_COLUMNS module-attribute

INELIGIBLE_VARIANT_COLUMNS = (
    "chr",
    "pos",
    "ref",
    "alt",
    "variant_id",
    "variant_class",
    "reason",
)

Header of the ineligible-variant report staged by :func:prepare_scoring_batches.

SpecMapper module-attribute

SpecMapper = Callable[
    [ContainerTaskSpec], ContainerTaskSpec
]

Rewrites a task's scheduling fields before submission; validated_mapped_spec bounds the change.

TransferMapper module-attribute

TransferMapper = Callable[[Transfer], Transfer]

Maps one binding-declared transfer to its deployment form; validate_transfer_mapping bounds the change.

ScoreReusePolicy dataclass

ScoreReusePolicy(
    accepted_lineage_ids: tuple[str, ...] = (),
    exact: bool = False,
)

Additional acceptable lineages, in preference order after the current run.

This is an explicit scientific decision by the caller, scoped to the model and genome being queried. It is neither inferred from version numbers nor transitive. An empty policy accepts only the current run, plus the lineages its binding declares equivalent when the store is given the plugin and a descriptor. Acceptance changes reads, never the lineage stamped on newly computed scores.

An exact policy, built with :meth:exactly, instead lists every lineage to read, the current one first, and resolves to exactly that list.

exactly classmethod

exactly(lineage_ids: Iterable[str]) -> Self

Read exactly lineage_ids, in that order of preference, such as a frozen Resolution.

The first ID must be the run's current lineage, which need not be registered; every other ID must be. Equivalent lineages are not added. A Resolution passes its lineage_ids.

ordered_ids

ordered_ids(current: ScoreLineage) -> tuple[str, ...]

Return the lineage IDs this policy names for a run whose current lineage is current, current first.

ArtifactFileCoverage dataclass

ArtifactFileCoverage(
    variant_ids: tuple[str, ...],
    variant_id: str | None = None,
)

What one collected artifact file covers.

variant_ids are the variants its record lists; variant_id is the variant its :class:~altar.results.ArtifactKey names, set only for a variant-scoped file.

CohortScoringResult dataclass

CohortScoringResult(
    preparation: ScoringPreparation,
    runs: tuple[ScoringRunResult, ...],
)

What one cohort's preparation produced and what executing its batches wrote.

added property

added: int

Primary rows persisted by every batch of this cohort.

skipped property

skipped: int

Primary rows a batch produced that the store already held.

variants_without_rows property

variants_without_rows: int

Batch variants that got no primary row, summed over every batch of this cohort.

ArtifactResultRouter

ArtifactResultRouter(store: ArtifactStore)

Verify, digest, and store the files of one selected artifact without decoding them.

Files are recorded under the producing run's score lineage, the one its primary rows are stored under, and deduplicated against that lineage's records only: a file every one of whose variants an earlier run under the same lineage already covered is skipped, as an already stored detail row is.

ArtifactRunResult dataclass

ArtifactRunResult(
    name: str, added: int, skipped: int, byte_size: int
)

Files stored for one selected artifact: added were handed to the store, skipped were already covered.

CollectedArtifact dataclass

CollectedArtifact(
    uri: str, key: ArtifactKey, variant_ids: tuple[str, ...]
)

One produced artifact file awaiting routing: where it is, which task wrote it, and the variants it covers.

DetailResultRouter

DetailResultRouter(store: DetailStore)

Validate and stream one named detail result into a generic detail store.

Rows are written under the producing run's score lineage, the one its primary rows are stored under, and deduplicated against that lineage's rows only: an accepted earlier lineage may make a variant a cache hit in preparation, but it never absorbs rows a later run actually produced.

route async

route(
    batches: AsyncIterable[DetailResultBatch],
    *,
    schema: DetailSchema,
    plugin: ModelPlugin,
    plugin_identity: PluginRunIdentity,
    model_id: str,
    model_name: str,
    lineage_id: str | None = None,
    batch_variant_ids: Collection[str] | None = None,
) -> DetailRunResult

Validate, deduplicate, and write one detail table's batches, and return its write counts.

batch_variant_ids are the variants the run asked for. A row for any other variant raises :class:ScoringOutputError. None skips the check.

DetailRunResult dataclass

DetailRunResult(
    name: str, added: int, skipped: int, batches: int
)

Write counts for one named detail table.

PrimaryResultRouter

PrimaryResultRouter(store: ScoreStore)

Validate and stream primary scalar batches into one generic score store.

route async

route(
    batches: AsyncIterable[ResultBatch],
    *,
    plugin: ModelPlugin,
    plugin_identity: PluginRunIdentity,
    model_id: str,
    model_name: str,
    genome_build: str,
    lineage_descriptor: LineageDescriptor | None = None,
    batch_variant_ids: Collection[str] | None = None,
) -> tuple[int, int, int]

Return (added, skipped, batches) after bounded validation and persistence.

lineage_descriptor, the run's LineageDescriptor, is passed to the store's write unit, which verifies it and, with compact lineages, records it when the unit registers the run's lineage.

batch_variant_ids are the variants the run asked for. A row for any other variant raises :class:ScoringOutputError. None skips the check.

ResultValidationError

Bases: ValueError

A decoded row violates the binding's versioned primary result schema.

ScoringOutputError

Bases: ResultValidationError

A run's output does not belong to the batch the engine submitted.

It is raised for a primary or detail row whose variant is not in the batch, and for a terminal output that was not produced.

ScoringRunResult dataclass

ScoringRunResult(
    plan: ScoringPlan,
    plugin_identity: PluginRunIdentity,
    added: int,
    skipped: int,
    batches: int,
    output_uris: tuple[str, ...] = (),
    details: tuple[DetailRunResult, ...] = (),
    artifacts: tuple[ArtifactRunResult, ...] = (),
    variants_without_rows: int = 0,
)

Durable identities and write counts produced by one engine run.

variants_without_rows is the number of the batch's variants that got no primary row. It is 0 for an inline plan whose :meth:~altar.models.ModelPlugin.run_variant_ids returns None.

PreparedScoringBatch dataclass

PreparedScoringBatch(
    request: ScoringRequest,
    variant_input: Transfer | None,
    variant_count: int,
)

One dense missing-only batch and the request that scores it.

A container request reads its batch from variant_input, the staged variant file a deployment maps into the plan's inputs. An inline request carries its batch in its own run inputs, so variant_input is None and request is complete as it stands.

ScoringPreparation dataclass

ScoringPreparation(
    batches: tuple[PreparedScoringBatch, ...],
    plugin_identity: PluginRunIdentity,
    candidate_count: int,
    duplicate_count: int,
    cache_hit_count: int,
    unscored_count: int,
    reuse_policy: ScoreReusePolicy = ScoreReusePolicy(),
    incomplete_detail_count: int = 0,
    ineligible_count: int = 0,
    ineligible_variants: Transfer | None = None,
    incomplete_artifact_count: int = 0,
    resolution: Resolution | None = None,
)

Summary and executable units produced from one complete candidate set.

incomplete_detail_count class-attribute instance-attribute

incomplete_detail_count: int = 0

Unscored variants whose primary row was cached but a declared detail table had no row.

A table whose :attr:~altar.models.DetailSchema.row_count_field is 0 on the cached row is complete without any row; such a variant is a hit, not counted here.

ineligible_count class-attribute instance-attribute

ineligible_count: int = 0

Unique candidates the manifest's variant_eligibility, the reference, or allele normalization excludes.

They are never looked up or scheduled.

ineligible_variants class-attribute instance-attribute

ineligible_variants: Transfer | None = None

Staged report of the ineligible candidates, or None when there are none.

A headered TSV with :data:INELIGIBLE_VARIANT_COLUMNS, in candidate order. reason is an :class:~altar.models.IneligibilityReason value and variant_class a :class:~altar.models.VariantClass value. The report always describes the latest preparation: when there are no ineligible candidates, a report left at the same location by an earlier preparation is deleted.

incomplete_artifact_count class-attribute instance-attribute

incomplete_artifact_count: int = 0

Unscored variants whose primary row and detail rows were cached but a selected artifact did not cover.

A variant counted in incomplete_detail_count is not counted here again.

resolution class-attribute instance-attribute

resolution: Resolution | None = None

The lineages every cache check read, in preference order, as the score store resolved them once.

The primary, detail, and artifact checks all used exactly these lineages: the current run's, then the registered lineages its binding declares equivalent, then reuse_policy's accepted ones (or exactly an exact policy's list). Read the job's results back with ScoreReusePolicy.exactly(resolution.lineage_ids) so they come from the same set. None when the score store keeps no compact lineages, in which case the checks read reuse_policy as given.

ScoringPreparationError

Bases: ValueError

The candidate set or prepared-batch contract is invalid.

ArtifactResult dataclass

ArtifactResult(name: str, local_path: str)

One opaque artifact file an inline binding produced for the run's selected artifact name.

local_path is a file on the local filesystem; the engine digests it and hands it to the artifact store without decoding it. The binding owns the file until the engine has routed it, so a temporary directory the binding cleans up before execute_inline returns is not a valid home for it.

DelimitedDetailResultCodec dataclass

DelimitedDetailResultCodec(
    schema: DetailSchema,
    source_names: Mapping[str, str] = dict(),
    allowed_extra_fields: tuple[str, ...] = (),
    identity_field: str = "variant_id",
    delimiter: str = "\t",
    null_tokens: tuple[str, ...] = ("", "NA", "NaN", "nan"),
)

Strict decoder for one headered, delimited named-detail result.

This is the detail-table counterpart to :class:DelimitedResultCodec. It deliberately shares the scalar parsing rules while returning batches tagged with the manifest's DetailSchema.name.

decode async

decode(
    storage: Storage,
    output_uris: Sequence[str],
    *,
    batch_size: int = 10000,
) -> AsyncIterator[DetailResultBatch]

Stream schema-shaped detail rows from every selected file.

DelimitedResultCodec dataclass

DelimitedResultCodec(
    schema: ResultSchema,
    source_names: Mapping[str, str] = dict(),
    row_identity_fields: Mapping[str, str] = dict(),
    allowed_extra_fields: tuple[str, ...] = (),
    identity_field: str = "variant_id",
    delimiter: str = "\t",
    null_tokens: tuple[str, ...] = ("", "NA", "NaN", "nan"),
    optional_extra_fields: tuple[str, ...] = (),
)

Strict decoder for a headered delimited primary-result file.

source_names maps each declared result name to its runtime-file spelling. row_identity_fields maps transport columns such as chr/pos/ref/alt to scalar dtypes and preserves them for storage adapters that need decomposed loci. allowed_extra_fields names required runtime columns that are intentionally ignored; optional_extra_fields names ignored diagnostics accepted when present. Every other undeclared column is rejected. This makes both preservation and projection explicit binding decisions instead of silently dropping new runtime output.

decode async

decode(
    storage: Storage,
    output_uris: Sequence[str],
    *,
    batch_size: int = 10000,
) -> AsyncIterator[ResultBatch]

Stream every collected file in order and validate its complete input schema.

DetailResultBatch dataclass

DetailResultBatch(
    name: str, rows: tuple[dict[str, object], ...]
)

Rows for one binding-declared named detail table.

DetailResultCodec

Bases: Protocol

Decode collected runtime outputs for one manifest-declared detail table.

decode

decode(
    storage: Storage,
    output_uris: Sequence[str],
    *,
    batch_size: int = 10000,
) -> AsyncIterator[DetailResultBatch]

Yield bounded batches bearing the declared detail-table name.

InlineScoringResult dataclass

InlineScoringResult(
    primary_rows: tuple[dict[str, object], ...],
    details: tuple[DetailResultBatch, ...] = (),
    artifacts: tuple[ArtifactResult, ...] = (),
)

Primary rows plus lossless named details and artifacts returned by an inline binding.

artifacts holds exactly the artifacts the run's output selection names: one :class:ArtifactResult per selected :class:~altar.models.ArtifactSchema, covering the variants of primary_rows.

ResultBatch dataclass

ResultBatch(rows: tuple[dict[str, object], ...])

One bounded batch of decoded primary result rows.

ResultDecodeError

Bases: ValueError

A runtime output cannot be decoded as the binding's declared result schema.

ScoringResultCodec

Bases: Protocol

Decode collected runtime outputs into bounded, schema-shaped primary batches.

decode

decode(
    storage: Storage,
    output_uris: Sequence[str],
    *,
    batch_size: int = 10000,
) -> AsyncIterator[ResultBatch]

Yield decoded batches without loading a complete output into memory.

container_artifact_coverage

container_artifact_coverage(
    plan: ContainerScoringPlan,
    selected: Sequence[ArtifactSchema],
    batch_variant_ids: Sequence[str],
) -> Mapping[str, ArtifactFileCoverage]

Map every logical path an expanded plan classifies under a selected artifact to what that file covers.

batch_variant_ids are the batch's variants in the order of its variant file. A batch-scoped file covers all of them; the i-th per-variant file covers the i-th variant. The plan must already be expanded (:func:expand_variant_artifact_outputs) with the same batch, so a per-variant artifact lists exactly one path per variant.

expand_variant_artifact_outputs

expand_variant_artifact_outputs(
    plan: ContainerScoringPlan,
    selected: Sequence[ArtifactSchema],
    variant_count: int,
) -> ContainerScoringPlan

Return plan with each selected variant-scoped artifact's pattern expanded to variant_count files.

selected are the artifacts the request's output selection names and variant_count is the number of rows of the batch's variant file. The pattern's terminal output becomes one output per row, in row order, on the same task, and the artifact's classification lists those paths in the same order. A plan without a selected per-variant artifact, or one already expanded, is returned unchanged; a batch-scoped artifact classified under a pattern, a per-variant artifact classified under more than one pattern or file, and a pattern no terminal task declares are rejected with ValueError.

score_cohort async

score_cohort(
    plugin: ModelPlugin,
    request_template: ScoringRequest,
    *,
    candidate_variants: Transfer,
    storage: Storage,
    score_store: ScoreStore,
    model_name: str,
    max_variants_per_batch: int,
    destination_transfer: TransferMapper,
    detail_store: DetailStore | None = None,
    artifact_store: ArtifactStore | None = None,
    backend: ExecutionBackend | None = None,
    reference_fasta: str | PathLike[str] | None = None,
    input_transfer: TransferMapper | None = None,
    output_transfer: TransferMapper | None = None,
    spec_mapper: SpecMapper | None = None,
    runtime: RuntimeContext | None = None,
    reuse_policy: ScoreReusePolicy = ScoreReusePolicy(),
    cache_query_size: int = 10000,
    poll_interval_s: float = 5.0,
    timeout_s: float = 86400.0,
    progress: Callable[[str], None] | None = None,
    batch_size: int = 10000,
) -> CohortScoringResult

Prepare the cohort's cache-miss batches and execute each one, in order.

The preparation arguments are those of :func:~altar.scoring.prepare_scoring_batches; the execution arguments are those of :func:~altar.scoring.run_scoring, applied to every prepared request; an artifact_store serves both, so a request that selects artifacts counts a variant as cached only when its artifacts are stored, and stores them when it is scored. For a container plan, each batch's staged variant file is mapped into the plan's declared variant input before input_transfer sees the remaining inputs, so a caller maps only the resources it stages itself. A batch that fails stops the cohort; the batches before it are already persisted and are cache hits on the next call, so re-running the same cohort resumes at the failed one.

timeout_s bounds each batch's wait on its tasks, not the whole cohort. progress, when given, first receives how many variants are left to score and in how many batches, before anything is submitted, then :func:~altar.scoring.run_scoring's lines for each batch, prefixed with batch <n> of <total>: when the cohort has more than one batch.

executable_scoring_plan async

executable_scoring_plan(
    plugin: ModelPlugin,
    request: ScoringRequest,
    *,
    storage: Storage | None = None,
    input_transfer: TransferMapper | None = None,
) -> ScoringPlan

Return the checked plan that executes request: the one to submit, and the one its outputs are ingested by.

The binding's plan is built and its identity, genome, and declared schema are checked, without touching a store. When the request selects no variant-scoped artifact, that plan is returned as it is, with no storage read. A variant-scoped artifact is one file per variant of the batch, which the binding declares as one output at a {row} pattern (:meth:~altar.models.ScoringLayout.artifact_variant_result_pattern) because its plan is built before the batch is known. Then the batch's staged variant file is read, through storage at the location input_transfer maps the plan's declared variant input to, and the pattern is expanded to one declared output per row (:func:expand_variant_artifact_outputs), so a backend collects every file like any other output. Expansion is deterministic given the same variant file.

:func:run_scoring executes this plan and :func:ingest_scoring_results rebuilds it when it is not given one. A host that drives execution itself submits it, and can freeze it to pass to the ingest later. An inline plan cannot produce a variant-scoped artifact yet, so selecting one for an inline binding is rejected here, before any compute.

ingest_scoring_results async

ingest_scoring_results(
    plugin: ModelPlugin,
    request: ScoringRequest,
    *,
    model_name: str,
    score_store: ScoreStore,
    storage: Storage,
    detail_store: DetailStore | None = None,
    artifact_store: ArtifactStore | None = None,
    input_transfer: TransferMapper | None = None,
    output_transfer: TransferMapper | None = None,
    batch_size: int = 10000,
    plan: ScoringPlan | None = None,
) -> ScoringRunResult

Decode, validate, and persist the collected outputs of a container scoring request another driver ran.

This is the ingest step of :func:run_scoring without its execution: no backend, no submission, no polling. A host that submits the plan's tasks itself (and reconciles them with its own ledger) calls it once the terminal outputs are collected.

Without plan, the plan is rebuilt with :func:executable_scoring_plan. Plan construction is deterministic given the same binding and the same staged variant file, so it is the plan the host submitted as long as the installed binding has not changed since.

plan is the plan the host submitted, which it froze when it admitted the work, for example with :func:~altar.models.scoring_plan_to_json. The ingest then uses that plan and its identity and never rebuilds them, so an image or default the installed binding changed since does not change what the outputs are stored as. Before anything is written, the plan is checked against the installed binding and the request:

  • its plugin ID, kind, version, and result schema must be the binding's (:meth:~altar.models.ModelPlugin.validate_result_identity), and it must support the current Altar API;
  • the request must describe the same scientific run: its configuration, output selection, genome build, and genome, with the plan's runtimes, must give the plan's lineage (:meth:~altar.models.LineageDescriptor.verify);
  • a binding that reports runtime identities for a configuration (:meth:~altar.models.ModelPlugin.scoring_runtime_identities) must report the plan's for the request's;
  • a task label that names the job, model, scoring batch, or batch count must name the request's;
  • the plan's result files must name the primary result, every detail table, and the artifacts the request selects, and the plan must stage the batch's variant file.

The exact configuration_hash may differ, for example after the binding migrated saved configurations to a newer schema version. The rows carry the plan's. A mismatch raises :class:~altar.models.IncompatibleResultError, ValueError, or :class:ScoringOutputError before any store is touched.

output_transfer (the identity by default) maps each terminal output to the URI the host's backend collected it at, exactly as it would for :func:run_scoring. input_transfer locates the batch's staged variant file, the shard input at the request layout's variant-file path. It lists the variants a selected artifact covers and the variants the outputs may hold rows for: a row for any other variant raises :class:ScoringOutputError, and the batch's variants without a primary row are counted in :attr:ScoringRunResult.variants_without_rows. A plan that stages no variant file, or a file storage does not find, raises :class:ScoringOutputError, as does a missing primary or detail output.

The host owns its outputs' freshness. Before it submits the plan's tasks, it removes any file at a terminal output's location, as :func:run_scoring does. With a backend that adopts running tasks, it leaves each location the task writes in place (writes_output_in_place) to a backend that declares clears_outputs_on_fresh_start.

The outputs are then classified by logical path, decoded with the binding's codecs, validated, and routed as :func:run_scoring routes them: selected artifacts, then every detail table, then the primary rows through one :meth:~altar.results.ScoreStore.primary_write unit, last, as the per-variant commit marker. A duplicate row, or a row for a variant the manifest's variant_eligibility excludes, raises :class:ResultValidationError; on a store whose primary write is atomic, no primary row of the ingest is persisted. Ingesting the same outputs again skips the rows already stored. As in :func:run_scoring, a compact-lineage store records the run's lineage descriptor when the ingest registers its lineage, and the fields ModelPlugin.unavailable_score_fields names for the request are stored as null.

Only container plans have outputs to collect. An inline plan, rebuilt or passed as plan, executes in-process, so it raises TypeError here and runs with :func:run_scoring.

run_scoring async

run_scoring(
    plugin: ModelPlugin,
    request: ScoringRequest,
    *,
    model_name: str,
    score_store: ScoreStore,
    detail_store: DetailStore | None = None,
    artifact_store: ArtifactStore | None = None,
    backend: ExecutionBackend | None = None,
    storage: Storage | None = None,
    runtime: RuntimeContext | None = None,
    input_transfer: TransferMapper | None = None,
    output_transfer: TransferMapper | None = None,
    spec_mapper: SpecMapper | None = None,
    ledger: TaskLedger | None = None,
    poll_interval_s: float = 5.0,
    timeout_s: float = 86400.0,
    progress: Callable[[str], None] | None = None,
    batch_size: int = 10000,
) -> ScoringRunResult

Build, execute, decode, validate, and persist one model scoring request.

Inline plans need only a plugin, request, and store. Container plans additionally require an execution backend and storage adapter. A binding with detail tables needs detail_store; a request whose output selection names artifacts needs artifact_store. Transfer mappers translate binding-declared logical transfers into a deployment's concrete input/output URIs without teaching the binding about that deployment. A supplied task ledger must be scoped to this run.

This is the reference driver: it submits, polls, and cancels a container plan's tasks itself. A host that drives execution its own way ingests the collected outputs with :func:ingest_scoring_results, the same step this function runs after collection. The plan it executes is :func:executable_scoring_plan's.

Before it stages or submits anything, it deletes through storage any file already at a terminal output's mapped location. For a backend whose adopts_active_tasks is true, it skips each location the backend's writes_output_in_place says the task writes itself. The backend clears those when it launches a fresh attempt and keeps an adopted task's, so a restarted driver can collect what an adopted task wrote. Such a backend must declare clears_outputs_on_fresh_start; one that does not raises ValueError before any work starts. A terminal output that is missing after collection raises :class:ScoringOutputError. After a successful ingest it cancels every task it submitted, which releases finished containers and Jobs; a failed cancellation is logged, because the rows are already stored.

A container plan's tasks are checked every poll_interval_s seconds. When some are still open timeout_s seconds after they were submitted, the open tasks are cancelled through the backend and a :class:TimeoutError is raised; nothing of the request is stored. A backend whose cancel does nothing, such as the Modal backend, leaves those tasks running until their own limit. progress, when given, receives short lines for a person while the tasks run: one when they are submitted, naming the time limit, then the counts of running, waiting and finished tasks when they change, at most every 15 seconds, and at least once a minute while they run.

The run's variants are read from the batch's staged variant file, through storage at the location input_transfer maps it to, before any task is submitted. A plan that stages no variant file, or a file storage does not find, raises :class:ScoringOutputError. An inline plan's variants are the ones :meth:~altar.models.ModelPlugin.run_variant_ids returns. A row for any other variant raises :class:ScoringOutputError, and the batch's variants without a primary row are logged and counted in :attr:ScoringRunResult.variants_without_rows. An inline run whose run_variant_ids returns None skips both checks.

The run's :class:~altar.models.LineageDescriptor, derived from the plan's identity and the request's inputs, is recorded before compute and passed to the primary write unit, so a compact-lineage store holds it when the run registers its lineage and the lineage takes part in equivalence from then on. A store that keeps no compact lineages records nothing; a BigQuery store whose descriptor table is not provisioned records nothing and emits one :class:~altar.results.DescriptorTableNotProvisionedWarning per store instance.

The stored rows carry the request's genome_build, the assembly of its variants. When the model was trained on another assembly, the fields ModelPlugin.unavailable_score_fields names are stored as null.

prepare_scoring_batches async

prepare_scoring_batches(
    plugin: ModelPlugin,
    request_template: ScoringRequest,
    *,
    candidate_variants: Transfer,
    score_store: ScoreStore,
    storage: Storage,
    destination_transfer: TransferMapper,
    max_variants_per_batch: int,
    cache_query_size: int = 10000,
    reuse_policy: ScoreReusePolicy = ScoreReusePolicy(),
    detail_store: DetailStore | None = None,
    reference_fasta: str | PathLike[str] | None = None,
    artifact_store: ArtifactStore | None = None,
) -> ScoringPreparation

Prepare dense, cache-miss-only batches and construct their requests.

request_template identifies the model, genome, output selection, and logical layout, but is never executed. It must not already identify a batch. The function derives its expected scientific identity from a plan, verifies any stored scores are compatible, and creates final :class:ScoringRequest values only after the complete global miss set is known.

The candidate input is a canonical headerless TSV with chr, pos, ref, alt[, variant_id] columns. Its first occurrences are retained in input order and duplicate canonical loci are removed on disk. Candidates that plugin.manifest.variant_eligibility excludes are then set aside before any cache lookup: they are never scheduled, and they are staged through destination_transfer as a report at the layout's ineligible_variants_file and returned as ineligible_variants. destination_transfer must place that report at a storage location: a mapper that leaves its URI as the bare logical path, such as the identity mapper, would make storage resolve it against the process working directory, so it is rejected with :class:ScoringPreparationError before the cache is queried or anything is staged. The candidates the manifest admits are then checked, in this order, against the reference and against allele normalization, and set aside in the same way with the first failing :class:~altar.models.IneligibilityReason. With reference_fasta, the indexed FASTA of the run's genome build, a variant whose canonical chromosome names no contig is contig_missing, one whose REF is not the reference bases at its position (compared case-insensitively, so soft-masking matches) is reference_mismatch, and one that :func:altar.variants.left_align would rewrite is not_normalized. Without a reference only the reference-free half of that normalization, :func:altar_identity.minimal_representation, is checked. The reference is a run input, not model configuration: it enters no identity and its digest is not recorded here, since the plan already carries the genome as a content-addressed resource. Cache queries and batches are bounded by cache_query_size and max_variants_per_batch respectively; the complete cohort and miss set are held in a temporary SQLite relation rather than process memory.

Both plan shapes share that lifecycle and differ only in how a batch reaches its request. For a container plan each batch is staged as a variant file through destination_transfer and returned as the batch's variant_input. For an inline plan each batch is bound into a request's run inputs through :meth:~altar.models.ModelPlugin.with_run_variants, whose template may carry any run variants (they are ignored), and variant_input is None. A binding that scores inline without overriding that hook is rejected with :class:CapabilityNotSupported.

A binding that declares named detail tables requires the detail_store that :func:~altar.scoring.run_scoring will write to. A variant is then a hit only if its primary row is cached under an accepted lineage and every declared detail table has at least one row for it under an accepted lineage; otherwise it is prepared again and the engine's key-level deduplication skips rows that already exist. The resolved lineages apply to details exactly as to primary scores: the current run's lineage comes first and its stored table must have the same scientific meaning, and each other lineage's table counts only when it is the same named table at the same declared version with the same row identity and field meanings (:meth:~altar.results.StoredDetailTable.supplies); a lineage whose table is absent or differs supplies nothing, so its variants are prepared again rather than paired with details of another shape. A variant that legitimately produced zero rows for a table is a hit when the schema declares a :attr:~altar.models.DetailSchema.row_count_field and the cached primary row holds 0 there; without that signal it remains a miss on every preparation, which costs compute but never loses rich output.

A request whose output selection names artifacts requires the artifact_store that scoring will write to, and a variant is then a hit only if, besides the above, every selected artifact has a record covering it under an accepted lineage whose artifact is the same named artifact at the same declared version (:meth:~altar.results.StoredArtifact.supplies). Artifacts are batch-scoped and content-addressed, so a re-prepared miss is scored again as part of a new batch's file; the records of the earlier file remain.

Before the first cache query, a score store that keeps compact lineages records the run's :class:~altar.models.LineageDescriptor (derived from the template plan's identity and the request's inputs, whether or not any variant is then scored) and resolves the run's lineages once, :meth:~altar.results.ScoreStore.resolve_lineages with that descriptor and plugin as the equivalence rules: the current lineage, then every registered lineage the binding's equivalence_projection declares equivalent, then reuse_policy's accepted lineages, or exactly the lineages of an exact policy (:meth:ScoreReusePolicy.exactly). The primary, detail, and artifact checks all read that one resolution, and it is returned as the receipt's resolution for reading the job back with ScoreReusePolicy.exactly(resolution.lineage_ids). With the default identity projection no other lineage is equivalent, so the checks read what reuse_policy names, as before.

Warehouse deployments may implement the same lifecycle with a relation-native export while preserving the returned request semantics, including the eligibility filter.