Score reuse and provenance¶
A run keeps each model's scores in a store and reuses them in later runs that ask for the same computation. Each export records which computation and which dataset release produced its values.
When a score is reused¶
A run reuses a stored score when the store holds a score for the same variant, under the same model key, from the same computation. The computation is defined by everything that can change the number:
- the model's weights and other resources, identified by the SHA-256 digest of their contents;
- every model setting that changes scores, such as a cell type, a tissue or ontology filter, a sequence window or averaging with the reverse strand;
- the outputs selected, and the reduction that turns many tracks into one score;
- the genome build of the variants, and the reference genome's digest for a model that reads the sequence;
- the binding's plugin version, which the binding raises when its scoring logic changes;
- the exact runtime image, or the hosted service's declared identity.
These do not change the computation, so a run that differs only in them reuses the scores:
- the job ID, the batch a variant was scored in, and which other variants were in the cohort;
- where the files are: the same weights served from a new location, identified by the same digest;
- credentials, and the connection a run reads data through;
- the prioritization rule and its thresholds;
- a newer Altar release;
- a new package release of the binding that keeps its plugin version and image.
A prioritization rule is applied when results are assembled, after scoring. Changing a threshold therefore re-evaluates the stored scores and runs no model.
The scores stored from one computation are a lineage. When a setting changes, the new scores are a new lineage
beside the old one. A store that keeps lineages holds both, and each run reads the lineage of its own settings. A
SQLite store that altar run creates keeps lineages. A store without lineages refuses a changed setting under the
same model key. The quickstart shows this in Change a setting.
A new plugin version or a new runtime image starts a new lineage, and the old scores are not reused. Altar does not assume that two plugin versions or two images compute the same numbers. A binding can declare two of its lineages equivalent, and your own code can accept an older lineage on purpose. See Reuse scores across model updates.
A store never mixes values from different computations under one model key without recording which is which. A score whose recorded computation does not match the one a run asks for is refused, not reinterpreted.
Annotation sources are looked up again in every run. Their columns record the release and genome build each source served.
What provenance records¶
Every export records what produced it. For each model, keyed by its model key, it records:
- the model's display name (the spec's
model_name, or the model key when the spec sets none), the binding's plugin ID and the column prefix of its export columns; - the computation the run asked for: its lineage ID, the plugin version, the configuration hashes (complete and scientific), the reducer, the output selection and outputs, the result schema's hash and version, the runtime images and the genome build;
- the lineages the run read, and whether each is the current one, one the binding declares equivalent, or one accepted on purpose;
- for each lineage the exported rows came from, how many rows it contributed and the computations the store recorded writing under it;
- how many variants were already scored and how many rows this run added.
A variant the run found already scored is attributed to the lineage that stored it, not to this run. A store that keeps no lineages records none, and the producers of its rows are unknown.
For each annotation source, the export records the source's ID, schema version, release and the genome build it served. Read its provenance shows where these records are and how to read them.
The export does not record everything a reader of your analysis needs. Keep these with your results:
- the package versions of Altar and of each binding, which the export does not record;
- the run spec, including each model's settings and each prioritization rule;
- the reference genome and its digest;
- for a hosted model, the provider, any model or API version the provider reports, the outputs and tissues selected, and the date of the request;
- the checksums of any data files you supplied.
Prefer references that cannot change: images named by digest, datasets pinned to a release, and resources named by their checksum. A floating image tag or a hosted service's default can change while the binding's plugin version stays the same. A hosted service may not expose a fixed model version. Then say so in your methods, and do not claim the results can be reproduced exactly.
The export holds each model's primary scores. A model that reduces many tracks to one score can keep each track's value in a detail table, or in an artifact you select. Its model page says which; see Where results live.
Next¶
- Limitations lists what the scores and annotations cannot tell you.
- Reuse scores across model updates accepts an older lineage on purpose.