Models, sources and bindings¶
A run gets its values from two kinds of component. A model computes a score for each variant during the run. An annotation source looks each variant up in a dataset that already exists. Both come from bindings: Python packages, installed separately from Altar, that each connect one model or dataset to it.
Model instances¶
A model architecture is a modeling approach, such as ChromBPNet or Enformer. A model instance is one trained model with a fixed biological scope: its weights, the cell type or tissue it models, its folds, the output tracks it reads and how it reduces them to one score. A K562 DNase ChromBPNet model and a HepG2 one are two instances of one architecture.
A run spec declares each instance in a [models.<name>] section. type names the architecture, and the section's
configuration sets the scope. The <name> is the model key. The store keeps the instance's scores under it, and
the run summary and the export's provenance name the instance by it. Two instances in one spec need two keys.
Every setting that can change a score is part of the instance: the weights, identified by their digest, the cell type, the sequence window, the selected outputs and the reducer. A setting that changes only where or how the work runs is not, such as where the weights are stored, the batch size or the backend. Score reuse and provenance lists both kinds.
How a model runs¶
A model runs in one of two ways:
- In a container. The model's runtime is a container image that holds its framework, such as TensorFlow or
PyTorch, and the program that scores a batch. The binding names the image by digest, as
name@sha256:<digest>, and refuses a tag, which can move to another image. The execution backend pulls the image and runs the tasks. ChromBPNet, Cherimoya, Enformer, Borzoi, LegNet and Sei run this way. - Through a hosted API. The binding calls the provider from the
altarprocess, with your API key. AlphaGenome runs this way and needs no Docker.
Your Python environment holds Altar and the bindings. A container model's framework is only in its image.
Weights and other resources¶
A model reads its weights, and sometimes other files such as a peak set, from files you fetch once. Each file is
identified by its SHA-256 digest. A registry lists models by their saved configuration and files, and altar
resources fetch downloads an entry's files, such as LegNet's legnet/wtc11, and checks their digests; a model
section names the entry with resources.
Other container models' bindings have their own resource command, and a model section names the manifest it writes
with resource_manifest. Each model's page says where its files come from. Before a task runs, each file is
checked against its digest.
The digest identifies the file, not its location. Weights moved to another disk or bucket keep their digest, and the stored scores are reused. Different weights have a different digest, and their scores are stored as a new set.
Some models also derive files from their weights before they score, such as ChromBPNet's per-fold calibration. Prepare model resources runs that step.
Annotation sources¶
An annotation source is a dataset that a run looks variants up in, together with the installed code that reads
it, such as gnomAD or ClinVar. A lookup runs no model. Each source records the release and the genome build of the
data it served. Many sources read the copy their publisher hosts, with no download, and altar pull copies one
into storage you own; see Choose where data is read from.
Evidence that runs no model is a source, not a model, even when the values are a model's predictions. Precomputed SpliceAI, AlphaMissense and GPN-Star scores are looked up, so they are annotation sources and record a dataset release. To decide which a dataset is, ask whether inference runs during the analysis.
A run spec declares each source in a [sources.<name>] section, and altar sources lists the installed ones. A
source can declare its own prioritization rule, such as ClinVar's prioritize_pathogenic.
Variant–gene link sources¶
A variant–gene link source says which genes a regulatory element that the variant overlaps is predicted to
regulate. It holds one link per element, gene and biosample, so one variant can have many. A run spec declares it
in a [sources.<name>] section like any other source, and it adds each variant's links to the export. Open Targets
E2G and the ENCODE-rE2G and scE2G atlases are link sources. From variants to
evidence describes what a link keeps and the columns it adds.
Bindings¶
A binding is a Python package that connects one model, dataset or service to Altar, such as altar-legnet or
altar-popfreq, which serves gnomAD. Installing a binding makes its models or sources available to a run:
altar models and altar sources list them, and a section's type names one.
A model binding provides:
- the configuration the model accepts, and the checks on it;
- the score columns, each with its meaning, units and direction;
- the model's default prioritization rule;
- the plan for a batch, and the code that reads its tasks' output files;
- for a container model, a default image pinned by digest, and often a resource command that fetches the official weights.
A source binding provides the source's columns, the code that looks variants up, and the release it serves. Many
also provide a builder command, such as altar-popfreq-build, that writes your own copy of the dataset.
A binding records a plugin version, which it raises when its scoring changes. A new plugin version starts a new set of stored scores; see Score reuse and provenance.
Each binding's page under Models and data gives its science, settings and limits. To write a binding, see Choose an interface.
Next¶
- From variants to evidence describes what a score, an annotation and a link each mean.
- Score variants with a model adds a model to a run spec.