Package and register a plugin¶
A plugin is a class in an installed package that Altar finds through a Python entry point. The entry point says which implementation is installed. A model plugin also carries a manifest, which says whether its plans and stored results are compatible with the ones Altar already holds. Each guide in this section shows the entry point for its own interface; the rules below apply to all of them.
Package layout¶
Ship a binding as its own distribution. It depends on altar, imports only Altar's public modules
(altar.models, altar.sources, altar.execution, altar.results and the others listed under
Maturity labels), and exposes its classes from its
own namespace, such as altar_example_model.
[project]
name = "altar-example-model"
dependencies = ["altar>=0.4,<0.5"]
[project.entry-points."altar.model_plugins"]
EXAMPLE = "altar_example_model:ExamplePlugin"
Pin Altar to one minor series.
Entry-point groups¶
| Group | Interface | Status |
|---|---|---|
altar.model_plugins |
model bindings (ModelPlugin) |
Stable |
altar.execution_backends |
execution backends (ExecutionBackend) |
Stable |
altar.score_stores |
score stores (ScoreStore) |
Stable |
altar.detail_stores |
named detail-table stores (DetailStore) |
Stable |
altar.artifact_stores |
artifact stores (ArtifactStore) |
Stable |
altar.annotation_sources |
annotation sources (AnnotationSource) |
Stable |
altar.variant_gene_link_sources |
variant–gene link sources (VariantGeneLinkSource) |
Stable |
altar.storage |
object and file storage (Storage) |
Stable |
altar.job_queues |
job queues (JobQueue) |
Provisional |
altar.auth |
authentication providers (AuthProvider) |
Provisional |
altar.connections |
named connections to object storage and warehouses (Connection) |
Provisional |
altar.registries |
registries of models and genomes a package ships (PackagedRegistry) |
Provisional |
The entry-point key is the name a run spec or your code uses to select the class. For a model plugin it must equal
the class's model_type. For every other group it must equal the class's name.
Altar discovers and validates all twelve groups the same way. Four groups are provisional:
altar.job_queueshas only theInMemoryJobQueuetest double.altar.authhas only the development-onlyDevAuthProvider.altar.connectionsships thegcpandfsspectypes. The run driver, the BigQuery stores and the gnomAD, ClinVar, Open Targets E2G and phyloP sources read through them.altar.registrieslists the registries a package ships: core'sgenomeand LegNet'slegnet. The other bindings still ship their own fetch commands. Publish a registry specifies the format.
Resolve by name¶
Use the typed helper for the interface you need:
altar.registry.PluginRegistry is for an application that lists every installed plugin of a group, or a test that
registers a class without installing it:
installed()andnames()read distribution metadata without importing any plugin module.get(name)imports and validates only the requested class.discover()imports every candidate and returns aPluginDiscoveryReport. It holds the valid plugins, and one diagnostic for each broken, incompatible or duplicate candidate.register(name, implementation)adds a class in code. It refuses a name that is already installed or registered unless you passreplace=True.
Two installed distributions that declare the same name are both rejected, and the error names both distributions
and their entry-point values. An application that lists plugins should show discover().failures in its
diagnostics, so one broken optional plugin does not hide the healthy ones.
Discovery checks¶
The registry checks every class it loads against its group's base class and name attribute. A model plugin must also:
- declare a manifest of kind
modelthat supports the running Altar API; - include the
materialize_resultscapability, which the model result contract needs; - use its
model_typeas its entry-point key; - use a manifest plugin ID that no other installed plugin uses.
The manifest¶
Every model plugin declares an immutable PluginManifest. model_type is only the discovery key. The manifest is
the record of the binding that Altar saves with plans and results. It holds:
plugin_id, a globally unique reverse-domain name, and the plugin kind, capabilities andarchitecture;plugin_version, the binding's release, andaltar_api, the range of Altar plugin API versions it supports;configuration_schema_version, the format version of saved configurations;result_schema, the ordered primary result fields with their version, and optionaldetail_schemasandartifact_schemas;prioritization, the name and version of the default prioritization rule, and the annotation columns it reads;runtimes, the named runtime roles, each with an identity policy;- optionally,
variant_eligibility: the variant classes and the longest allele the model can score.
PluginManifest.to_json() writes canonical JSON. Object keys are sorted, separators are fixed, non-finite numbers
are refused, unordered sets are sorted and ordered sequences keep their order. A manifest without a
variant_eligibility declaration leaves it out of the JSON.
Add a model binding builds a manifest field by field.
Runtime identity policies¶
Each runtime role declares how its identity is recorded:
record_exactaccepts a tag or a provider identifier and records the exact value used.digest_requiredaccepts only an immutablename@sha256:...image. A mutable tag fails before submission. Use it for a published container binding.declared_exactis for a provider-managed service. The manifest declares the service's stable public identity, and the binding changes it when the service's model changes.
A model runtime receives staged local paths. It never reads a storage provider itself.
Result identity¶
Building a plan produces a PluginRunIdentity. It travels with every task of a container plan and with an inline
plan, and run_scoring() passes it to the store with the rows. It records:
- the plugin ID and version, the supported Altar API range and the API version the plan was built with;
- the hash of the result schema;
- the output reducer and the exact runtime identities, such as the image digest;
- a hash of the complete validated configuration, and a hash of its scientific part;
- a hash of the output selection;
- the genome build and, when the model reads a reference, a hash of the reference genome's identity.
It leaves out the job ID, the model ID, the variants, batch boundaries, paths and runtime clients. Many jobs and batches therefore fill one cache. It also leaves out prioritization: changing a threshold changes no stored score, and assembling the results again applies the new rule.
The result-schema hash covers each field's name, type, meaning and nullability, in order. Display labels and the schema's version string stay in the manifest but do not enter the hash. Reordering fields changes it.
What goes into the configuration¶
Put every setting of the model instance that can change a score into the ModelConfiguration: checkpoints,
resource digests or revisions, tissue and ontology filters, masking and sequence windows. Altar hashes it.
- Name each resource with a
ResourceReferencethat carries a SHA-256digest. The digest decides identity, so the same bytes served from another URI keep the cache. Without a digest, the URI and an immutablerevisiondecide it. A selector such as a file name stays part of the identity, so a multi-file selector needs a digest of the whole bundle. Every field aResourceReferencesubclass adds is part of the identity. Keep location details such as a region in theuri. - Adding a digest to a resource that had only a URI changes its identity once.
- Put a credential in a
SecretReferencefield. Altar leaves every field whose type containsSecretReferenceout ofModelConfiguration.scientific_dict(), at any depth and even when its value isNone. Don't combine a credential and a scientific choice in one union-typed field. - Supply the reference FASTA as
run_inputs.genome, with its digest, whenever sequence bytes affect scores.scoring_run_identity()takes the validatedrun_inputsand records the genome itself.
When stored results are reused¶
Each result role has its own compatibility fingerprint, a ResultCompatibility: one for primary scores and one
for each detail table. It covers the scientific configuration, the output selection, the genome, the reducer, the
exact runtimes, the plugin version and that role's schema. It leaves out exact configuration provenance,
credentials, labels, descriptions and the Altar API version.
A store keeps every exact identity that wrote compatible rows, and refuses rows with an incompatible identity
under the same model ID. A score lineage is the set of scores one computation
produced. A store that keeps compact score lineages keys each lineage on the run's scientific
part, PluginRunIdentity.scientific_dict(). A new plugin release or image starts a new lineage. A caller accepts
an earlier lineage explicitly, through ScoreReusePolicy; Altar never infers it from a version number. See
Score reuse and provenance for what a reader sees, and
Reuse scores across model updates for the lineage
rules.
To assemble each variant's results, materialize() takes the identity the current configuration expects, as a
ScoredModel. The store checks the recorded identity against it before decoding any row, and raises
IncompatibleResultError on a scientific mismatch. Altar checks API compatibility when a plugin is instantiated, when a
run identity is built, when a saved container task is rebuilt for execution and before results are read.
Versioning¶
Use plugin_version for every binding release, including releases that change values. It is the only release
counter in score identity. configuration_schema_version and the result schema's version describe saved formats.
Change them when a format needs a different reading, not with every release.
These changes are additive, when old consumers stay correct:
- widening the supported
altar_apirange after running the conformance suites against the new version; - adding a capability that changes no existing plan or result;
- adding an unrelated detail table or artifact, or changing labels and descriptions;
- widening
variant_eligibility. Eligibility selects inputs and never changes an eligible variant's score, so it is not part of the run identity; - adding an optional configuration field whose default keeps the old meaning;
- fixing the implementation without changing produced values.
These change results, and need a new plugin_version:
- adding, removing, renaming or reordering a result column;
- changing a column's type, nullability or biological meaning;
- changing units, normalization, reduction, sign convention or what a missing value means;
- changing how
to_model_scoredecodes a stored value; - narrowing
variant_eligibilityso it excludes variants an earlier release stored valid scores for. Those rows keep their identity and stay readable, so bumpplugin_versionor purge them.
When result fields change, also bump the result schema's major version and ship the new schema hash. Don't rewrite an old manifest in place. Keep the old release available to read old rows, or give users an explicit migration that reads with the old identity, transforms the rows, validates them against the new schema and records the new identity. Document scientific changes in the release notes, and accept a previous lineage explicitly when its scores can be reused.
Changing a threshold, a predicate, the policy's meaning or its required annotations is a prioritization change. It leaves the score identity unchanged. Compatible scores can be prioritized again under the new rule, and results assembled under the old rule must be assembled again.
Keep imports light¶
A binding must import without its scientific runtime or provider credentials. Put TensorFlow, PyTorch, model weights and conflicting scientific environments in a runtime image. Import an optional cloud or model SDK inside the method that needs it.
Validate the package¶
Subclass the conformance suite for your interface from altar.testing. Run PluginDiscoveryContract after
installing the package, so it checks the real entry point. Test in a clean environment that holds only the package
and its declared dependencies. Validate an implementation lists every suite.
Next¶
- Validate an implementation runs the conformance suites against your package.