Skip to content

Maturity labels

Altar is in beta, at version 0.x. Every module, command, run-spec section and backend of the altar and altar-identity packages carries one of three maturity labels. The tables below list which has which, and which import paths are public.

Labels

Label What it means
Stable Works and is documented. Its names and behaviour are settled.
Provisional Works and is documented. Its names and behaviour are still being designed.
Preview Works and is documented, but has been tested only against fakes, emulators or some real conditions. The table below says which.

What has which label

What Label Notes
altar.models Stable Except the provisional names listed below
altar.run Stable Except the preparation, validation, interpretation and per-source summary names below
altar.scoring Stable Except ScoringOutputError
altar.execution Stable Except the job-queue names, MissingTaskStatusError, and the GPU pool check (ExecutionBackend.gpu_pools, require_gpu_pool, UnknownPoolError)
altar.results Stable Except the BigQuery export helpers below
altar.sources Stable Except the published-dataset, builder, link-dataset and position-score names below
altar.predicates Stable Except SqlDialect
altar.registry Stable
altar.variants Stable Except the streaming-ingest schemas and the augmented-VCF names below
altar.testing Stable Except the suites and test doubles below
altar_identity Stable See Identity package
altar.interpretation, altar interpret, altar interpretations, [interpretations.<name>] and [interpretation_store] in a run spec Provisional Every name
altar.connections, [connections] in a run spec Provisional Every name
altar.access Provisional Every name
altar.registries, altar resources, the registry format, [registries], [variants] reference and [models.<name>] resources in a run spec Provisional Every name
altar run, altar spec-schema, altar models, altar sources Stable The text they print while running is for people to read. A program reads the export and the run summary
altar run --validate, altar prepare Provisional They run through provisional names of altar.run
altar pull Provisional It runs through the provisional published-dataset names of altar.sources
The run-spec file format Provisional A spec carries no format version
[export] layout = "long" in a run spec, and the long tables it writes Provisional Every table, column and file name
_columns.tsv, the list of a wide export's columns Provisional Its columns and the values of kind
Local Docker execution backend (local) Stable
Kubernetes and Modal execution backends Preview Tested with the execution-backend conformance suite against fake clients only; not run on a live cluster or a live Modal account
SQLite score, detail and annotation stores; SQLite and in-memory variant–gene link stores Stable
Storage adapters local, https and fsspec, and the storage artifact store Stable
BigQuery score and detail stores, BigQuery sources, and the table option of a source with a published dataset Preview Tested on the BigQuery emulator and on DuckDB; not run against live BigQuery
altar pull --bigquery Preview Tested against a fake BigQuery client only; not run against live BigQuery

The Python names of the BigQuery store and sources, such as BigQueryScoreStore, are Stable, except the provisional names listed below. Their Preview label describes how the backend was tested.

Platform support lists the test evidence for each backend, store and storage adapter.

Identity package

VariantKey and the other identity rules are defined in altar-identity (import altar_identity), a separate distribution that altar depends on. The public modules re-export its names as the same objects, so an isinstance check or an except clause behaves the same whichever path imported them:

Names Re-exported by
VariantKey, VariantIdentityError, canonical_chromosome, canonical_variant_id altar.models, altar.sources, altar.variants
has_verification_record, VERIFICATION_RECORD_SUFFIX altar.execution

These re-exports carry the same label as the rest of their module. Bindings import them through a public module, not from altar_identity. altar.variants.read_variants_frame loads a variant table as a DataFrame; altar_identity.read_variants yields VariantKey values from a container variant file.

Model runtimes import altar_identity directly, because most of them run on a Python version that cannot install Altar core. They use the names in its __all__. read_variants, read_variant_rows and batched read the variant file Altar stages for a container. sha256_file, verify_file and the verification-record helpers check staged resources.

altar-identity supports Python 3.9 and later, and its wheel declares no dependencies. altar pins it with ==, so each Altar release requires exactly one altar-identity version, and a runtime image built from the same commit runs the same identity code.

Provisional names

A provisional name is importable from a stable public module like any other name. Each module lists its provisional names in __provisional__, a tuple that is a subset of __all__:

import altar.sources

print(altar.sources.__provisional__)

The API reference pages say which names are provisional. A method or attribute of a stable class can be provisional too. Its docstring then says so, and the reference page shows that docstring. An entry-point group is provisional when the contract it registers is: altar.auth, altar.job_queues, altar.connections and altar.registries. See Entry-point groups.

Module Provisional names
altar.execution JobQueue, get_job_queue_registry (entry-point group altar.job_queues)
altar.execution MissingTaskStatusError, raised when a backend's poll returns no status for a task
altar.execution UnknownPoolError and require_gpu_pool, the check of a GPU pool name against the pools a backend defines (ExecutionBackend.gpu_pools, also provisional)
altar.access Every name: AuthIdentity, AuthProvider, DevAuthProvider (for development only), EnvironmentSecretResolver, InvalidTokenError, get_auth_registry (entry-point group altar.auth)
altar.connections Every name: Connection, FsspecConnection, GcpConnection, get_connection_registry (entry-point group altar.connections)
altar.registries Every name, among them RegistryIndex, RegistryEntry, FileReference, PackagedRegistry and get_registry_source_registry (entry-point group altar.registries)
altar.sources PublishedDataset (with its cluster_by), PublishedLocation, and the AnnotationSource.published_dataset hook that returns one; PublishedCopy, OwnedDatasetCopy and OwnedTableCopy, which PublishedDataset.select_copy returns; pull_dataset, pull_to_bigquery, PullSummary, BigQueryLoadSummary and PullProgress, behind altar pull; bigquery_release_labels and check_bigquery_release, which record and check a BigQuery copy's release as table labels; the AnnotationSource.annotation_contract hook, with which altar sources lists a class's columns without building it
altar.sources BuildOutput, DEFAULT_INPUT_ATTEMPTS, storage_option_connection and add_release_argument, with which a builder writes to a URI, retries a remote input, and takes storage options and --release from its command line; canonical_chromosome_runs, minimal_representations and the methods OpenInput.header_lines, OpenInput.tsv_batches, OpenInput.convert_batches, RecordOrder.advance_batch and PartitionedWriter.append_batch, with which a builder parses, converts and writes Arrow record batches
altar.sources write_link_dataset, LinkDatasetSummary, link_dataset_row, link_dataset_release_fields and LINK_DATASET_METADATA: the Parquet link dataset layout (one typed row per link, chromosome= partitions, 32,000-row groups sorted by start) and its writer; VariantGeneLinkAnnotationSource.from_dataset, which reads it, and from_bigquery_table, which reads it loaded into BigQuery
altar.sources POSITION_SCORE_METADATA, position_score_release_fields and PositionScoreAnnotationSource: the Parquet position score layout (one row per scored base, chr= partitions, pos strictly increasing, 128,000-row groups) and the source that reads it, with its indels option; the parquet_options keyword of PartitionedWriter
altar.sources allelic_key, which turns a canonical variant ID into the key an AllelicRecordBackend looks up, and minimal_variant_key, the trimmed key that allelic_key and the position and link sources look up
altar.sources unknown_options_message and owned_dataset_storage_options, with which a source's from_options refuses an unknown option and reads an owned dataset through a named connection
altar.predicates SqlDialect, which Predicate.to_sql and Term.to_sql take as dialect
altar.run SourceRunSummary, the per-source provenance in RunSummary.sources and _run_summary.json
altar.run ModelSectionSummary, each section whose resources lists several registry entries, in RunSummary.sections and RunPlan.sections
altar.run PotentialMiss, the counts in RunSummary.potential_misses of cohort rows a lookup may miss; validate and the RunPlan it returns, behind altar run --validate
altar.run prepare, validate_preparation, PreparationSummary and ModelPreparationSummary, behind altar prepare
altar.run interpret, validate_interpretations, InterpretationSettings, InterpretationStoreSettings, InterpretationsSummary, InterpretationOperationSummary and InterpretationProduct, behind altar interpret and the [interpretations.<name>] and [interpretation_store] spec sections
altar.interpretation Every name: InterpretableModel, InterpretationMethod, InterpretationSelection, InterpretationDescriptor, InterpretationOperation, the preparation, workflow, ingest and read types, prepare_interpretation, ingest_interpretation_results, read_interpretation, interpret, InterpretationRunResult, InterpretationRunError, InterpretationStore, StorageInterpretationStore and their key, record and row types
altar.scoring ScoringOutputError, raised when a run's output holds a row for a variant outside the submitted batch, or a terminal output was not produced
altar.results VariantIdentity, DisplayRider, RiderColumn, export_col_alias, safe_identifier, and the BigQueryScoreStore.count_unscored method, which an application uses to run the BigQuery store in production (Run at warehouse scale on BigQuery)
altar.results wide_column_manifest, ColumnManifestEntry, column_manifest_tsv, write_column_manifest and COLUMN_MANIFEST_FILE, which list a wide table's columns; write_long_export, long_score_tables, LongScoreTable and LongModelInfo, which write the long format (Export results)
altar.results ResultBuckets, ResultBucket, ResultBucketDescriptor, RESULT_BUCKET_SCHEMA_VERSION, result_bucket_schema, genomic_sort_key, bucket_export_layout, bucket_rows_for and plan_buckets, the form in which a store hands an export to the writers; render_wide_tsv, render_wide_parquet, render_prioritized_ndjson, render_long_export and render_occurrence_buckets, which write it to any storage; ExportStaging and ExportScanLimitExceeded, BigQuery's staging setting and its cost guard, and ExportDuplicateRowsError, the refusal of an export that repeats a variant (Export results)
altar.testing AuthProviderContract, ConnectionContract, InMemoryJobQueue, JobQueueContract, PublishedDatasetContract, check_registry, assert_valid_registry
altar.testing render_bucket_exports, compare_wide_tsvs, WideTsvComparison and CellDifference, which render every export layout and compare two wide downloads by variant, and compare_prioritized_ndjson, NdjsonComparison and FieldDifference, which compare two prioritized NDJSON exports by value
altar.testing InterpretationStoreContract, InterpretableModelContract
altar.testing InProcessExecutionBackend and InMemoryDetailStore, test doubles that run a model's tasks through a Python function and keep detail rows in memory
altar.testing FakeBigQueryClient, the recording client Altar's own BigQuery tests use
altar.variants OCCURRENCE_ARROW_SCHEMA, CANONICAL_ARROW_SCHEMA, INVALID_ARROW_SCHEMA, canonical_sample_selection_digest
altar.variants AnnotationInfoField, check_annotation_info_fields and augment_run_export, with the vcf_info_fields() hook a source class declares its augmented-VCF fields with (Write an augmented VCF)

These methods and attributes of stable classes are provisional, and their docstrings say so:

Member What it does
ModelPlugin.reads_reference_genome Declares that a model reads no local reference genome, so the reference stays out of its identity
ModelPlugin.prepared_run_inputs Checks the files altar prepare published against a scoring request
ModelPlugin.run_variant_ids Returns the variants an inline plan scores, so the engine can check its output rows
InterpretableModel hooks The methods a binding implements to declare interpretation methods
ExecutionBackend.adopts_active_tasks, clears_outputs_on_fresh_start, writes_output_in_place Declare how a backend treats the outputs of a task it resubmits or adopts
ExecutionBackendContract.write_task_output, task_output, start_task The hooks the backend suite uses to test output clearing
ModalExecutionBackend.remove_from_volume Deletes a stale output from a Modal Volume before a task starts
WorkflowExecutionError.completed_stages, collected_outputs, and run_model_workflow(collect_completed=...) Report and keep the stages of a workflow that finished before a failure
ModelSettings.entries, ModelSettings.section and ModelRunSummary.section The registry entries a section's resources names, and the section whose list a model was expanded from
ExportSettings.layout, RunSummary.column_manifest_uri and RunSummary.export_tables Choose the wide or long export, and name a wide export's column list and a long export's tables

Private namespaces

  • altar.plugins.*, modules below altar.testing, and other implementation modules are private. They can be imported, but they are not part of the documented interface, and a package outside Altar should not depend on them.
  • Command modules below altar.variants are not public Python imports. The names altar.variants itself exports are.
  • The package root, altar, exports only __version__.

Every stable and provisional name imports in a plain pip install altar environment, without optional cloud SDKs, model frameworks, credentials or a source checkout.

Typing scope

The altar distribution ships py.typed. Strict mypy checks the public modules, the types their exported signatures reach, and every binding. It does not check some implementation modules below altar.variants.

Next