Skip to content

Use Altar from Python

Every altar command calls the altar Python library, and your own code can make the same calls. Start here to run a spec, look up evidence and read what a run stored. The later pages of this guide cover code that ingests variants and drives scoring, execution and storage itself.

Install Altar and the bindings you use as described in Install Altar. Every example here is a complete script. Altar's calls are async, so each script runs them with asyncio.run. In a notebook, await the call directly instead.

You want to Use Page
run a TOML or JSON spec and get its export altar.run.run below
look variants up in annotation sources altar.sources.fetch_annotations below
ingest a VCF and write an augmented VCF altar.variants Ingest variants and write augmented VCFs
score a cohort with a model you configure in code altar.scoring Score a cohort
run container tasks on a backend you hold altar.execution Run container tasks
store, assemble and export results altar.results Store and assemble results
reuse scores across model updates altar.results Reuse scores across model updates
write a prioritization rule altar.predicates Write a prioritization rule
run and read interpretations (provisional) altar.interpretation Run and read interpretations
serve annotations from your own tables altar.sources Serve annotations from your own tables
run at warehouse scale altar.results, altar.sources Run at warehouse scale on BigQuery

Import from the modules the Python API reference lists; it also gives every signature. Maturity labels lists which of their names are provisional.

Run a spec

altar.run.run does what altar run does: it reads a spec, scores the cohort's cache misses with every model, annotates it with every source, and writes the export. It takes the path of a .toml or .json spec, a mapping with the same content, or a RunSpec. Run spec describes every section.

import asyncio

from altar.run import run

summary = asyncio.run(run("annotate.toml"))
print(summary.export_uri, summary.variant_count)

It returns a RunSummary. summary.to_dict() is the JSON that altar run prints and writes to the export as _run_summary.json; Read a run's results explains its fields and the export's columns. The run logs one line per stage to the altar.run logger at INFO. Call logging.basicConfig(level=logging.INFO) to see them.

A mapping is useful when a program builds the spec. This one annotates three variants with gnomAD, read from the published copy:

import asyncio
from pathlib import Path

from altar.run import run

Path("cohort.tsv").write_text("chr7\t117559590\tATCT\tA\nchr17\t43045711\tG\tT\nchr1\t1000000\tA\tG\n")
spec = {
    "run": {"job_id": "annotate-demo", "genome_build": "hg38"},
    "variants": {"path": "cohort.tsv"},
    "sources": {"gnomad": {"type": "gnomad"}},
}
summary = asyncio.run(run(spec))

run also takes objects a spec cannot hold:

  • annotation_sources: sources built in code, added to the ones the spec declares. The run leaves them open; you close them.
  • runtimes: an inline model's runtime context, keyed by its model key (the <name> in [models.<name>]). A hosted model reads its API key through the runtime.
  • connections: connections built in code, replacing the spec's connections of the same name.

altar run resolves an API key named in a spec from the environment. In Python, give the model's runtime a resolver yourself. EnvironmentSecretResolver is provisional:

import asyncio

from altar.access import EnvironmentSecretResolver
from altar.run import run
from altar_alphagenome import AlphaGenomeRuntime

runtime = AlphaGenomeRuntime(secret_resolver=EnvironmentSecretResolver())
summary = asyncio.run(run("alphagenome.toml", runtimes={"alphagenome": runtime}))

Check a spec without running it

altar.run.validate makes every check a run makes before it reads the cohort and returns a RunPlan, the same report as altar run --validate. load_spec reads a file into a RunSpec, and parse_spec checks a mapping. All three raise RunSpecError, which names each field to fix. validate and RunPlan are provisional.

from altar.run import RunSpecError, validate

try:
    plan = validate("annotate.toml")
except RunSpecError as error:
    print(error)
else:
    print(plan.to_dict())

altar.run.interpret runs a spec's interpretation sections; see Run a spec's interpretations.

Look up evidence

An annotation source answers per-variant lookups. Build a source the way a spec does with its class's from_options(), which takes the same options as [sources.<name>.options]. With no options, the gnomAD and ClinVar sources read their published copies. fetch_annotations looks variants up in several sources at once:

import asyncio

from altar.sources import fetch_annotations, get_annotation_source_registry

VARIANTS = ["chr7:117559590:ATCT:A", "chr17:43045711:G:T", "chr1:1000000:A:G"]


async def main() -> None:
    registry = get_annotation_source_registry()
    with registry.get("gnomad").from_options() as gnomad, registry.get("clinvar").from_options() as clinvar:
        rows = await fetch_annotations([gnomad, clinvar], VARIANTS)
    for variant_id, columns in rows.items():
        print(variant_id, columns.get("gnomad_af"), columns.get("clinvar_classification"))


asyncio.run(main())
chr7:117559590:ATCT:A 0.00788405 Pathogenic
chr17:43045711:G:T None Pathogenic
chr1:1000000:A:G None None

The registry finds an installed source class by its spec type, here from altar-popfreq and altar-clinvar. Importing the class from its binding, such as altar_popfreq.GnomADSource, works the same way. A source built in code is yours to close, so the example opens each in a with block.

fetch_annotations returns an entry for every ID you pass. A source with no row for a variant adds no columns to its entry: a miss is an absent key, never a null or a zero. source.annotate(variant_ids) on one source leaves the variant out of its result instead. IDs must be canonical (chr7:117559590:ATCT:A). A lookup trims each ID to its minimal representation but does not left-align it, so left-align indels against the reference first; see Build and check variant IDs. Each source's page under Models and data lists its columns.

A variant–gene link source, such as Open Targets E2G, answers a different question: which genes a locus may regulate, with one link per gene and biosample. It pages through links with query_links, or reads them as per-variant annotation columns through VariantGeneLinkAnnotationSource. The Open Targets E2G and ENCODE-rE2G atlas pages show both.

To serve annotations from a table you maintain, see Serve annotations from your own tables. To write a source class, see Add an annotation source.

Read what a run stored

altar run writes three things a Python program can read:

  • The export, one row per variant, in export/ of the run directory, [run] work_dir, which is the job_id by default. Read it with pandas.read_parquet or pyarrow.parquet.read_table; Read a run's results describes the columns.
  • The score store, the [store] the spec names, or scores.sqlite in the cache, [run] cache_dir, when it names none. It keeps every model's scores across runs.
  • The detail and artifact stores, which hold a model's rows that do not fit one row per variant. Read details and artifacts reads them back.
import pandas as pd

table = pd.read_parquet("annotate-demo/export")
print(table[["variant_id", "prioritized", "gnomad_af"]])

The export is the simplest read, and it records the provenance of every column. Read the score store directly to assemble results for other variants or models than one run exported. Assembly needs each model's run identity, which a scoring call returns; Assemble variant results shows it.

Next