Use Altar from Python¶
Every altar command calls the altar Python library, and your own code can make the same calls. Start here to run a
spec, look up evidence and read what a run stored. The later pages of this guide cover code that ingests variants and
drives scoring, execution and storage itself.
Install Altar and the bindings you use as described in Install Altar.
Every example here is a complete script. Altar's calls are async, so each script runs them with asyncio.run. In a
notebook, await the call directly instead.
| You want to | Use | Page |
|---|---|---|
| run a TOML or JSON spec and get its export | altar.run.run |
below |
| look variants up in annotation sources | altar.sources.fetch_annotations |
below |
| ingest a VCF and write an augmented VCF | altar.variants |
Ingest variants and write augmented VCFs |
| score a cohort with a model you configure in code | altar.scoring |
Score a cohort |
| run container tasks on a backend you hold | altar.execution |
Run container tasks |
| store, assemble and export results | altar.results |
Store and assemble results |
| reuse scores across model updates | altar.results |
Reuse scores across model updates |
| write a prioritization rule | altar.predicates |
Write a prioritization rule |
| run and read interpretations (provisional) | altar.interpretation |
Run and read interpretations |
| serve annotations from your own tables | altar.sources |
Serve annotations from your own tables |
| run at warehouse scale | altar.results, altar.sources |
Run at warehouse scale on BigQuery |
Import from the modules the Python API reference lists; it also gives every signature. Maturity labels lists which of their names are provisional.
Run a spec¶
altar.run.run does what altar run does: it reads a spec, scores the cohort's cache misses with every model,
annotates it with every source, and writes the export. It takes the path of a .toml or .json spec, a mapping
with the same content, or a RunSpec. Run spec describes every section.
import asyncio
from altar.run import run
summary = asyncio.run(run("annotate.toml"))
print(summary.export_uri, summary.variant_count)
It returns a RunSummary. summary.to_dict() is the JSON that altar run prints and writes to the export as
_run_summary.json; Read a run's results explains its fields and the export's columns.
The run logs one line per stage to the altar.run logger at INFO. Call logging.basicConfig(level=logging.INFO)
to see them.
A mapping is useful when a program builds the spec. This one annotates three variants with gnomAD, read from the published copy:
import asyncio
from pathlib import Path
from altar.run import run
Path("cohort.tsv").write_text("chr7\t117559590\tATCT\tA\nchr17\t43045711\tG\tT\nchr1\t1000000\tA\tG\n")
spec = {
"run": {"job_id": "annotate-demo", "genome_build": "hg38"},
"variants": {"path": "cohort.tsv"},
"sources": {"gnomad": {"type": "gnomad"}},
}
summary = asyncio.run(run(spec))
run also takes objects a spec cannot hold:
annotation_sources: sources built in code, added to the ones the spec declares. The run leaves them open; you close them.runtimes: an inline model's runtime context, keyed by its model key (the<name>in[models.<name>]). A hosted model reads its API key through the runtime.connections: connections built in code, replacing the spec's connections of the same name.
altar run resolves an API key named in a spec from the environment. In Python, give the model's runtime a
resolver yourself. EnvironmentSecretResolver is provisional:
import asyncio
from altar.access import EnvironmentSecretResolver
from altar.run import run
from altar_alphagenome import AlphaGenomeRuntime
runtime = AlphaGenomeRuntime(secret_resolver=EnvironmentSecretResolver())
summary = asyncio.run(run("alphagenome.toml", runtimes={"alphagenome": runtime}))
Check a spec without running it¶
altar.run.validate makes every check a run makes before it reads the cohort and returns a RunPlan, the same
report as altar run --validate. load_spec reads a file into a RunSpec, and parse_spec checks a mapping. All
three raise RunSpecError, which names each field to fix. validate and RunPlan are provisional.
from altar.run import RunSpecError, validate
try:
plan = validate("annotate.toml")
except RunSpecError as error:
print(error)
else:
print(plan.to_dict())
altar.run.interpret runs a spec's interpretation sections; see
Run a spec's interpretations.
Look up evidence¶
An annotation source answers per-variant lookups. Build a source the way a spec does with its class's
from_options(), which takes the same options as [sources.<name>.options]. With no options, the gnomAD and ClinVar
sources read their published copies. fetch_annotations looks variants up in several sources at once:
import asyncio
from altar.sources import fetch_annotations, get_annotation_source_registry
VARIANTS = ["chr7:117559590:ATCT:A", "chr17:43045711:G:T", "chr1:1000000:A:G"]
async def main() -> None:
registry = get_annotation_source_registry()
with registry.get("gnomad").from_options() as gnomad, registry.get("clinvar").from_options() as clinvar:
rows = await fetch_annotations([gnomad, clinvar], VARIANTS)
for variant_id, columns in rows.items():
print(variant_id, columns.get("gnomad_af"), columns.get("clinvar_classification"))
asyncio.run(main())
chr7:117559590:ATCT:A 0.00788405 Pathogenic
chr17:43045711:G:T None Pathogenic
chr1:1000000:A:G None None
The registry finds an installed source class by its spec type, here from altar-popfreq and altar-clinvar.
Importing the class from its binding, such as altar_popfreq.GnomADSource, works the same way. A source built in code
is yours to close, so the example opens each in a with block.
fetch_annotations returns an entry for every ID you pass. A source with no row for a variant adds no columns to its
entry: a miss is an absent key, never a null or a zero. source.annotate(variant_ids) on one source leaves the
variant out of its result instead. IDs must be canonical (chr7:117559590:ATCT:A). A lookup trims each ID to its
minimal representation but does not left-align it, so left-align indels against the reference first; see
Build and check variant IDs. Each source's page under
Models and data lists its columns.
A variant–gene link source, such as Open Targets E2G, answers a different
question: which genes a locus may regulate, with one link per gene and biosample. It pages through links with
query_links, or reads them as per-variant annotation columns through VariantGeneLinkAnnotationSource. The
Open Targets E2G and
ENCODE-rE2G atlas pages show both.
To serve annotations from a table you maintain, see Serve annotations from your own tables. To write a source class, see Add an annotation source.
Read what a run stored¶
altar run writes three things a Python program can read:
- The export, one row per variant, in
export/of the run directory,[run] work_dir, which is thejob_idby default. Read it withpandas.read_parquetorpyarrow.parquet.read_table; Read a run's results describes the columns. - The score store, the
[store]the spec names, orscores.sqlitein the cache,[run] cache_dir, when it names none. It keeps every model's scores across runs. - The detail and artifact stores, which hold a model's rows that do not fit one row per variant. Read details and artifacts reads them back.
import pandas as pd
table = pd.read_parquet("annotate-demo/export")
print(table[["variant_id", "prioritized", "gnomad_af"]])
The export is the simplest read, and it records the provenance of every column. Read the score store directly to assemble results for other variants or models than one run exported. Assembly needs each model's run identity, which a scoring call returns; Assemble variant results shows it.
Next¶
- Ingest variants and write augmented VCFs turns a VCF into the canonical variants to score.
- Score a cohort scores variants with a model you configure in code.
- Python API reference lists every signature.
- Tutorial 0: look up what is known about five variants runs a spec and looks variants up from Python in a notebook.