Read a run's results¶
altar run writes one table with a row per variant, a summary of what the run did, and a record of which data
and settings each column came from. A model's detail tables and artifacts stay in the run's stores. A separate
command adds the results to the input VCF as INFO fields.
Before you start¶
- A finished run. The examples use the quickstart's runs, whose files are in a directory
named after each
job_id:first-annotationfrom Annotate variants andfirst-scoresfrom Score variants. - pandas or pyarrow to read Parquet. Both come with Altar.
Read the export¶
The export directory is <work_dir>/export unless [export] destination names another. It holds the table, a list
of the table's columns and a copy of the run summary:
Read the directory with pandas or pyarrow. Both skip _columns.tsv and _run_summary.json, because their names
start with an underscore:
The table has one row per variant of the cohort, sorted by variant_id. Its columns are:
variant_id, the variant aschr:pos:ref:alt;prioritized, true when a model or a prioritizing source flags the variant;- each source's columns, such as
gnomad_afandclinvar_classification. Each source's page lists them; - each model's score columns and its own
prioritizedcolumn, under a prefix of the model key and a short hash, such aslegnet_wtc11__60a71072bf29_activity_delta. A store that keeps lineages adds the model'sscore_lineage_id, which names the set of score-changing settings the row's scores came from; prioritized_model_idsandprioritized_source_names, the models and sources that flagged the variant.
An empty value means the source holds no record of the variant, or the model did not score it. It is not a zero. What "prioritized" means describes the flags.
Find a model's columns¶
_columns.tsv lists the table's columns in order, one row per column. Its columns are:
| Column | What it holds |
|---|---|
column |
The column's name in the table. |
kind |
variant_key (variant_id), annotation (a source's column), score (a model's score), prioritization (a prioritized flag or one of the two lists of what flagged the variant), lineage (a model's score_lineage_id), or display (a column a BigQuery display rider adds). |
field |
The value's own name: a score's name, such as activity_delta, which every model of one binding shares; a source's column name; or the column's name otherwise. |
dtype, repeated |
The value's type, float, int, bool, str, json or struct, and true when the column holds a list of it. |
label |
A score's or a source column's display label. |
source |
The source of an annotation column. |
model_id, model_name |
For a model's column: the model key and its display name. |
plugin_id, plugin_version |
For a model's column: the binding that scored it, and its release. |
result_schema_version, result_schema_hash |
For a model's column: the version and hash of the binding's score columns. |
A model column's row, from the scores run:
column legnet_wtc11__60a71072bf29_activity_delta
kind score
field activity_delta
dtype float
model_id legnet-wtc11
plugin_id org.kundajelab.altar.model.legnet
…
Select one model's scores by its key, or one score across every model by its name:
import pandas as pd
columns = pd.read_csv("first-scores/export/_columns.tsv", sep="\t")
table = pd.read_parquet("first-scores/export")
legnet = columns.query("model_id == 'legnet-wtc11' and kind == 'score'")["column"]
deltas = columns.query("field == 'activity_delta'")[["column", "model_id"]]
scores = table[["variant_id", *legnet]]
The list depends only on the table's columns. In Python, altar.results.wide_column_manifest returns it for a
table you write yourself; see Export results.
Choose the format¶
[export] format = "tsv" writes gzipped, tab-separated text with a header row, part-000000000000.tsv.gz, in
place of Parquet:
Parquet keeps list and record columns, such as ClinVar's conditions and genes, as lists. TSV writes them as JSON text, and the run warns that one such cell can exceed a spreadsheet's cell limit:
altar run: warning: the TSV holds clinvar_conditions, clinvar_conflicting_classifications, clinvar_molecular_consequences, clinvar_origins, clinvar_genes as JSON text, which can exceed a spreadsheet's cell limit; leave [export] format out for Parquet, which keeps them as lists
[export] prioritized_only = true keeps only the prioritized rows. The run spec reference
lists the export keys.
Long format for many models¶
A run with hundreds of models gives a table with thousands of columns. [export] layout = "long" writes the same
values as a few narrow tables, with a row per variant and model in place of a block of columns per model:
Use it when a run has many models, or when you load the results into a database or a data frame and filter by model. Keep the default wide table for a few models, a spreadsheet, or an augmented VCF, which reads the wide table only. The long format is provisional.
The export directory then holds one directory per table, each with one part, part-000000000000.parquet, or
part-000000000000.tsv.gz with format = "tsv":
$ ls first-scores/export first-scores/export/scores
first-scores/export:
_provenance.json _run_summary.json models scores variants
first-scores/export/scores:
legnet_v0_0_1__15afe8e41575
| Table | One row per | Columns |
|---|---|---|
variants |
variant | variant_id, prioritized, each source's columns, prioritized_model_ids and prioritized_source_names: the wide table's columns other than the models' |
models |
model | model_id (the model key), model_name, column_prefix (the prefix of its columns in the wide table), plugin_id, plugin_version, result_schema_version, result_schema_hash, table (the score table its rows are in), lineage_id (the lineage the run requested for it), and registry, registry_id and registry_entry_sha256 for a model named by a registry entry |
scores/<table> |
variant and model | variant_id, model_id, score_lineage_id on a store that keeps lineages, the binding's score columns under their own names, such as activity_delta, and the model's prioritized |
Models of one binding and one version of its score columns share a score table, so 500 ChromBPNet models write one
table with a logfc column. The table's name is the binding's name, its score columns' version and a short hash,
such as chrombpnet_v0_0_1__6b809b253c84, and is the same in every run. A model that did not score a variant has
no row for it. Every other value is the one the wide table holds. The provenance is in _provenance.json for both
formats.
Join a score table to the models and the variants on model_id and variant_id:
import pandas as pd
export = "first-scores/export"
models = pd.read_parquet(f"{export}/models")
variants = pd.read_parquet(f"{export}/variants")
table = models.loc[models["plugin_id"] == "org.kundajelab.altar.model.legnet", "table"].iloc[0]
scores = pd.read_parquet(f"{export}/{table}")
legnet = scores.merge(models[["model_id", "model_name"]], on="model_id").merge(variants, on="variant_id")
Turn one score into a variant-by-model table:
Or query the files in place with DuckDB:
SELECT s.variant_id, m.model_name, s.activity_delta, v.gnomad_af
FROM 'first-scores/export/scores/legnet_v0_0_1__15afe8e41575/part-*.parquet' AS s
JOIN 'first-scores/export/models/part-*.parquet' AS m USING (model_id)
JOIN 'first-scores/export/variants/part-*.parquet' AS v USING (variant_id)
WHERE m.model_id = 'legnet-wtc11' AND s.prioritized;
Detail tables and artifacts stay in their stores in either layout; see
Read details and artifacts. altar.results.write_long_export writes the same tables
from Python.
Read its provenance¶
altar run prints the run summary as JSON on standard output, and writes the same JSON to
_run_summary.json in a local export directory. For the annotation run, abbreviated:
{
"job_id": "first-annotation",
"genome_build": "hg38",
"candidate_variants": "first-annotation/variants/first-annotation.candidates.tsv",
"variant_count": 5,
"models": [],
"export_uri": "first-annotation/export",
"sources": [
{
"name": "gnomad",
"section": "sources.gnomad",
"identity": {
"source_id": "org.kundajelab.altar.annotation.gnomad",
"schema_version": "1.0.0",
"release": "4.1.1-genomes",
"genome_build": "hg38"
}
},
...
],
"potential_misses": [
{
"reason": "indel_not_left_aligned",
"count": 1,
"example": "chr7:117559590:ATCT:A",
"message": "1 indel was only trimmed, not left-aligned, ..."
}
],
"dropped_alleles": {},
"provenance": {...},
"export_layout": "wide",
"column_manifest_uri": "first-annotation/export/_columns.tsv",
"export_tables": {}
}
sourcesnames the release and genome build each source served.potential_misseslists the variants a lookup may miss, anddropped_allelescounts the VCF alleles the run skipped, by reason.modelshas one entry per model: its candidate, set-aside, already-scored and scored counts,ineligible_reasonswith the set-aside count by reason, and the path of its list of set-aside variants.provenancerecords which data and settings each column came from. The export carries it too.export_layoutiswideorlong.column_manifest_urinames a wide export's list of columns, andexport_tablesmaps each table of a long export to its directory.
Read the provenance from the export¶
The export carries the summary's provenance, so a file shared on its own still says where its columns came
from:
| Export | Where the provenance is |
|---|---|
| Parquet | the file's key-value metadata, under the key altar.provenance |
| TSV | _provenance.json, beside the table |
| Long format, either file format | _provenance.json, beside the tables |
Read it from a Parquet file with pyarrow:
import json
import pyarrow.parquet as pq
metadata = pq.read_metadata("first-scores/export/part-000000000000.parquet").metadata
provenance = json.loads(metadata[b"altar.provenance"])
provenance is {"format": 1, "models": {...}, "sources": {...}, "export": {...}}. export holds the layout,
and the column_manifest, _columns.tsv, beside a wide export. sources maps each source to the release and
build the summary records for it. models has one entry per model key. Its model_name is the spec's
model_name display name, or the model key when the spec sets none. LegNet's entry, abbreviated and with the
hashes shortened:
{
"model_name": "legnet-wtc11",
"column_prefix": "legnet_wtc11__60a71072bf29",
"plugin_id": "org.kundajelab.altar.model.legnet",
"requested": {
"lineage_id": "sha256:9c41…",
"plugin_version": "0.0.1",
"configuration_hash": "sha256:51d0…",
"runtimes": [
{"name": "model_runtime", "kind": "container_image", "identity": "riyavsinha/altar-legnet@sha256:43a4…"}
],
"genome_build": "hg38",
"outputs": ["scores"],
"…": "…"
},
"reuse": {"scope": "current", "lineages": [{"lineage_id": "sha256:9c41…", "step": "current"}], "…": "…"},
"this_run": {"cache_hits": 4, "rows_added": 0},
"producers": [{"lineage_id": "sha256:9c41…", "step": "current", "rows": 4, "recorded_runs": ["…"]}]
}
requestedis the identity this run asked for: the configuration, the selected outputs, the plugin version, the image and the genome build.reuselists the lineages the run read stored scores from.this_runcounts the variants already scored and the rows this run wrote.producersnames the lineage each exported row came from, with the runs the store recorded under it.
The provenance holds hashes, versions and image digests, and no file path, bucket or credential. Score reuse and provenance describes each field.
Tell exports apart¶
Two runs of one model with different settings export the same column names. Compare their provenance:
- A different
requested.lineage_idfor a model means its scores came from different score-changing settings. - The
score_lineage_idcolumn names the lineage of each row's scores. - A different
releaseorgenome_buildinsourcesmeans a source served different data.
Read details and artifacts¶
By default, a model's detail tables are in a SQLite file beside its store, scores.details.sqlite for
scores.sqlite, and artifacts are files under <cache_dir>/artifacts, with an index beside them. Both are in the
cache, .altar, when the spec names no store. Choose where results are stored sets other places. Read
both in Python. model_id is the model key, here sei, as in
Select details and artifacts:
import asyncio
from altar.execution import LocalStorage
from altar.results import SqliteDetailStore, StorageArtifactStore
async def main() -> None:
with SqliteDetailStore(".altar/scores.details.sqlite") as details:
async for row in details.read_rows(model_id="sei", detail_name="sequence_class_effects"):
print(row["variant_id"], row["class_label"], row["class_effect"])
with StorageArtifactStore(LocalStorage(), ".altar/artifacts", ".altar/artifacts/index.sqlite") as artifacts:
async for record in artifacts.read_records("sei", "chromatin_profile_effects"):
print(len(record.variant_ids), "variants in", await artifacts.locate(record))
asyncio.run(main())
Both read every lineage the store holds for that model key unless accepted_lineage_ids limits them. Each model's
page describes its detail columns and artifact layout. Store and assemble results covers
the stores in full.
Write an augmented VCF¶
The export is a table of normalized variants, so it cannot rebuild the input VCF's records, samples and
genotypes. When a run reads a VCF, you can get that VCF back with the run's values as INFO fields: pass the export
and the VCF to altar.variants.export.run_export. The command below assumes a run like the quickstart's
score.toml, with path = "cohort.vcf" in place of cohort.tsv:
python -m altar.variants.export.run_export \
--export-dir first-scores/export \
--input-vcf cohort.vcf \
--reference-fasta hg38.fa \
--output-vcf cohort.altar.vcf.gz
The command:
- reads the VCF against the FASTA, as the run did, and gives every ALT its record and its canonical variant ID;
- joins the export's rows onto those ALTs by variant ID;
- writes
cohort.altar.vcf.gz, its CSI index andcohort.altar.manifest.json, and prints a summary as JSON.
The run must have read the same VCF with [variants] reference_fasta set to the same FASTA, so both left-align
indels the same way. The command refuses an export whose run had no reference. The export must be Parquet. The
output name must end in .vcf.gz.
The INFO fields come from the export's sources, such as ALTAR1_GNOMAD_AF and ALTAR1_CLINVAR_*, each with the
release and build the export recorded, plus Altar's core fields, such as ALTAR1_STATUS and
ALTAR1_PRIORITIZED. A source with no INFO preset adds no field, and the summary lists it under
sources_without_preset. Write an augmented VCF lists the
presets, the core fields and how to declare more fields with --annotation-fields-json.
Check the result¶
- The table has as many rows as the summary's
variant_count, or fewer withprioritized_only. _columns.tsvhas one row per column of the table.- Each source in the summary's
sourcesnames the release and build you expected. - An augmented VCF's summary counts, under
occurrences, the VCF's ALTs and those that joined an export row.
If it fails¶
pd.read_parqueton a TSV export finds no Parquet file. Readpart-000000000000.tsv.gzwithpd.read_csv(path, sep="\t").- The augmented-VCF command refuses an export whose run read no reference. Add
[variants] reference_fasta, rerun, and pass the same FASTA.
Fix a run that stops or warns covers messages from the run itself.
Next¶
- Interpret variants explains a model's prediction for a variant.
- Score reuse and provenance explains lineages and what the provenance records.
- Store and assemble results reads the score store directly from Python.
- Tutorial 3: put model scores beside evidence assembles stored scores with evidence in a notebook and compares them with a run's export.