Run spec reference¶
A run spec is a TOML or JSON file that altar run, altar prepare and altar interpret read. Its name must end in
.toml or .json, and its root is a table of the sections below. The user guide shows
how to write one.
altar spec-schema prints the same sections as a JSON Schema.
The file format is Provisional. A spec carries no format version.
Rules for every section¶
- An unknown section or key is refused, with the path of the offending key.
[run]and[variants]are required.altar runneeds at least one[models.<name>]or[sources.<name>].- The
<name>of[models.<name>],[sources.<name>],[connections.<name>]and[interpretations.<name>], and[run] job_id, match^[A-Za-z0-9][A-Za-z0-9._-]*$: a letter or digit, then letters, digits,.,_or-. The<name>of[backends.<name>]matches^[A-Za-z_][A-Za-z0-9_-]*$: a letter or_, then letters, digits,_or-. - A relative path resolves against the directory the command starts in, not the spec's directory. This holds for
the spec's own paths and for paths inside
optionsandconfiguration. options,configurationandrun_inputshold keys that belong to the adapter, model or source the section names.altar models <type>andaltar sources <type>list them for models and sources. The sections below list them for the backends, connections, storage adapters and stores Altar ships.
| Section | Required | Read by | Label |
|---|---|---|---|
[run] |
Yes | run, prepare, interpret |
|
[variants] |
Yes | run, prepare (reference only), interpret |
|
[models.<name>] |
A model or a source, for altar run |
run, prepare, interpret |
|
[backend] |
For a model that runs in containers; always for prepare and interpret (or [backends.<name>]) |
run, prepare, interpret |
|
[backends.<name>] |
No; in place of [backend], for models on different backends |
run, prepare, interpret |
Provisional |
[connections.<name>] |
No | run, prepare, interpret |
Provisional |
[registries] |
No | run, prepare, interpret, altar resources --spec |
Provisional |
[storage] |
No | run, prepare, interpret |
|
[store] |
No | run |
|
[detail_store] |
No | run |
|
[artifact_store] |
When a model's output selects an artifact | run |
|
[sources.<name>] |
A model or a source, for altar run |
run |
|
[export] |
No | run |
|
[interpretations.<name>] |
For interpret |
interpret |
Provisional |
[interpretation_store] |
No | interpret |
Provisional |
The Python names that read a section marked Provisional are provisional too.
run¶
The run's name, genome build and the two directories it writes to.
| Key | Type | Default | Meaning |
|---|---|---|---|
job_id |
name | Required | The run's name. It names the run directory by default, the normalized cohort, variants/<job_id>.candidates.tsv, and the run's container tasks. It does not change which stored scores a run reuses. |
genome_build |
text | Required | The assembly of the cohort's coordinates, such as hg38. The GRC names (GRCh38, GRCh37, GRCm38, GRCm39), in any case and with a patch suffix such as .p14, are read as hg38, hg19, mm10 and mm39. Every model and source is checked against it before the cohort is read. |
work_dir |
path | <job_id> in the starting directory |
The run directory: this run's export and scratch files. Nothing a later command reads is kept there, so deleting it loses no reusable work. The run creates it. |
cache_dir |
path | $ALTAR_CACHE_DIR, else .altar in the starting directory |
The cache: everything a later run reuses, such as prepared model files and stored scores. It is created only when something is written there, so a run without models never creates it. |
The run directory holds what only this run reads:
| Path | Written by | Holds |
|---|---|---|
variants/<job_id>.candidates.tsv |
run, interpret |
The cohort, normalized. |
staging/jobs/<job_id>/ |
run |
The variant batches staged for this run's container tasks, and the ineligible-variant reports. |
results/jobs/<job_id>/ |
run |
The files this run's container tasks wrote. |
export/ |
run |
The export and _run_summary.json, unless [export] destination names another directory. |
interpretation-export/ |
interpret |
The product listing, the operations and the summary. |
The cache holds what a later run reads:
| Path | Written by | Holds |
|---|---|---|
staging/models/<name>/ |
prepare |
The files altar prepare publishes, which scoring reads. |
preparation-attempts/, preparation-state/ |
prepare |
Each preparation attempt, and the record of the last complete publication. |
scores.sqlite, scores.details.sqlite |
run |
The scores and their details, when the spec has models and no [store]. |
artifacts/ |
run |
Artifact files and their index, for the storage artifact store. |
interpretations/ |
interpret |
Interpretation products and their index, by default. |
interpretation-runs/, staging/interpretations/, results/interpretations/ |
interpret |
Each execution's progress, and the files it resumes from. Keep them with the interpretation products: a rerun after a partly failed execution needs both. |
backend-data/ |
run, prepare, interpret |
The local backend's directory mounted into containers, unless the backend's options name a data_root. |
Without work_dir, a run refuses to start when <job_id> already exists in the starting directory, holds files and
is not an earlier run's. Set work_dir to a new directory, or choose another job_id. Several runs can share one
cache, so a run with a new job_id reuses what earlier runs scored and prepared.
[run]
job_id = "cohort-2026-10"
genome_build = "hg38"
# work_dir = "runs/cohort-2026-10" # default: cohort-2026-10
# cache_dir = "/scratch/altar-cache" # default: $ALTAR_CACHE_DIR, else .altar
variants¶
The cohort, and the reference genome its variants are normalized against.
| Key | Type | Default | Meaning |
|---|---|---|---|
path |
path | Required for run and interpret |
A VCF, or a canonical variant TSV: headerless, one variant per line, chr, pos, ref and alt separated by tabs or spaces, then an optional label, the rest of the line, which the run ignores. altar prepare does not read it. |
format |
"auto", "vcf" or "tsv" |
"auto" |
How to read path. auto reads a name ending in .vcf or .gvcf, optionally followed by .gz or .bgz, as a VCF, and anything else as a TSV. A BCF file is refused. |
reference |
text | None | A genome of the genome registry, by its id, such as "grch38/ncbi/no-alt-analysis-set", or its alias, such as "hg38", with or without genome/; or an entry of another registry as "<registry>/<id>", fetched with altar resources fetch into [run] cache_dir. A name whose first segment is no registry's is a genome's id. The run reads the fetched FASTA and takes its digest from the entry, so it never hashes the file. It must be a genome of genome_build. A genome that has not been fetched stops the run, which prints the command to fetch it. Matched ignoring case. Set reference or reference_fasta, not both. Provisional. |
reference_fasta |
path | None | Your own reference FASTA of genome_build. Each variant is left-aligned against it, and each model that reads the genome gets it as its genome run input. Without a reference, variants are only trimmed. |
reference_digest |
"sha256:<64 hex digits>" |
None | The FASTA's expected digest. The run checks it before it reads the cohort, and writes a verification record, <fasta>.sha256-verified, beside the file so later runs skip the hash. Needs reference_fasta. |
[variants]
path = "cohort.vcf.gz"
reference_fasta = "my-genomes/hg38.fa"
# reference_digest = "sha256:..." # the digest the first run prints
Give a run its variants covers the cohort formats and the reference.
Find and fetch models and genomes covers altar resources.
models¶
One [models.<name>] section per model. <name> is the model key: the store keeps the model's scores under it,
the run summary and the export's provenance name the model by it, and its export column prefix ends with a hash of
it. Keep it the same across runs for stored scores to be reused.
A section whose resources is a list is one model per entry instead, in the list's order, and each model's key is
its entry's registry and id joined by -, in lowercase, with each / of the id written as - and each . as
_: "lab/ENCSR685ZMP" is the model lab-encsr685zmp, and "lab/k562/dnase" is lab-k562-dnase. That is the
key a section [models.lab-encsr685zmp] that names only that entry has, so either form
reuses the other's scores. The section's key then names the group in messages, the run summary and the
provenance, not a model. Score with many models shows one.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | Required, unless resources names an entry |
The model type, exactly as altar models lists it, such as "LEGNET". A near miss gets the installed spelling suggested. With resources, the entry gives it, and a type that differs is refused. |
model_name |
text | The model key | A display name: the readable part of the model's export column prefix, <model_name>__<12 hex digits of the hash of the model key>. Characters other than letters, digits and _ become _. The export's provenance records it in each model's model_name field. |
resources |
text, or a list of text | None | A registry entry, "<registry>/<id>", such as "legnet/wtc11", fetched with altar resources fetch into [run] cache_dir. The entry gives the model's type and its saved configuration, and the run reads its files from the cache. "<registry>/<id>@sha256:<hex>" pins one entry document. An entry that has not been fetched stops the run, which prints the command to fetch it. Matched ignoring case. A list names several entries, which may come from different registries: the section becomes one model per entry, keyed <registry>-<id> as above, and its other keys apply to each; an entry listed twice, a key another section has, a type that is not every entry's, and model_name are refused. Set resources or resource_manifest, not both. Provisional. |
resource_manifest |
path | None | A resource manifest a binding's resources command writes, or one you wrote for your own weights. It gives the weights and their digests, and other fields the binding records in it. Refused for a model whose configuration does not read one. |
configuration |
table | {} |
The model's configuration fields: every field; with resources, the fields to change in the entry's configuration; with resource_manifest, the fields it does not give. altar models <type> lists them. |
run_inputs |
table | {} |
The binding's run inputs beyond genome_build and genome, which the run fills from [run] and [variants]. |
output |
table | The binding's default | Which outputs to write: outputs, a list of output names, ["scores"] unless the binding sets another default, and reducer, the name of the reduction that makes the primary scores. Naming an artifact in outputs needs an [artifact_store]. |
max_variants_per_batch |
integer, at least 1 | 10000 |
The most variants one batch of container tasks scores. It does not change the scores. |
backend |
name | The spec's only backend | The [backends.<name>] the model's containers run on, for scoring, altar prepare and altar interpret. Required for a container model when the spec declares several backends; refused when the spec has a single [backend]. Provisional. |
pool |
text, or a table of stage to text | The binding's pools | The GPU pool the model's GPU tasks ask for, in place of the binding's: a pool its backend defines. A table sets it per stage, keyed by the operations the binding runs, score, preprocess and interpret: { score = "a100" }. A stage the table leaves out keeps the binding's pool; a task that asks for no GPU stays on the CPU. Provisional. |
Two value forms recur inside configuration:
| Form | Example | Meaning |
|---|---|---|
| Resource | { uri = "...", digest = "sha256:..." } |
A file, such as weights, identified by its digest. revision may name an immutable provider revision instead. |
| Secret | { provider = "env", name = "ALPHAGENOME_API_KEY" } |
A credential altar run reads from the environment variable name. The spec holds only the name. |
A container image, configuration.image, must be pinned by digest, as name@sha256:<64 hex digits>.
backend and pool say where a model runs, not what it computes: neither is part of the model's identity, so
scores stored from one backend are reused on another. The run summary and the export's provenance record each
model's backend and pools under compute. A pool the backend does not define, a stage the binding does not run,
and a pool on a model that runs inline are refused before anything is submitted, with the names that would do.
[models.legnet-wtc11]
resources = "legnet/wtc11"
[models.sei]
type = "SEI"
[models.sei.output]
outputs = ["scores", "sequence_class_effects", "chromatin_profile_effects"]
reducer = "official_nucleosome_adjusted_sequence_class_effect_v1"
[models.alphagenome]
type = "ALPHAGENOME"
[models.alphagenome.configuration]
credential = { provider = "env", name = "ALPHAGENOME_API_KEY" }
Score variants with a model walks through adding one.
backend¶
Where container tasks run. A spec needs one when any model runs in containers, and always for altar prepare and
altar interpret. An inline model, such as AlphaGenome, needs none.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | Required | "local", "kubernetes" or "modal", or a backend another package registers. |
options |
table | {} |
The backend's own settings, below. The run passes its [storage] to the backend itself. |
poll_interval_s |
number, above 0 | 5.0 |
Seconds between checks of the running tasks. |
timeout |
a duration such as "45s", "90m", "24h" or "2d", or a number of seconds |
"24h" |
How long the run waits for one set of tasks: one batch of one model, or one wave of workflow stages for prepare and interpret. Each batch and each wave gets the full time again. |
cpu_only |
true or false | false |
Drop every task's GPU request, so a model whose tasks ask for a GPU runs on the CPU. A model's pool is still checked; altar run --validate says its GPU requests are dropped. |
When a model, a preparation or an interpretation submits its tasks, the run says how long it waits for them, and while they run it says how many are running, waiting to start and finished, when that changes and at least once a minute:
[models.chrombpnet] 12,000 variants to score, in 3 batches
[models.chrombpnet] batch 1 of 3: 5 tasks submitted; waiting up to 24h
[models.chrombpnet] batch 1 of 3: 3 of 5 tasks running, 2 finished, 12m elapsed
Tasks still open when timeout runs out stop the run with exit status 1; see
The tasks ran too long for what happens to them on each backend.
GPU pools¶
A task that asks for a GPU names a pool, and the shipped models ask for the pool gpu. Each backend defines the
pools it serves in its own options, and a model's pool picks one of them. A pool the backend does not define is
refused before anything is submitted, with the pools it does define:
| Backend | Its pools |
|---|---|
local |
The names in its pools option, or the one pool gpu without it. |
kubernetes |
gpu, which needs no entry, and the names in pool_node_selectors and max_slots. |
modal |
The keys of pool_accelerators, gpu by default. |
| Another package's backend | The pools it declares; one that declares none accepts any name. |
local options¶
local runs each task with docker run on this machine. It is Stable.
| Option | Type | Default | Meaning |
|---|---|---|---|
data_root |
path | backend-data in the cache |
The directory mounted into every container at /mnt/volume. The run stages inputs there and reads outputs back. It must be a directory the run can create and write, apart from the cache's staging/. |
container_user |
"uid:gid" |
On Linux, the owner of data_root, or you when it does not exist yet; on macOS, the image's user |
The user the container runs as. Set it for a root-squashed NFS data_root. |
slots |
integer | 4 |
The free slots the backend reports. The run scores a model's batches one after another. |
pools |
table of { devices = [...] } |
One pool, gpu, of every GPU |
The GPU pools and the device indices each may use, such as { gpu = { devices = [1, 2, 3] } }. Each task gets count of its pool's free devices to itself, passed as docker run --gpus '"device=1,2"'; a task that finds too few free waits until a running one finishes. Without it, a task gets docker run --gpus <count>. |
A task that asks for a GPU starts with docker run --gpus, which needs the
NVIDIA Container Toolkit.
kubernetes options¶
kubernetes runs each task as a Kubernetes Job. It needs altar[kubernetes] and a kubeconfig or an in-cluster
service account. It is Preview: it is tested against a fake cluster only.
| Option | Type | Default | Meaning |
|---|---|---|---|
volume_uri |
text | None | The storage location every pod sees mounted at mount_root, such as gs://bucket/altar or file:///nfs/altar. The run copies inputs there through [storage] and copies outputs back. A spec's kubernetes backend needs it. |
namespace |
text | $ALTAR_K8S_NAMESPACE, else default |
The namespace Jobs are created in. |
pvc_name |
text | $ALTAR_K8S_PVC, else none |
The PersistentVolumeClaim mounted at mount_root in every pod. |
mount_root |
text | /mnt/volume |
The mount path inside each pod. |
image_pull_policy |
text | IfNotPresent |
The containers' imagePullPolicy. |
backoff_limit |
integer | 0 |
The Job's backoffLimit. |
run_as_user, run_as_group, fs_group |
integer | None | The pod's UID, primary GID and supplemental volume GID, for a root-squashed or restricted volume. |
pool_node_selectors |
table of tables | None | A pool name mapped to nodeSelector labels, so a GPU task lands on matching nodes. Each name is a pool the backend serves; gpu needs no entry. |
max_slots |
table | None | A pool name mapped to the most tasks it runs at once. Each name other than cpu is a pool the backend serves. |
digest_verifier_image |
text | busybox:1.36, pinned by digest |
The image of the init containers that check input digests and clear outputs. |
trust_verification_records |
true or false | true |
Accept an input from a current verification record instead of hashing it. Set false when others can write the volume. |
job_ttl_seconds_after_finished |
integer | One day | How long Kubernetes keeps a finished Job and its pods. |
modal options¶
modal runs each task as a Modal Sandbox. It needs altar[modal] and Modal credentials. It is Preview: it is tested
against a fake Modal only.
| Option | Type | Default | Meaning |
|---|---|---|---|
app_name |
text | $ALTAR_MODAL_APP, else altar-exec |
The Modal App the sandboxes attach to, created on first use. |
volume_name |
text | $ALTAR_MODAL_VOLUME, else altar-data |
The Modal Volume mounted at /mnt/volume. |
pool_accelerators |
table | { gpu = "A10G" } |
A GPU pool name mapped to a Modal accelerator, such as { gpu = "A100" }. Its keys are the pools the backend serves. |
trust_verification_records |
true or false | true |
Skip an input a current staging record says is already in the Volume. Set false when others can write the Volume. |
Run tasks on a GPU or a cluster covers each backend.
backends¶
Named backends, in place of a single [backend], so models run in different places: one [backends.<name>]
section per backend, each with the keys of [backend], type, options, poll_interval_s, timeout
and cpu_only. Provisional. A spec has [backend] or [backends.<name>], not both.
Each container model names the backend it runs on with its backend key, and may pick one of that backend's
GPU pools with pool. altar prepare and altar interpret run a model's preparation and
interpretation on the same backend. A model's line in altar run --validate names its backend and pools.
[backends.nandi]
type = "local"
options = { pools = { gpu = { devices = [1, 2, 3] }, a100 = { devices = [4, 5] } } }
[backends.cluster]
type = "kubernetes"
options = { volume_uri = "/shared/altar-volume", pool_node_selectors = { h100 = { "nvidia.com/gpu.product" = "H100" } } }
[models.borzoi]
type = "BORZOI"
backend = "nandi"
pool = "a100"
[models.enformer]
type = "ENFORMER"
backend = "cluster"
pool = { score = "h100" }
Models on different backends score one after another.
registries¶
Registries of models and genomes the spec can name, beside the installed ones and those of ALTAR_REGISTRIES.
Provisional. Each key is a registry's name, and its value the URI of the registry's index, or a table: uri, and
connection, a [connections.<name>] to read it through. A name the installed packages also use
replaces theirs for this spec. A local path is read in place; any other index is read by altar resources and
kept in [run] cache_dir, where a run reads it.
| Key | Type | Default | Meaning |
|---|---|---|---|
<name> |
text, or a table | The index: a local path, or an https://, gs://, s3:// or hf:// URI without a credential. |
|
uri |
text | Required in a table | The index's URI. |
connection |
name | None | The [connections.<name>] that reads the index and the files its entries name. Without one, gs://, s3:// and hf:// are read anonymously. |
allow_private_hosts |
boolean | false |
Let a registry hosted elsewhere use hosts on your own network, such as an intranet mirror. Without it, a fetch refuses a host that is an IP address or that points at this machine, a private network, or a cloud metadata service. |
A registry that is not on your disk is read only over https://, or from gs://, s3:// or hf://, and may
name only files at such addresses: not a file on your machine and not http://. Without a connection, cloud
files are read anonymously, never with your own credentials. A fetch stops a download that grows larger than the
entry says, or than 64 GB when it gives no size (ALTAR_FETCH_MAX_BYTES).
[registries]
lab = "https://example.org/registry/registry.json"
private = { uri = "gs://lab-models/registry.json", connection = "gcs" }
[models.k562]
resources = "lab/ENCSR296UHQ"
altar resources fetch lab/ENCSR296UHQ --spec spec.toml fetches an entry of a registry the spec names. Publish a
registry specifies the format.
connections¶
One [connections.<name>] section per way of reaching a bucket or a warehouse. Provisional. A [storage],
[store], [detail_store] or [sources.<name>] section uses one by naming it in its connection key. The
connection a section uses does not change the identity of its scores or its source. A spec
never holds a credential: a connection reads the credentials of the environment.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | Required | "gcp" or "fsspec", or a connection another package registers. |
options |
table | {} |
The connection's settings, below. |
allow_private_hosts |
boolean | false |
Let every registry that reads through this connection use hosts on your own network, as that registry's own allow_private_hosts does. |
gcp options¶
Google Cloud Storage and BigQuery, through Application Default Credentials.
| Option | Type | Default | Meaning |
|---|---|---|---|
project |
text | The project of the ambient credentials | The project billed for requester-pays reads, and the project BigQuery runs and bills queries in. |
anonymous |
true or false | false |
Read public objects with no credentials. BigQuery cannot be reached anonymously. |
impersonate_service_account |
text | None | A service account email to act as. Your credentials need roles/iam.serviceAccountTokenCreator on it. Not with anonymous. |
requester_pays |
true or false | false |
Bill project for reads from requester-pays buckets. Needs project; not with anonymous. |
fsspec options¶
Any fsspec filesystem, such as s3://, az://, https:// or hf://, with fixed options.
| Option | Type | Default | Meaning |
|---|---|---|---|
storage_options |
table | {} |
fsspec keyword arguments, such as { anon = true } or { endpoint_url = "..." }. Credential keys, such as key, secret or account_key, are refused. |
protocols |
list of text | Every non-local scheme | The URI schemes this connection serves, such as ["s3"]. |
[connections.public]
type = "gcp"
options = { anonymous = true }
[connections.lab]
type = "gcp"
[connections.lab.options]
project = "my-lab-project"
impersonate_service_account = "reader@my-lab-project.iam.gserviceaccount.com"
Choose where data is read from shows them in use.
storage¶
How the run reads and writes files: staged inputs, task outputs, the export and artifacts.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | "local" |
"local", "https" or "fsspec", or a storage adapter another package registers. |
options |
table | {} |
The adapter's settings, below. |
connection |
name | None | A [connections.<name>] key. Only fsspec takes one. |
| Type | Options | Reads and writes |
|---|---|---|
local |
fsync (true or false, default false): sync each new file to disk before a write returns. |
Local paths and file:// URIs. |
https |
timeout (seconds, default 300). |
Reads https:// and http:// URLs. Refuses every write. |
fsspec |
storage_options (table), or a connection, not both; fsync. |
Any scheme with an fsspec filesystem installed, such as gs:// with altar[gcs]. A scheme whose extra is missing names the extra to install. |
store¶
Where model scores are kept between runs, and the store the export reads through. Without this section, a spec
with models keeps its scores in SQLite at scores.sqlite in the cache, so a later run reuses them. A spec
without models then reads through an in-memory SQLite store, and the run writes no store file.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | "sqlite": at <cache_dir>/scores.sqlite with models, in memory without |
"sqlite" or "bigquery", or a score store another package registers. |
options |
table | {} |
The store's settings, below. |
connection |
name | None | A [connections.<name>] key. bigquery needs one; sqlite takes none. |
sqlite options¶
| Option | Type | Default | Meaning |
|---|---|---|---|
path |
path | ":memory:" |
The SQLite file. In memory, the scores are gone when the run ends. |
lineages |
true or false | true for a new store in a spec with models |
Keep each set of score-changing settings of a model as its own lineage under the same [models.<name>] key. A file that already holds scores keeps the mode it was created with, and a lineages value that disagrees with it is refused. |
bigquery options¶
Preview: tested on the BigQuery emulator and DuckDB; not run against live BigQuery. It needs altar[bigquery].
| Option | Type | Default | Meaning |
|---|---|---|---|
table_id |
text | Required | The score table, project.dataset.table. |
lineage_table_id |
text | None | The lineage catalog table. Without it, the store keeps no lineages. |
primary_stage_rows |
integer | 1000000 |
Rows per load when the store writes scores. |
export_staging_uri |
text | None | A gs:// prefix the export stages its results under, read back through the store's connection. Without it, the export reads them with the BigQuery Storage Read API. |
export_maximum_bytes_billed |
integer | None | Refuse an export whose estimated scan is larger, before any job runs, and cap each export job's billing. No limit by default. |
export_staging_max_jobs |
integer | 100 |
The most EXPORT DATA jobs one staged export runs. |
Score reuse and provenance explains lineages.
detail_store¶
Where a model's detail tables go: the results with many rows per variant, such as Enformer's per-track effects. The run builds it only for a model that declares detail tables.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | The [store] type |
"sqlite" or "bigquery", or a detail store another package registers. |
options |
table | Derived from [store] |
The store's settings: path for sqlite; table_id for bigquery. |
connection |
name | The [store] connection |
A [connections.<name>] key. bigquery needs one. |
The bigquery detail store is Preview, like the bigquery score store.
Without this section, the details go beside the scores:
[store] |
Details |
|---|---|
sqlite at scores.sqlite |
scores.details.sqlite, in the same directory |
sqlite in memory |
In memory |
bigquery with table_id = "p.d.scores" |
p.d.scores__details, through the same connection |
artifact_store¶
Where artifact files go: outputs the run stores without reading them, such as Sei's chromatin_profile_effects.
A spec needs it when a model's output names an artifact. The run builds it only then.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | Required | "storage", or an artifact store another package registers. |
options |
table | {} |
The store's settings, below. |
The storage type copies the files through the run's [storage]:
| Option | Type | Default | Meaning |
|---|---|---|---|
root_uri |
path or URI | <cache_dir>/artifacts |
Where the files go. |
index_path |
path | <cache_dir>/artifacts/index.sqlite |
The SQLite index of the stored files. |
sources¶
One [sources.<name>] section per annotation source. <name> names the section in messages and the run summary.
type names the installed source.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
text | Required | The source's name, exactly as altar sources lists it, such as "gnomad". A near miss gets the installed spelling suggested. |
options |
table | {} |
The source's settings. altar sources <type> lists them, and each source's page documents them. |
connection |
name | None | A [connections.<name>] key, for a source that reads through a connection. altar sources <type> lists connection under "Set in [sources.<type>] itself" when it does. |
A source with a published dataset, such as gnomad or clinvar, takes these options with the same meaning:
| Option | Type | Default | Meaning |
|---|---|---|---|
dataset |
path or URI | The published copy | Your own copy, such as one altar pull made or the binding's builder wrote. |
table |
text | None | Your own BigQuery table, project.dataset.table, read through a gcp connection. Preview: tested on the BigQuery emulator and DuckDB; not run against live BigQuery. |
release |
text | The published release | The release a table holds. Given only with table. |
location |
text | The primary location | Which published location to read, such as "hf". |
storage_options |
table | None | fsspec settings for reading the files. |
genome_build, where a source takes it, names the build of the data, which is checked against [run]
genome_build. The core sqlite and bigquery sources cannot be declared in a spec; build them in code and pass
them to altar.run.run(annotation_sources=[...]). Models and data lists every source.
[sources.gnomad]
type = "gnomad"
[sources.clinvar]
type = "clinvar"
options = { prioritize_pathogenic = true }
[sources.phylop]
type = "phylop100way"
options = { location = "hf" }
Annotate variants with evidence walks through adding sources.
export¶
The table altar run writes, with one row per variant, or the tables of the long format.
| Key | Type | Default | Meaning |
|---|---|---|---|
format |
"parquet", "tsv" or "csv" |
"parquet" |
parquet keeps list and record columns as lists and records. tsv is gzipped, tab-separated text that writes them as JSON text. csv is a deprecated name for tsv, with a warning. |
layout |
"wide" or "long" |
"wide" |
wide writes one table with a block of columns per model, and _columns.tsv, a list of its columns, beside it. long writes a models table, a variants table and one score table per binding and version of its score columns, with a row per variant and model. Provisional. |
destination |
path | <work_dir>/export |
The directory the export is written to. |
prioritized_only |
true or false | false |
Keep only the prioritized rows. Refused when the spec has no model and no source that prioritizes, since the export would be empty. |
bucket_rows |
integer | About 128 MiB of results | How many variants the export reads from the store and holds at a time. |
Rows are in genomic order: chromosomes 1, 2, …, X, Y and M by position, then any other contig.
Read a run's results describes the columns, and Long format for many models the long tables.
interpretations¶
One [interpretations.<name>] section per interpretation request, which altar interpret runs. altar run
leaves them alone. Provisional.
| Key | Type | Default | Meaning |
|---|---|---|---|
model |
name | Required | A model key of the spec. The request uses that model's type, configuration, resource manifest and run inputs, and stores its products under the key. |
method |
text | One of method and methods is required |
One method, as altar interpretations <type> lists it. A method also runs every method it requires. |
methods |
list of text | One of method and methods is required |
Several methods, each once and each run as its own operation. List none that another listed method already runs. |
target |
text | The methods' only target | The model output to explain. Required when the methods explain more than one target. |
variants |
list of text | The run's cohort | Canonical variant IDs, chr:pos:ref:alt, to interpret instead of the cohort. At least one. |
configuration |
table | {} |
Settings added to the model's configuration that only interpretation reads, such as ChromBPNet's motifs and reference_seed (0 by default). |
run_inputs |
table | {} |
Run inputs added to the model's own. |
A spec whose only work is interpretation needs no [store]. It needs a [backend], except for
altar interpret --read. The product listing goes to interpretation-export/ in the run directory; the products,
each execution's progress and the files it resumes from go to the cache.
Interpret variants walks through a request.
interpretation_store¶
Where altar interpret keeps interpretation products. Provisional.
| Key | Type | Default | Meaning |
|---|---|---|---|
type |
"storage" |
"storage" |
The only type. |
options |
table | {} |
root_uri and index_path, below. Any other key is refused. |
| Option | Type | Default | Meaning |
|---|---|---|---|
root_uri |
path or URI | <cache_dir>/interpretations |
Where the product files go, through the run's [storage]. |
index_path |
path | <cache_dir>/interpretations/index.sqlite |
The SQLite index of the stored products. |