Write an integration page¶
Every model and annotation source has one page under altar/docs/integrations/. Every page follows one template,
with the same sections in the same order for a model and a source. The facts the code holds,
such as columns, types, configuration fields, genome builds and compute, are generated into the page from
altar models --json and altar sources --json. You write the rest: what the integration is for, how to get its
weights or data, how it was validated, its limits and its citation.
Cherimoya is an example model page, and gnomAD an example source page.
The template¶
The page starts with a # title and a one-paragraph lead that says what the integration adds to a variant, and in
which assay, dataset or organism. The card, a generated table of the integration's facts, follows the lead. Then
come these ## sections, in this order:
| Section | Required | What goes in it |
|---|---|---|
| When to use it | yes | Three to five bullets: the question it answers, and when to pick a neighbouring integration instead. |
| Add it to a run | yes | The generated run-spec section, then the steps: install, fetch weights or pull data. |
| Outputs | yes | The generated columns table. A ### subsection for each detail table or artifact, with its generated block. |
| Configuration | yes | The generated table of configuration fields or source options, then prose for the subtleties. |
| Inputs | yes | Genome builds, variant classes, windows, allele matching and reference requirements. |
| Where it comes from | yes | Models: the weights, and ### Choose an experiment or ### Choose tracks when they are selectable. Sources: ### Release and identity and ### Build another release. Both: what changes the identity of stored values. |
| Compute and scale | yes | Models: GPU or CPU, memory, measured times and determinism. Sources: what a lookup reads, local or BigQuery, and sizes. |
| Prioritization | yes | The rule, and a link to What prioritized means. |
| Interpretation | no | For a model with interpretation methods. |
| Filtering | no | For a source whose columns are used to filter variants. |
| Region classification | no | For a source that classifies regions. |
| Validation | yes | The label, what it rests on, the records, and a "Not validated:" list. |
| Limits | yes | What the integration does not do, one bullet each. |
| Cite and licence | yes | A fenced bibtex block for each citable upstream work, then the licences of the code, weights or data, image and binding. |
| Python and extension | yes | The configuration class, the plan's shape, Python entry points and BigQuery. |
The section names are exact. A page has no other ## sections. The optional sections, when present, sit between
Prioritization and Validation in the order above.
Front matter¶
The page starts with YAML front matter. MkDocs reads it as page metadata and does not show it. It holds the facts the code does not:
---
integration:
kind: model
name: cherimoya
display: Cherimoya
summary: Counts effect and profile distance from a BPNet-style model
validation: Smoke-tested
interpretation: Provisional
licence: MIT code; CATv1 weights CC BY 4.0
links:
code: https://github.com/jmschrei/cherimoya
weights: https://huggingface.co/programmable-genomics/CATv1
columns:
counts_log2fc:
units: log2
reading: positive means the alternate allele gains accessibility
profile_l1:
units: 0 to 2
reading: larger means a bigger change in profile shape; no direction
---
| Key | Kinds | Required | Value |
|---|---|---|---|
kind |
all | yes | model, source or link-source. |
name |
all | yes | The name altar models <name> or altar sources <name> takes. |
display |
all | yes | The name the page and the catalog show. For a model, it must match the model's row in the validation status table. |
summary |
all | yes | A short phrase for the catalog's "What it adds" column. |
validation |
all | yes | The validation label. See Validation labels. |
licence |
all | yes | A short licence summary. The card links it to #cite-and-licence. |
links |
all | yes | Upstream links, any of code, weights, data, paper, preprint and docs. |
install |
all | no | pypi, the default, or source-checkout for a binding that is not on PyPI. |
columns |
model, source | yes | One key per column the code declares. See Columns. |
interpretation |
model | when it has interpretation methods | The maturity label of interpretation: Stable, Provisional or Preview. |
tasks |
model | no | How the work is split, as a phrase that follows the device, such as per fold, then a CPU summary or per batch. The card's Compute row and the catalog's Compute column both use it. |
image |
model | no | A sentence added to the card's Image row, such as its platform and size. |
variant_classes |
source, link-source | yes | The variant classes the source covers. Sources do not declare them in code. |
genome_builds |
all | for a link source | A phrase after the genome builds the code declares, such as by score set. When the code declares none, as for a link source or a model whose files set the build, it stands alone, such as Any; set by the model's files. |
data |
source, link-source | no | A phrase the catalog's Data column adds after "Bring your own" or the published release, such as built from DeepMind's release. |
Columns¶
Every column the code declares must match exactly one key under columns. A key that names a column covers it.
A key with *, ? or [...] is a glob, and covers each column that no key names exactly. The page shows a glob
as one row that names the pattern and the columns it matched. A key that matches no column is an error.
A model column's meaning comes from its manifest, word for word. Its key may add units: and reading:, which
fill the table's "Units and reading" column. A source column has no meaning in code, so its key needs a
meaning:. Do not edit a manifest's meaning to change the page: the meaning is part of the result schema's
identity, so changing it changes the identity of stored scores.
Validation labels¶
A model's label is one of Upstream parity (GPU and CPU), Upstream parity (CPU), Reference comparison,
Smoke-tested and Not validated. It must equal the model's label in the validation status table. A source's
label is Matches upstream on spot checks or Not spot-checked.
Generated blocks¶
The script owns the text between a pair of markers:
You write the markers, empty, where each block goes, and the script fills them. It never changes text outside the markers.
| Block | Where | What it holds |
|---|---|---|
card |
Under the lead, before the first ## section |
Kind, install, genome builds, variant classes, compute or data, validation, licence and upstream links. |
spec |
Add it to a run | The run-spec section altar models or altar sources prints. |
outputs |
Outputs | The columns: name, type, meaning, and units and reading when the front matter sets them. |
details name=<name> |
Outputs | One per detail table or artifact the model declares: its columns, or its media type and contents. |
configuration |
Configuration | A model's configuration fields, or a source's options with the section each is set in. |
A configuration field's meaning comes from its pydantic.Field(description=...), and a source option's from the
source's option_descriptions
(Describe the source to altar sources).
Give every field and option a description: the check reports one without. A description is not part of
configuration identity, which hashes values, so adding one does not change stored scores.
An option a run spec cannot carry, such as prioritize_predicate, shows "Only in code" where it is set.
A link source, kind: link-source, is a binding whose command fills a variant–gene link store, such as the
ENCODE-rE2G and scE2G atlases. A run reads its links through the core
variant_gene_links source, which the card's Kind row names. altar sources <name> lists no columns, options or
genome builds for a link source, so its page has only the card block: write its run-spec section, columns and
options by hand. A binding that is also an annotation source under its own name, such as
Open Targets E2G, is documented as kind: source.
The catalog tables in the integrations index are generated blocks too. Each row
summarizes one page's card. Every page in integrations/ needs front matter; the check reports a page without it.
Run the generator¶
The generator runs altar models --json and altar sources --json in the current environment, so sync it with
every binding installed first:
uvx uv@0.12.17 sync --all-packages --extra test --extra kubernetes
.venv/bin/python scripts/gen_integration_pages.py --update
Commit the regenerated blocks with the change that caused them. When a rebase conflicts in a generated block, take
either side and run --update again.
--check, the default, changes nothing. scripts/check_local.py runs it. It fails and lists each problem:
- a block whose content differs from what the code would generate;
- a missing marker, a marker the page does not need, or a block outside its section;
- a column with no key, or with more than one glob key, and a key that matches no column;
- a source column key with no
meaning, or a model column key that sets one; - an unknown front-matter key or link, or a required key that is missing;
- a validation label outside the vocabulary, or a model label that differs from the validation status table;
- a configuration field or option without a description;
- a
##section that is missing, out of order or not in the template, or a page without exactly one#title; - an integration that
altar modelsoraltar sourcesdoes not list in this environment.