Skip to content

Give a run its variants

A run spec is the TOML or JSON file that altar run reads. Its [run] and [variants] sections name the variants a run reads, the reference genome it checks them against, and the genome build of their coordinates.

Before you start

  • Install Altar and at least one source or model binding; see Install Altar. Every spec needs a source or a model besides its variants; Annotate variants and Score variants add them.
  • Know the genome build of your coordinates, such as hg38.
  • A reference genome is optional for annotation and required by a model that reads sequence, such as LegNet. UCSC's hg38 FASTA is a 984 MB download and 3.3 GB unpacked. altar resources fetch hg38 downloads it; see Find and fetch models and genomes.

The examples use the quickstart's five-variant cohort.tsv and its annotate.toml spec.

Give it a cohort

[variants] path names a VCF or a variant TSV:

[run]
job_id = "first-annotation"
genome_build = "hg38"

[variants]
path = "cohort.tsv"

A path ending in .vcf, .vcf.gz or .vcf.bgz is read as a VCF, and any other path as a TSV. format = "vcf" or format = "tsv" overrides the suffix. BCF is not read; convert it to VCF first.

The TSV has no header. Each line is one variant: chromosome, position, REF and ALT, separated by tabs or spaces. Anything after ALT is a label for your own use, which the run ignores; it may contain spaces. Blank lines are skipped:

chr7   117559590  ATCT  A
chr17  43045712   T     C   BRCA1 missense
chr1   1000000    A     G

A VCF follows the VCF specification, so its columns are separated by tabs only.

The run reads both formats the same way:

  • Chromosome aliases such as 1 and chr01 become chr1.
  • A variant that appears twice is read once.
  • Each ALT of a multi-allelic VCF record becomes its own variant.
  • A VCF is read one line at a time, so a large one does not need to fit in memory.

The run writes the variants it read to <work_dir>/variants/<job_id>.candidates.tsv, after normalizing each one. Variant identity and normalization describes how a variant is normalized, and why a reference genome changes the result for indels.

Give it a reference genome

[variants] reference names a genome you fetched with altar resources fetch. It must be a genome of the cohort's build:

altar resources fetch hg38
[variants]
path = "cohort.tsv"
reference = "hg38"

hg38 is the alias of grch38/ucsc/hg38, UCSC's GRCh38 FASTA. A genome's id is <assembly>/<authority>/<product>, and altar resources genome lists them with who publishes each: UCSC's grch38/ucsc/hg38, grch37/ucsc/hg19, grcm39/ucsc/mm39 and grcm38/ucsc/mm10, which also answer to hg38, hg19, mm39 and mm10; NCBI's GRCh38 no-alt analysis set, grch38/ncbi/no-alt-analysis-set; and ENCODE's grch37/encode/male-hg19 and grcm38/encode/mm10-no-alt-analysis-set. Name a genome by its id or its alias, without genome/. Find and fetch models and genomes says how they differ and where the files go. A genome that has not been fetched stops the run, which prints the command to fetch it.

To use your own FASTA, name its path with reference_fasta instead. The first run that reads it writes an index, such as hg38.fa.fai, beside it, so put it in a directory you can write:

[variants]
path = "cohort.tsv"
reference_fasta = "my-genomes/hg38.fa"

With a reference, the run left-aligns each indel. gnomAD and ClinVar store indels left-aligned, so an annotation run needs the reference to find an indel written further right in a repeat. A model that reads sequence, such as LegNet, gets the same FASTA. A model that reads none, such as AlphaGenome, does not, and its scores are the same with or without it.

Pin the reference by its digest

When a model reads the reference, the run identifies the FASTA by its SHA-256 digest. A genome named with reference has its digest in the catalog, and its fetch wrote a verification record, so the run never hashes it. For your own FASTA, the first run hashes the whole file and prints the digest:

[variants] hashing hg38.fa (3.3 GB) to identify and verify the reference
[variants] hashed hg38.fa in 2.6s; reference_digest = "sha256:5be01555d98347fdb3714dc84c6f77c9d8bc774adcf32c6f7a8fa06f5baf5e51"

Copy the digest into the spec:

[variants]
path = "cohort.tsv"
reference_fasta = "hg38.fa"
reference_digest = "sha256:5be01555d98347fdb3714dc84c6f77c9d8bc774adcf32c6f7a8fa06f5baf5e51"

The next run hashes the file once more to check it, and writes a verification record, hg38.fa.sha256-verified, beside it. Later runs read the record and the file's size and times instead of hashing it again. A run that cannot write in the FASTA's directory keeps no record and hashes the file every time.

A FASTA that no longer matches the digest stops the run before it reads any variant. Restore the expected file, or change the digest only when you meant to change the reference: a model that reads it then scores under a new reference identity. A reference_digest without a reference_fasta is refused. A fetched genome that changed stops the run too; altar resources fetch with its name checks it and downloads it again.

Match the genome build

[run] genome_build is the build of the cohort's coordinates. Before the run reads the cohort, it checks that every source and model can serve that build. A source that cannot stops the run, and the message names each such source with what to do. The quickstart's sources with genome_build = "hg19":

altar run: error: [sources.gnomad] serves hg38 only; the run is hg19.
[sources.clinvar] reads its publisher's copy, which is hg38; the run is hg19. Point it at hg19 data with its dataset option (its binding documents how to build one).
Remove [sources.gnomad] or lift the cohort over to hg38; if the cohort is already on hg38, set [run] genome_build = "hg38".

Do one of these:

  • Remove a source that serves only another build, such as gnomAD, or lift the cohort over to its build.
  • Give a source that supports your build its own data on that build. ClinVar reads hg19 data you build from ClinVar's GRCh37 VCF; see ClinVar.
  • If the cohort is on the source's build after all, correct genome_build.

Altar spells builds the UCSC way: hg38, hg19, mm10, mm39. It reads the GRC names (GRCh38, GRCh37, GRCm38, GRCm39) as the same builds, and the run summary records the UCSC spelling.

genome_build names the build of the variants, not of the models. A model trained on another build, such as a mouse ChromBPNet model scoring a human cohort, reads the cohort's reference. See Variant identity and normalization.

Check the result

altar run --validate (Provisional) checks the spec, opens each source, and says how the run would read the cohort, without reading it. For the quickstart's annotate.toml, on standard error:

$ altar run --validate annotate.toml
[sources.gnomad] opened gnomad 4.1.1-genomes (hg38) from gs://kundajelab-altar/popfreq/gnomad-4.1.1-genomes (published, gcs) in 1.3s
[sources.clinvar] opened clinvar 20260928 (hg38) from gs://kundajelab-altar/clinvar/clinvar-20260928-hg38 (published, gcs) in 0.4s
the spec is valid; nothing was read or written. The run would:
  read cohort.tsv (hg38, trimmed only, no reference)
  annotate with [sources.gnomad]: release 4.1.1-genomes on hg38
  annotate with [sources.clinvar]: release 20260928 on hg38
  keep scores in sqlite (in memory), and write parquet to first-annotation/export
  keep this run's files in first-annotation, and reusable files in .altar

With a reference, the first line reads left-aligned against hg38.fa in place of trimmed only, no reference.

After a run, its JSON summary gives the number of variants it read as variant_count and the normalized file as candidate_variants:

{
  "job_id": "first-annotation",
  "genome_build": "hg38",
  "candidate_variants": "first-annotation/variants/first-annotation.candidates.tsv",
  "variant_count": 5,
  ...
}

What happens to variants the run cannot use

The run never drops a variant without counting it. Where each kind ends up:

  • A VCF ALT that names no sequence is skipped and is not in the export. That covers ., the spanning deletion *, a symbolic allele such as <DEL>, a breakend, an ALT equal to REF, and an IUPAC code such as R. The run counts them by reason in the summary's dropped_alleles and prints a warning at the end:

    altar run: warning: skipped 1 VCF ALT allele that names no sequence a variant key can hold: 1 symbolic (<...>) (e.g. chr1:1000000:A:<DEL>)
      The export does not have it.
    
  • A variant on a contig the FASTA lacks, or whose REF is not the reference's bases, is kept in the export but only trimmed. A model sets it aside as contig_missing or reference_mismatch.

  • A variant a model does not score, such as a deletion for LegNet, which scores SNVs only, is set aside by that model. It stays in the export with empty columns for that model, and the model's line names the file that lists it; see Run it.
  • A variant a lookup may miss, such as an indel without a reference, is kept, and the run ends with a warning; see Variants a lookup may miss.

If it fails

An invalid row stops the run with exit status 2 and names the line. A VCF whose name does not end in a VCF suffix is refused when it starts with a VCF header; set [variants] format = "vcf". A reference that no longer matches reference_digest stops the run too; see Pin the reference by its digest.

Fix a run that stops or warns covers the other messages.

Next