Enformer¶
Enformer predicts how a human SNV changes the signal of the chromatin, CAGE and transcription-factor tracks you select from its 5,313 human tracks (hg38), from 393,216 bases of sequence around the variant.
| Kind | Container model; type = "ENFORMER" |
| Install | pip install altar-enformer (PyPI) |
| Genome builds | hg38 |
| Variant classes | SNVs |
| Compute | An NVIDIA GPU per batch; up to 16 GB of memory per task |
| Image | riyavsinha/altar-enformer@sha256:52aafa8c…, pinned by digest. linux/amd64, 2.8 GB compressed, pulled anonymously from Docker Hub on first use |
| Validation | Upstream parity (CPU) (Validation) |
| Licence | Apache 2.0 code and weights |
| Upstream | Code, Weights, Paper |
When to use it¶
- Use Enformer to ask how an SNV changes predicted DNase-seq, ATAC-seq, CAGE, histone or transcription-factor ChIP-seq signal near it, in many cell types and tissues from one model. You choose the tracks (Choose tracks).
- The primary row names the track with the largest effect, and the
track_effectsdetail table keeps every selected track. The scores match DeepMind's own variant-scoring recipe on a CPU (Validation). - Choose Borzoi to ask how a variant changes the expression of each nearby gene. Enformer sums each track over its whole output window and reports no gene.
- Choose ChromBPNet or Cherimoya for base-resolution accessibility in one experiment, and for indels. Enformer scores SNVs only.
- Choose AlphaGenome for a hosted model that also scores indels.
Add it to a run¶
[models.enformer]
type = "ENFORMER"
resource_manifest = "enformer.resources.json" # altar-enformer-resources writes it
[models.enformer.configuration]
# track_indices = [0] # required, list of integer, unless the resource manifest gives it
- Install the binding with the extra the resource command needs:
pip install "altar-enformer[prepare]". -
Fetch the model and the track table:
-
Name the manifest the command wrote as
resource_manifest, and choose the tracks (Choose tracks): -
Set
[variants] reference_fastato an hg38 FASTA, and give the spec a[backend].
altar models enformer lists the primary columns, the track_effects detail table, the configuration fields, the
default image and the resource command. Score variants with a model walks through a run.
Select details and artifacts shows where the run keeps the
detail rows.
Outputs¶
| Column | Type | Meaning | Units and reading |
|---|---|---|---|
top_track_index |
int | Output index with largest absolute SAD | 0 to 5,312; an index in targets_human.txt |
top_track_id |
str | Identifier of the output with largest absolute SAD | text; such as an ENCODE file accession |
top_track_sad |
float | Alternate minus reference summed activity for the track with largest absolute SAD | summed track signal; positive means the alternate allele raises the signal |
top_track_sadr |
float | Log2 alternate/reference summed-activity ratio for the track with largest absolute SAD | log2; same sign as SAD |
n_tracks_scored |
int | Number of selected human output tracks | count |
For each selected track, the runtime sums the prediction over Enformer's 896 output bins of 128 bases, for the reference and the alternate sequence. It then computes the two Basenji variant statistics that DeepMind's Enformer release links to:
SAD = alternate_signal - reference_signal
SADR = log2(1 + alternate_signal) - log2(1 + reference_signal)
The primary table has one row per variant, for the selected track with the largest absolute SAD.
DeepMind's simpler usage notebook takes the mean over bins instead of the sum, which divides SAD by 896 and does
not change which track ranks first. SAR, the sum of per-bin log ratios, is a different Basenji statistic and is
not computed.
track_effects¶
A detail table, version 0.0.1: Official Enformer SAD and SADR effects for every selected human output track.
| Column | Type | Meaning |
|---|---|---|
track_index |
int | Zero-based Enformer human output index |
track_id |
str | Identifier from targets_human.txt |
ref_signal |
float | Reference activity summed over 896 bins |
alt_signal |
float | Alternate activity summed over 896 bins |
sad |
float | Alternate minus reference summed activity |
sadr |
float | Log2 alternate/reference summed-activity ratio with pseudocount 1 |
The table has one row per variant and selected track. ref_signal and alt_signal are the summed signals SAD and
SADR are computed from. Read it as Read a run's results
describes.
Configuration¶
| Field | Type | Default | Meaning |
|---|---|---|---|
weights |
resource | required | A tar archive of the TensorFlow SavedModel, with a SHA-256 digest. altar-enformer-resources writes it to a resource manifest. |
track_metadata |
resource | required | DeepMind's targets_human.txt track table, with a SHA-256 digest. Its index column maps each output track to its identifier. |
track_indices |
list of integer | required | The tracks to score, as indices into targets_human.txt from 0 to 5,312, in increasing order. |
image |
text | "riyavsinha/altar-enformer@sha256:52aafa8c…" |
The scoring image, as name@sha256:<digest>. Tags are refused. |
Fields go under [models.<name>.configuration]. A resource manifest gives weights and track_metadata instead:
name its path as resource_manifest in [models.<name>].
track_indicesis required. It must be strictly increasing, so one selection always has one identity.imagemust be aname@sha256:<digest>reference built fromruntimes/enformer(Python and extension).
Inputs¶
- Genome builds: hg38 only, the assembly of the published human model.
- Variant classes: SNVs only. Batch preparation sets indels and multi-nucleotide variants aside and reports them, and the rest of the cohort scores. The runtime refuses a non-SNV in a batch staged some other way.
- Window: 393,216 bases around the variant. The prediction covers the central 114,688 bases.
- Reference:
[variants] reference_fasta. - Tracks:
track_indices, a strictly ordered list of indices intotargets_human.txt. Choose tracks for the tissues and assays your question is about.
Where it comes from¶
An Enformer model instance needs two resources, each with a SHA-256 digest: a tar archive holding the TensorFlow
SavedModel, and DeepMind's targets_human.txt track table. altar-enformer-resources fetch-official fetches both.
It downloads the SavedModel from Kaggle
(deepmind/enformer/tensorFlow2/enformer/1) through kagglehub, and targets_human.txt from the Basenji
repository at tag 0.5. It checks all 5,313 rows of the track table, builds a deterministic archive (about 935 MB),
and writes enformer.resources.json. For files you have already downloaded,
altar-enformer-resources package --saved-model … --track-metadata … --output-dir … writes the same manifest.
Inside the runtime image, enformer-score smoke loads the model from those exact bytes and runs one full-size
prediction.
A model instance is defined by the archive and track-table digests, the ordered track selection, the image and the reference's digest. Resource locations are not part of it, so moving the same bytes keeps every stored score. Another image stores scores under a new identity (Python and extension).
Choose tracks¶
targets_human.txt has one row per track. Its index column is the value track_indices takes, identifier is
the track's ID, such as an ENCODE file accession or a FANTOM5 CAGE library, and description starts with the assay:
DNASE, ATAC, CHIP (histone marks and transcription factors) or CAGE. This reads the table at the tag the
fetch command pins, and lists the DNase-seq and CAGE tracks for K562:
import pandas as pd
targets = pd.read_csv(
"https://raw.githubusercontent.com/calico/basenji/0.5/manuscripts/cross2020/targets_human.txt",
sep="\t",
)
assay = targets.description.str.split(":").str[0]
k562 = targets[assay.isin(["DNASE", "CAGE"]) & targets.description.str.contains("K562")]
print(k562[["index", "identifier", "description"]].to_string(index=False))
print(k562["index"].tolist())
It prints nine tracks, seven DNase-seq and two CAGE, and their indices:
Give that list as track_indices. The table has 674 DNase-seq, 10 ATAC-seq, 3,991 ChIP-seq and 638 CAGE tracks.
Some describe treated samples or cell-cycle phases, such as DNASE:K562 G1 phase, so read the descriptions before
you select. The prioritization rule reads only the top track, so its meaning depends on the selection.
Compute and scale¶
- GPU: each batch task asks for one NVIDIA GPU and up to 16 GB of memory. On an NVIDIA L40S, a cold start that loads the model and predicts one window took about 30 seconds.
- CPU:
[backend] cpu_only = true(run spec) clears the GPU request. The validation ran this way: 20 variants across all 5,313 tracks took 203 seconds on a CPU host. - Scale: Altar splits the cohort into batches, and each batch is one retryable container task. The runtime loads the model once per task, predicts in small batches, and writes rows as it goes, so it never holds the whole job or the whole detail table in memory.
- Image:
linux/amd64, 2.8 GB compressed. It runs TensorFlow 2.13 on Python 3.10 with CUDA 11.8.
Prioritization¶
A variant is prioritized when abs(top_track_sad) >= 0.5. This is a screening rule, not a calibrated threshold.
Its meaning depends on the tracks you select. What prioritized means explains how
rules combine.
Validation¶
Validation label: Upstream parity (CPU). The default image, run through altar run with cpu_only, matches
DeepMind's TF-Hub variant-scoring recipe (the "Score the variant" cell of enformer-usage.ipynb) on 20 hg38 SNVs.
Across all 5,313 human tracks the largest SAD difference is 6.8e-13, and every variant's top track matches. Record:
cpu-official-reference-b021894.json.
On an NVIDIA L40S, an earlier build of the same runtime passed a full-size smoke test and an independent check of
the SAD and SADR formulas and the top-track choice. That record,
l40s-semantics-2.json,
makes no comparison with DeepMind's recipe.
Not validated:
- parity with DeepMind's recipe on a GPU;
- the default image on a GPU;
- indels, which the binding does not score.
The validation status table lists every model.
Limits¶
- hg38 SNVs only.
- Scores are raw predicted signal. Track percentiles and other calibration are not part of the output.
- The primary row reports one track. Use
track_effectsto compare tracks. - The prioritization rule is a screening rule, not a calibrated threshold.
Cite and licence¶
Cite the Enformer paper:
@article{avsec2021enformer,
author = {Avsec, {\v{Z}}iga and Agarwal, Vikram and Visentin, Daniel and Ledsam, Joseph R. and
Grabska-Barwinska, Agnieszka and Taylor, Kyle R. and Assael, Yannis and Jumper, John and Kohli, Pushmeet and
Kelley, David R.},
title = {Effective gene expression prediction from sequence by integrating long-range interactions},
journal = {Nature Methods},
volume = {18},
number = {10},
pages = {1196--1203},
year = {2021},
doi = {10.1038/s41592-021-01252-x}
}
targets_human.txt comes from the Basenji repository's directory for the cross-species paper, which describes the
human training tracks:
@article{kelley2020crossspecies,
author = {Kelley, David R.},
title = {Cross-species regulatory sequence activity prediction},
journal = {PLoS Computational Biology},
volume = {16},
number = {7},
pages = {e1008050},
year = {2020},
doi = {10.1371/journal.pcbi.1008050}
}
- Code: DeepMind's Enformer code is licensed Apache 2.0.
- Weights: the Kaggle model is licensed Apache 2.0.
targets_human.txtcomes from calico/basenji, licensed Apache 2.0. - Runtime image: built from
runtimes/enformerin the Altar repository (MIT) on NVIDIA's CUDA 11.8 runtime image. It bundles TensorFlow and other third-party packages under their own licences, and CUDA under NVIDIA's terms. It contains no weights. - Binding:
altar-enformeris MIT-licensed.
Python and extension¶
EnformerConfiguration holds weights (the SavedModel archive), track_metadata, track_indices and image.
EnformerConfiguration.from_resource_manifest reads the fetch command's manifest.
Models and plans is the API reference for the model interface it implements.
The scoring plan has one GPU task per batch. Primary and detail files go to the run's score store and detail store.
To use your own image, build it from the repository root, push it, and set image to the pushed digest
(<registry>/altar-enformer@sha256:…), never the tag:
docker build --platform linux/amd64 -f runtimes/enformer/Dockerfile -t <registry>/altar-enformer:<tag> .
docker push <registry>/altar-enformer:<tag>
Scores from another image are stored under a new identity, and the default image does not reuse them.