SpliceAI¶
The spliceai source adds Illumina's precomputed SpliceAI splice-change scores to each variant. It looks values up
in a copy of Illumina's score files that you build once. It runs no model.
| Kind | Annotation source; type = "spliceai" |
| Install | altar-spliceai, from a source checkout: pip install -e bindings/spliceai in a clone of the repository (how) |
| Genome builds | hg19, hg38 |
| Variant classes | SNVs and indels Illumina scored: substitutions, 1-base insertions and 1–4-base deletions within genes |
| Data | Build your own with altar-spliceai-build |
| Validation | Not spot-checked (Validation) |
| Licence | Non-commercial use only; Illumina's terms |
| Upstream | Code, Paper |
When to use it¶
- Use SpliceAI to ask whether a variant within a gene gains or loses a splice acceptor or donor, how strongly, and how far from the variant.
- The scores come from Illumina's precomputed files. A variant the files lack, such as a longer indel or one outside a gene, gets no columns.
- Build the masked score set for variant interpretation and the raw set for alternative-splicing analysis (Release and identity).
- Choose AlphaMissense for a missense variant's effect on the protein, and ClinVar for what submitters have classified.
- The scores are free for academic and not-for-profit use only. Check that your use is permitted before you build a copy (Cite and licence).
Add it to a run¶
- Install the binding from a clone of the repository:
pip install -e bindings/spliceai. It is not on PyPI; see Install from a source checkout. -
Obtain Illumina's score files and build a copy (Build another release):
-
Name the copy as
dataset, which is required:
altar sources spliceai lists the columns and options. The run spec reference
describes the [sources.<name>] section, and Annotate variants with evidence walks
through a run. Tutorial 3: put model scores beside evidence adds a copy
you build to a notebook's results.
Outputs¶
| Column | Type | Meaning |
|---|---|---|
spliceai_ds_max |
float | The greatest of the top record's four delta scores, from 0 to 1. |
spliceai_symbol |
str | The top record's gene symbol. |
spliceai_ds_*, 4 columns: ag, al, dg, dl |
float | The top record's delta scores for acceptor gain, acceptor loss, donor gain and donor loss, from 0 to 1. |
spliceai_dp_*, 4 columns: ag, al, dg, dl |
int | Where each change happens, in bases from the variant: positive is downstream, negative upstream. |
spliceai_gene_scores |
list of struct (symbol, ds_ag, ds_al, ds_dg, ds_dl, dp_ag, dp_al, dp_dg, dp_dl, ds_max) |
Every gene record, highest delta score first: symbol, the four ds_*, the four dp_* and ds_max. |
spliceai_genome_build |
str | The genome build of the copy, hg38 or hg19. |
spliceai_masked |
bool | true for Illumina's masked scores, false for raw. |
spliceai_source_release |
str | The upstream release the copy was built from, as given to --release. |
SpliceAI scores a variant once for each gene it falls in. The scalar columns come from one gene record, the one
with the greatest delta score, and spliceai_gene_scores keeps every record.
Each record of spliceai_gene_scores has the same fields as the scalar columns, without the spliceai_ prefix:
symbol: the gene symbol.ds_ag,ds_al,ds_dgandds_dl: the delta scores for acceptor gain, acceptor loss, donor gain and donor loss, from 0 to 1.dp_ag,dp_al,dp_dganddp_dl: where each change happens, in bases from the variant.ds_max: the greatest of the record's four delta scores.
A delta score or position that SpliceAI writes as . is null. A record whose four delta scores are all null
sorts last. Records with the same score are ordered by gene symbol. Stores with nested types keep
spliceai_gene_scores as an array of records; SQLite stores it as JSON text.
A variant with no record has no row, so every column is empty in the export. Empty means "not scored", not a delta score of 0; see Variants a lookup may miss.
An augmented VCF written from a run's export carries the top record as ALTAR1_SPLICEAI_* INFO fields, with
ALTAR1_SPLICEAI_MASKED set to 1 for masked scores, and every gene as one entry of ALTAR1_SPLICEAI_GENES. The
release and build are in the INFO header lines; see
Write an augmented VCF.
Configuration¶
| Option | Set in | Type | Default | Meaning |
|---|---|---|---|---|
dataset |
[sources.spliceai.options] |
path | required | the copy altar-spliceai-build wrote, as a path or URI |
genome_build |
[sources.spliceai.options] |
text | not set | the genome build of the data, checked against it when given |
prioritization_threshold |
[sources.spliceai.options] |
number | 0.5 |
prioritize a variant whose spliceai_ds_max is at least this value, between 0 and 1 |
storage_options |
[sources.spliceai.options] |
table | not set | fsspec settings for reading the files, such as credentials |
connection |
[sources.spliceai] |
not set | connection = "<name>": the [connections.<name>] section to read the data through |
datasetis required: the copyaltar-spliceai-buildwrote, as a path or a URI such asgs://my-bucket/spliceai-1.3-raw-hg38/.prioritization_thresholdis between 0 and 1. The default is0.5(Prioritization).genome_build, when given, must be the copy's build:hg38orhg19.storage_optionsis not allowed withconnection.connection = "<name>", set in[sources.spliceai]itself, names the[connections.<name>]section to read through; see Pass credentials through connections.
An unknown option stops the run with an error that lists the valid ones.
Inputs¶
- Genome builds: hg38 or hg19, one per copy. A run on another build stops before it reads the cohort, and the error names the source.
- Variant classes: the variants in Illumina's files, which are substitutions, 1-base insertions and 1–4-base deletions within genes.
- Allele matching: the builder trims each record to its minimal representation, and the source looks each variant up at its own, so a variant matches however its alleles are padded.
Where it comes from¶
Illumina distributes the precomputed score files through BaseSpace; the
SpliceAI repository links to them. Each build has four files:
spliceai_scores.<raw|masked>.<snv|indel>.<hg19|hg38>.vcf.gz. A copy for one score set uses the SNV file and the
indel file of that set.
Altar publishes no copy of SpliceAI and does not redistribute the scores; see Cite and licence. Keep your copy on disk or in your own bucket; see Read your own copy.
Release and identity¶
Illumina publishes raw and masked score sets. Masked files set a delta score to 0 when it reports a gain of an existing splice site or a loss of a non-site, so raw and masked scores are different quantities. Illumina recommends raw scores for alternative-splicing analysis and masked scores for variant interpretation. Build each set into its own copy.
The source's identity is the org.kundajelab.altar.annotation.spliceai contract, version 1.0.0, with the release
label <release>-raw or <release>-masked, such as 1.3-raw, and the copy's genome build. A run records it in
its provenance. Where the copy is read from is not part of it.
The score files record neither their build nor their masking, so the builder records the release, the masking, the genome build and the file layout in every Parquet file. The source reads them from the first file in sorted order and refuses to open:
- a copy whose first file records no release, or a masking other than
trueorfalse; - a copy built for another genome build than a
genome_buildoption names; - a copy laid out in
CHROM=<chromosome>/directories. Rebuild it with the command below.
Build another release¶
altar-spliceai-build streams the score VCFs into a Parquet copy:
pip install -e bindings/spliceai
altar-spliceai-build --genome-build hg38 --release 1.3 --raw \
--input spliceai_scores.raw.snv.hg38.vcf.gz spliceai_scores.raw.indel.hg38.vcf.gz \
--output spliceai-1.3-raw-hg38
--genome-build (hg38 or hg19), --release and one of --raw or --masked describe the files; the files
themselves state none of them. An input named like Illumina's files must name the same masking and build.
--release names Illumina's release, such as 1.3.
--input and --output take a local path or a URI (gs://, s3://, https://, hf://); a URI needs its
provider's extra, such as altar[gcs]. A licensed copy kept in a bucket is built in place:
altar-spliceai-build --genome-build hg38 --release 1.3 --raw \
--input gs://my-bucket/illumina/spliceai_scores.raw.snv.hg38.vcf.gz \
gs://my-bucket/illumina/spliceai_scores.raw.indel.hg38.vcf.gz \
--output gs://my-bucket/spliceai-1.3-raw-hg38/
Other options:
--overwritedeletes a previous build in--outputfirst.--input-storage-optionand--output-storage-optionpass non-secret fsspec options, such astoken=anon. Credentials come from the provider's ambient ones.--input-attemptssets how often a remote read or upload is retried when its stream fails or ends early.--input -reads standard input;--stdin-namegives the file name it stands for.
An --output URI receives each input's files as they finish, so running the same command again resumes a build
that stopped.
The builder writes one row per allele and gene record to <output>/chr=<chromosome>/<input name>.parquet. Each
file holds variant_id, the key columns chr, pos, ref and alt (trimmed to their minimal
representation), spliceai_symbol, the four ds_* and the four dp_*. It refuses:
- a file whose header does not declare the
SpliceAIINFO field; - a bgzipped input without bgzip's end-of-file block, such as a truncated download;
- input not sorted by position within each chromosome, as Illumina's files are;
- an allele scored twice for one gene across all inputs, such as the same file given twice;
- a build that writes no rows.
It skips and counts records without a SpliceAI value, and alleles that are empty, equal to REF, or hold a base
other than A, C, G or T.
Compute and scale¶
- A lookup runs no model and needs no GPU.
- A lookup reads your copy from disk or from your own bucket.
- On BigQuery, load the built Parquet files into a table and read it with
BigQueryAllelicRecordBackend(Python and extension).
Prioritization¶
The source prioritizes a variant by default when spliceai_ds_max is at least 0.5, SpliceAI's recommended
cutoff. The SpliceAI paper also characterizes 0.2 (high recall) and 0.8 (high precision). Set another cutoff
with prioritization_threshold, between 0 and 1:
What prioritized means explains how rules combine.
Validation¶
Validation label: Not spot-checked. There is no recorded comparison of a built copy with Illumina's files. The builder checks the header, bgzip's end-of-file block, the sort order and duplicate scores as it reads (Build another release).
Not validated:
- values in a built copy against Illumina's VCFs, for either score set or genome build;
- the BigQuery backend on live BigQuery. It is Preview: tested on the BigQuery emulator and DuckDB.
Limits¶
- Illumina's files cover substitutions, 1-base insertions and 1–4-base deletions within genes. Other variants, and variants outside genes, get no row.
- A copy holds one genome build and one score set. A run on another build stops before it reads the cohort, and the error names the source.
- The scores are SpliceAI's predictions from the released model. Altar does not run SpliceAI on variants the files lack.
- Altar publishes no copy; each user obtains the licensed files and builds one.
altar-spliceaiinstalls from a source checkout only.
Cite and licence¶
Cite the SpliceAI paper, and name the release and score set you used, such as "SpliceAI 1.3 raw scores, hg38":
@article{jaganathan2019spliceai,
author = {Jaganathan, Kishore and Kyriazopoulou Panagiotopoulou, Sofia and McRae, Jeremy F. and
Darbandi, Siavash Fazel and Knowles, David and Li, Yang I. and Kosmicki, Jack A. and Arbelaez, Juan and
Cui, Wenwu and Schwartz, Grace B. and Chow, Eric D. and Kanterakis, Efstathios and Gao, Hong and Kia, Amirali
and Batzoglou, Serafim and Sanders, Stephan J. and Farh, Kyle Kai-How},
title = {Predicting splicing from primary sequence with deep learning},
journal = {Cell},
volume = {176},
number = {3},
pages = {535--548.e24},
year = {2019},
doi = {10.1016/j.cell.2018.12.015}
}
- Scores: Illumina states that the precomputed scores are free for academic and not-for-profit use, and that other use requires a commercial licence from Illumina. Obtain the files from Illumina and check that your use is permitted.
- Code and models: the SpliceAI code is under the PolyForm Strict License 1.0.0 and the trained models under CC BY-NC 4.0. See the SpliceAI repository.
- Binding:
altar-spliceaiis MIT-licensed. Altar does not redistribute SpliceAI scores, code or models.
Python and extension¶
SpliceAISource.from_options() builds the same source as a run spec, with the same options as keywords.
SpliceAISource.from_dataset() builds it over a copy, and takes the release, masking and build from the copy's
metadata:
from altar.sources import fetch_annotations
from altar_spliceai import SpliceAISource
source = SpliceAISource.from_dataset("spliceai-1.3-raw-hg38", prioritization_threshold=0.2)
annotations = await fetch_annotations([source], ["chr1:1014143:C:T"])
source.close()
To read another store, pass any AllelicRecordBackend and a
SpliceAIRelease(genome_build=..., masked=..., source_release=...) to
SpliceAISource(backend, release=..., genome_build=...).
Every file holds its chromosome and variant_id, so a BigQuery load of <output>/*.parquet needs no hive
partitioning and can cluster on variant_id. BigQueryAllelicRecordBackend reads such a table with the key columns
chr, pos, ref and alt, and staged_annotation_source stages the annotations for a BigQuery store; see Run at
warehouse scale on BigQuery. The BigQuery backend is Preview: tested on the BigQuery emulator
and DuckDB; not run against live BigQuery.