Choose where results are stored¶
A run keeps each model's scores in a store. A later run that asks for the same scores reads them back from the store and starts no model. Four sections of a run spec decide where a run's results go:
| Section | What it keeps |
|---|---|
[store] |
The score store. It holds each model's scores, one row per variant, which become the model's columns in the export. |
[detail_store] |
A model's detail tables, with many rows per variant, such as one row per track. Without this section, they go beside the scores. |
[artifact_store] |
Files a model writes and the run stores without reading, such as Sei's chromatin_profile_effects matrix. |
[storage] |
How the run reads and writes files: inputs staged for container tasks, task outputs, the export and artifact files. |
Annotation sources are not stored. The run looks them up again every time and records the release each one served. Where results live explains what each store keeps, and Score reuse and provenance when a stored score counts as the same computation and is reused.
Before you start¶
- A spec that scores with a model; see Score variants with a model. The examples continue from the
quickstart's
score.toml. - For BigQuery:
pip install "altar[bigquery]", a Google Cloud project, a BigQuery dataset that already exists, and Application Default Credentials, such as fromgcloud auth application-default login. - For a Cloud Storage bucket:
pip install "altar[gcs]". Other buckets need their own extra, such asaltar[s3].
Keep scores in a SQLite file¶
A spec with a model and no [store] keeps its scores in a SQLite file in the cache, .altar/scores.sqlite. The
cache is .altar in the directory you start altar run in, unless [run] cache_dir or the ALTAR_CACHE_DIR
environment variable names another. The quickstart's score.toml uses this default.
- The first run creates the cache and the file.
- Every later run that uses the same cache reuses the scores in it, whatever its
job_idor cohort. Start every run from the same directory, or set the samecache_dir. - Deleting a run's own directory,
<job_id>/by default, keeps the scores: they are in the cache. - A new file keeps lineages: a changed model setting adds a new set of scores beside the old ones under the same model key. Change a setting and rerun shows it.
- A model's detail tables go to a second file beside the first,
.altar/scores.details.sqlite.
A [store] section names another file:
- The first run creates the file, in a directory that must exist.
- A relative
pathresolves against the directory you startaltar runin. Start every run from the same directory, or write an absolute path.
Without path, the store is in memory. The scores are gone when the run ends, and the next run scores every
variant again.
An annotation spec, with sources and no model, may leave out [store]. The run then assembles the export in
memory and writes no store file, and creates no cache.
Keep scores in BigQuery¶
Preview
The BigQuery score and detail stores are tested on the BigQuery emulator and DuckDB. They have not been run against live BigQuery.
A bigquery store keeps the scores in a BigQuery table. It reaches BigQuery through a
connection, a [connections.<name>] section that says
which project to use and bill:
[connections.lab]
type = "gcp"
options = { project = "my-project" }
[store]
type = "bigquery"
connection = "lab"
options = { table_id = "my-project.altar.scores", lineage_table_id = "my-project.altar.score_lineages" }
table_idis the score table. The dataset must exist; the store creates the table on the first run.lineage_table_idnames a table that records each lineage. Without it, the store keeps no lineages, and a changed model setting under the same model key stops the run.- Queries run in the connection's project, and are billed to it.
- Detail tables go to
my-project.altar.scores__details, through the same connection. - The export is still written to
<work_dir>/exporton this machine.
Run at warehouse scale on BigQuery covers the tables the store creates, and joining annotation tables to the scores in SQL.
Put detail tables somewhere else¶
A model with detail tables, such as Enformer's track_effects, writes them on every run. [detail_store] names
where they go when the default place beside the scores does not suit:
The run opens the detail store only for a model that has detail tables. A bigquery detail store takes
table_id and needs a connection, as the score store does. A variant counts as already scored only when the
score store holds its scores and the detail store holds its detail rows, so keep both sections the same across
runs.
Store artifacts¶
A model writes an artifact only when its output selects one; Select details and
artifacts shows how. The spec then needs an [artifact_store]:
The storage artifact store copies each file to <cache_dir>/artifacts and records it in a SQLite index,
<cache_dir>/artifacts/index.sqlite, so a later run with the same cache finds them. root_uri puts the files
elsewhere, and index_path the index.
Read and write files in a bucket¶
[storage] sets how the run reads and writes files. The default, local, reads and writes paths on this
machine. fsspec reads and writes local paths too, and adds bucket URIs such as gs:// and s3://. With a
connection, it uses that connection's project and credentials.
This spec keeps the artifact files in a Cloud Storage bucket, billed to my-project:
[connections.lab]
type = "gcp"
options = { project = "my-project" }
[storage]
type = "fsspec"
connection = "lab"
[artifact_store]
type = "storage"
options = { root_uri = "gs://my-bucket/altar/artifacts" }
The artifact index stays at <cache_dir>/artifacts/index.sqlite, on this machine. A Kubernetes backend copies
its tasks' files through [storage] too; see Run on Kubernetes.
Check the result¶
altar run --validate names the score store:
It checks that the store's type is installed, but does not open the store: a BigQuery table it cannot reach
stops the run, not --validate.
Run the same spec a second time. Each model's line counts the variants the store already held:
[models.legnet-wtc11] 5 candidates, 1 set aside (1 deletion; listed in first-scores/staging/jobs/first-scores/variants/legnet-wtc11/ineligible.tsv), 4 already scored, 0 scored in 0 batches, in 0.0s
The run summary's models entry gives the same count as cache_hit_count.
If it fails¶
-
A
bigquerystore withoutconnectionstops the run:altar run: error: store: 'bigquery' cannot be built from the spec's options (BigQueryScoreStore.__init__() missing 1 required positional argument: 'client'); it reaches its backend through a connection, so declare one and name it with store.connectionDeclare a
gcpconnection and name it withconnection, as in Keep scores in BigQuery. -
OperationalError: unable to open database filemeans the directory of a SQLitepathdoes not exist. Create it, or leave out[store]to keep the scores in the cache. - A
connectionon asqlitestore, or onlocalstorage, is refused, such asstore.connection: 'sqlite' does not read through a connection; remove store.connection. - A
connectionthat names no[connections.<name>]section is refused, with the names the spec declares. - A rerun that scores everything again found no stored scores it could use. The run scored everything again lists what to check.
- A changed setting in a store without lineages stops the run; see A changed setting meets a store without lineages.
Next¶
- Prepare model resources runs a model's preparation step before scoring.
- Read a run's results reads the export, and the detail tables and artifacts the stores hold.
- The run spec reference lists every key of
[store],[detail_store],[artifact_store]and[storage]. - Store and assemble results opens the same stores from Python.