Skip to content

Release Altar

Altar releases every distribution in the repository together, at one version, from one tag. Publishing a GitHub release runs .github/workflows/release.yml, which checks, tests, builds, and uploads all of them to PyPI. When GitHub Actions cannot run, scripts/release_local.py runs the same steps from a maintainer's machine.

Distributions

PyPI project Source directory
altar-identity identity/
altar altar/
altar-alphagenome bindings/alphagenome/
altar-alphamissense bindings/alphamissense/
altar-borzoi bindings/borzoi/
altar-cherimoya bindings/cherimoya/
altar-chrombpnet bindings/chrombpnet/
altar-clinvar bindings/clinvar/
altar-conservation bindings/conservation/
altar-e2g-atlas bindings/e2g-atlas/
altar-enformer bindings/enformer/
altar-gpnstar bindings/gpnstar/
altar-legnet bindings/legnet/
altar-opentargets-e2g bindings/opentargets-e2g/
altar-popfreq bindings/popfreq/
altar-sei bindings/sei/
altar-spliceai bindings/spliceai/

They upload in dependency order. altar pins the exact altar-identity version, so altar-identity goes first, then altar, then the bindings, which require altar. Model runtime images are not published to PyPI.

A release can publish only some bindings, as 0.1.0, 0.2.0 and 0.3.0 did. 0.1.0 published altar-identity, altar, altar-chrombpnet, altar-cherimoya, altar-alphamissense, and altar-opentargets-e2g; 0.2.0 added altar-popfreq and altar-clinvar. 0.3.0 added altar-alphagenome, altar-legnet, altar-borzoi, altar-conservation, altar-enformer and altar-sei, 14 projects in all. altar-e2g-atlas, altar-gpnstar and altar-spliceai have never been published, so PyPI has no project by those names yet. Every distribution still gets the version and is built and checked; see Publish a subset.

One-time PyPI setup

Do this once, before the first release.

  1. Create a PyPI account and turn on two-factor authentication. PyPI requires it to upload.
  2. Optionally, create a PyPI organization for the lab. PyPI approves organizations by hand, which can take days. A project can move into an organization later, so a release does not need to wait for one.
  3. In the GitHub repository, open Settings → Environments and create an environment named pypi. Every upload job in release.yml runs in it. To guard uploads, add yourself as a required reviewer and limit deployments to tags matching v*.
  4. On PyPI, open Your account → Publishing, and add a pending GitHub publisher for each project you publish: all 17 in the table above, or only the subset a release publishes. Use the same values for every project:

    Field Value
    PyPI project name the project, e.g. altar-chrombpnet
    Owner kundajelab
    Repository name altar
    Workflow name release.yml
    Environment name pypi

    A pending publisher lets the workflow create the project on its first upload. It does not reserve the name, so release soon after adding them. If you created an organization, add the pending publishers from the organization's Publishing page instead.

Release with GitHub Actions

  1. On a branch, set the new version everywhere it appears:

    • version in each of the 17 pyproject.toml files;
    • __version__ in altar/altar/__init__.py and identity/altar_identity/__init__.py;
    • the altar-identity==<version> dependency in altar/pyproject.toml;
    • the altar>=X.Y.Z,<X.(Y+1) dependency in every binding, at every release. Its lower bound is the release version itself, because a binding in this repository can import names that only the same altar release provides. For example, 0.2.0 needs altar>=0.2.0,<0.3.
    • version in CITATION.cff and altar/CITATION.cff.

    Both release paths refuse a binding whose altar range differs from >=X.Y.Z,<X.(Y+1), even when the range admits the release version.

    ALTAR_API_VERSION in altar/altar/plugins/manifest.py is the version of the plugin interface, not of the package. Bindings declare the range of it they support. Change it only when the plugin interface changes; see Package and register a plugin. 2. Relock the workspace and every runtime, because their lockfiles record these versions:

    uvx uv@0.12.17 lock
    for project in runtimes/*/; do uvx uv@0.12.17 lock --project "$project"; done
    
  2. Regenerate the tutorial exports on a machine with a GPU, with scripts/export_tutorials.py, and commit the pages and altar/docs/start/notebooks/exports.json it writes. The script runs every notebook in examples/tutorials/ and records the SHA-256 of each one. release_local.py refuses a release whose exports are stale: a notebook, or their helper _series.py, changed after its page was exported. On a machine whose GPUs are all in use, add --cpu.

    uvx uv@0.12.17 sync --all-packages --extra test --extra kubernetes --group tutorials
    .venv/bin/python scripts/export_tutorials.py
    
  3. In CHANGELOG.md, write the version's section from the changes merged since the last release, headed ## [X.Y.Z] - YYYY-MM-DD, and add its link reference at the bottom.

  4. Run the local gate and a release dry run, then merge the branch:

    python3 scripts/release_local.py vX.Y.Z
    
  5. Tag the merge commit on main and push the tag:

    git switch main && git pull --ff-only
    git tag -a vX.Y.Z -m "Altar X.Y.Z"
    git push origin vX.Y.Z
    
  6. Publish a GitHub release for the tag, with the version's CHANGELOG section as its notes. Keep --prerelease for an alpha or beta, and drop it for a stable release. release.yml publishes every distribution, so a subset release cannot use this path.

    gh release create vX.Y.Z --verify-tag --prerelease --title "Altar X.Y.Z" --notes-file notes.md
    

Publishing the release starts release.yml. It fails before building if the tag and any version or internal pin disagree. It runs the identity, core, and binding tests, strict mypy, the boundary and public-API checks, and uv lock --check for the workspace and every runtime. It then builds a wheel and an sdist of each distribution, runs twine check --strict, and installs every wheel together on Python 3.12 and 3.13. There it checks each version and import and finds each binding's entry points through Altar's plugin discovery. It stops before any upload if PyPI already holds a file of this version with other bytes. Last, it uploads altar-identity, then altar, then the 15 bindings in parallel.

If an upload fails, for example because a project has no trusted publisher, fix the cause and re-run the failed jobs from the Actions page. Files already on PyPI are skipped.

Release from a local checkout

Use this path when GitHub Actions cannot run. It needs Linux or macOS, Python 3.11 or newer to run the script, uv on PATH (the script runs uv 0.12.17 through uvx), and network access. If your python3 is older, run the script as uvx uv@0.12.17 run --no-project --python 3.12 scripts/release_local.py.

  1. Prepare the release as in steps 1 to 4 above and merge it. Tag the merge commit with an annotated tag as in step 6, and check out the tag in a clean clone or worktree:

    git switch --detach vX.Y.Z
    
  2. Run the dry run:

    python3 scripts/release_local.py vX.Y.Z
    

    The script stops at the first failure and uploads nothing. It:

    • refuses modified tracked files, and untracked files inside any distribution's directory;
    • refuses a tag that disagrees with any version or internal pin, including a binding whose altar requirement is not >=X.Y.Z,<X.(Y+1), and a CHANGELOG without the version's section;
    • refuses tutorial exports that were not made from the notebooks being released, as step 3 above describes;
    • syncs a temporary environment from uv.lock and runs scripts/check_local.py --also-python 3.13 in it;
    • replaces dist/ with one wheel and one sdist per distribution, in dist/<project>/, and records the commit, the gate result, and each file's SHA-256 in dist/release-stamp.json;
    • refuses an sdist that holds any file git does not track, such as one a global gitignore hides;
    • runs twine check --strict on all 32 files;
    • installs altar-identity and altar alone and checks the public API, then installs every wheel in a clean environment on Python 3.12 and on 3.13, checks each version, imports each package, and checks that Altar's plugin discovery finds and validates each binding's entry points;
    • reads the files PyPI holds for the version from its JSON API, and refuses any file name whose SHA-256 differs from the one built, and any file PyPI holds that the build lacks. A file the build has and PyPI lacks is allowed, so a partial upload can be completed;
    • runs uv publish --dry-run for each distribution in upload order, which reports any file already on PyPI.

    The gate takes several minutes. --skip-gate skips it for a quick repeat, but a build made that way cannot be published. To publish only some bindings, add --only here and in step 4, as described in Publish a subset.

  3. Create a PyPI API token at Account settings → API tokens. Before the first release the projects do not exist, so the token's scope must be the entire account. Read it into the environment without echoing it or saving it in shell history:

    read -rs UV_PUBLISH_TOKEN && export UV_PUBLISH_TOKEN
    
  4. Publish the files the dry run built and checked:

    python3 scripts/release_local.py vX.Y.Z --publish --from-dist
    unset UV_PUBLISH_TOKEN
    

    --from-dist skips the gate and the build. It accepts dist/ only if its stamp names the commit at HEAD, records a passed gate, and matches every file's SHA-256. It then repeats the other checks and uploads with uv publish in dependency order. Without --from-dist, --publish runs the gate and builds again.

    --publish refuses to run without an annotated tag at HEAD or without a dated CHANGELOG section. Only the uploading uv publish processes receive UV_PUBLISH_TOKEN; the script never prints the token or passes it as an argument. Files already on PyPI are skipped, so if an upload fails part-way, fix the cause and run the same command again.

  5. Delete the API token on PyPI.

  6. The projects now exist without a trusted publisher. For each project you published, open Manage → Publishing and add a GitHub publisher with the values from the one-time setup.
  7. Push the tag if you have not, and publish the GitHub release for it as in step 7 above. If Actions runs release.yml, its upload jobs skip the files already on PyPI. Without the trusted publishers from step 6, those jobs fail, although nothing is lost. After a subset release, do not publish a GitHub release.

To rehearse on TestPyPI, use a TestPyPI account and token and add --index testpypi. TestPyPI does not mirror third-party packages, so install from it with pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ altar.

Check the distributions between releases

--candidate runs the build checks on the workspace at its own version, such as 0.4.0.dev0 on main. It takes no tag, needs no CHANGELOG section, and never uploads. It builds all 17 distributions, checks their sdists and metadata, installs every wheel together on Python 3.12 and 3.13, checks plugin discovery, and compares the files with PyPI:

python3 scripts/release_local.py --candidate --skip-gate --out-dir /tmp/altar-candidate

Between releases a binding's altar range may keep an earlier floor of the same minor series, as described in After a release. scripts/check_local.py --release-build runs this command.

Publish a subset

--only names the bindings to publish. altar-identity and altar are always published, because every binding requires them. For 0.2.0:

python3 scripts/release_local.py v0.2.0 --only altar-chrombpnet altar-cherimoya altar-alphamissense \
  altar-opentargets-e2g altar-popfreq altar-clinvar
python3 scripts/release_local.py v0.2.0 --publish --from-dist --only altar-chrombpnet altar-cherimoya \
  altar-alphamissense altar-opentargets-e2g altar-popfreq altar-clinvar

The script still runs the gate and builds and checks every distribution, so the release is validated as a whole and dist/release-stamp.json covers every file. It also installs only the published wheels in a clean environment and checks their versions, imports, and entry points, which shows that the published set works on its own. Only the uv publish step, dry run or real, is limited to the selection. It refuses an unknown name, and a selection that leaves out a workspace distribution that a selected one requires.

The stamp records the selection. --from-dist refuses a --only selection, or its absence, that differs from the build's, so pass the same --only to the dry run and to the publish. The summary lists every file that was or would be uploaded, in upload order.

Do not publish a GitHub release for a subset release

release.yml uploads every binding in the workspace. If you publish a GitHub release for a subset release's tag, its upload jobs fail for every project without a trusted publisher, or, where a pending publisher exists, publish bindings you meant to hold back. Push the tag, but leave the GitHub release unpublished, and add trusted publishers, pending or not, only for the projects you published.

Publish the documentation

The site at https://kundajelab.github.io/altar/ describes the latest release. .github/workflows/docs.yml builds it with --strict on every pull request and every push to main that touches the docs, the library source or the docs configuration, but it deploys only in two cases:

  • when a GitHub release is published, from the release's tag;
  • when a maintainer runs the workflow by hand (workflow_dispatch), from the branch or tag they choose.

A release from a local checkout publishes no GitHub release, so publish its docs by hand, from the release tag:

gh workflow run docs.yml --ref vX.Y.Z

Run it the same way to publish a documentation fix between releases, from a branch cut from the release tag. The github-pages environment must allow that branch or tag in its deployment rules.

After a release

  1. Install from PyPI in a clean environment and check that the binding registers:

    python3.12 -m venv /tmp/altar-release-check
    /tmp/altar-release-check/bin/pip install --no-cache-dir "altar==X.Y.Z" "altar-chrombpnet==X.Y.Z"
    /tmp/altar-release-check/bin/python -c \
      'import altar; from altar.models import get_model_plugin; print(altar.__version__, get_model_plugin("CHROMBPNET"))'
    

    A new release can take a few minutes to appear in the index. 2. Check that each project you published has a trusted publisher on PyPI, and that no API token is left. 3. Check that each project page on PyPI shows its README and links. 4. Start an ## [Unreleased] section at the top of CHANGELOG.md for the next release. 5. On a branch, move every distribution to the next development version and merge it. PyPI holds the released files, so main must not keep claiming their version, and the release checks refuse to publish other bytes under a released version. - Set version in the 17 pyproject.toml files, __version__ in altar/altar/__init__.py and identity/altar_identity/__init__.py, and the altar-identity==<version> dependency in altar/pyproject.toml to the development version. - Set each binding's altar range as the table below says. A range's ceiling <X.(Y+1) excludes every pre-release and development version of X.(Y+1), so a move to the next minor version must raise the range. - Leave CITATION.cff and altar/CITATION.cff at the published release. - Relock the workspace and every runtime, as in step 2 of the release.

    Next version Example after 0.3.0 Each binding's altar range
    The next pre-release of the same version 0.3.0rc2.dev0 after 0.3.0rc1 Unchanged, >=0.3.0rc1,<0.4
    The next patch version 0.3.1.dev0 Unchanged, >=0.3.0,<0.4
    The next minor version 0.4.0.dev0 Raised to the development version, >=0.4.0.dev0,<0.5

    release_local.py --candidate and the release tests check these ranges between releases: the ceiling is the next minor version, and the floor is a version of the same minor series no later than the development version. The next release raises the floor to the release version.

    A dry run with a tag accepts only a release version. Between releases, check main with --candidate instead.