Skip to content

CI: cache apothecary tarballs by digest - #8652

Draft
danoli3 wants to merge 1 commit into
openframeworks:masterfrom
danoli3:cache-apothecary
Draft

danoli3 wants to merge 1 commit into
openframeworks:masterfrom
danoli3:cache-apothecary

Conversation

@danoli3

@danoli3 danoli3 commented Nov 5, 2026

Copy link
Copy Markdown
Member

Cache apothecary tarballs by digest

Every OF job re-downloads the same latest packages from openframeworks/apothecary. download_libs.sh already skips curl when libs/download has the right size. This restores that directory from Actions cache.

Measure first. Restore from Actions cache is sometimes slower than curl from GitHub releases. The dedicated workflow times both; do not treat this as a wall-clock win until those numbers land.

Key

One key per platform/arch slice:

apothecary-<tag>-<platform>-<arch>-<gcc|x>-<sha256(name=digest)[:16]>

Exact match only (no restore-keys prefix). Empty key (missing digests / API failure) skips cache and falls through to curl.

Live latest examples:

slice size key suffix
linux 64 gcc10 7.2 MB 82114ff8701fbc0a
osx ~88 MB d086b7c2d20ab86e
macos (ios/tvos) ~353 MB 0a0072518d4c02f9
vs 64 ~406 MB 47acf6a0fc730c1a

Linux gcc 14 remaps to the published gcc10 archives, same as download_libs.sh. linux aarch64 and arm64 stay distinct. VS ARM64 keys include the x64 + arm64ec zips that download_libs.sh also installs (~1.3 GB).

Change

  • scripts/ci/apothecary_cache_key.sh — digest lookup (fixture via APOTHECARY_RELEASE_JSON)
  • .github/actions/cache-apothecary — actions/cache@v6.1.0 on libs/download
  • Wired before download_libs / of update libs in of.yml, linux-lib-paths.yml, nightly.yml, manual-release.yml, manual-web-examples.yml
  • .github/workflows/measure-apothecary-cache.yml — seed, then time restore vs curl for linux ~7 MB (ubuntu-24.04) and osx ~88 MB (xcode-27). Summary table in the job summary.
  • Unit test in validate-library-downloads.yml

Cache budget

Repo cap is ~10 GB and already holds ccache/sccache. These are compressed tarballs, not extracted trees. VS 2022+2026 × x64/ARM64/ARM64EC is the heavy slice. LRU eviction if we go over.

Fork PRs often cannot save caches (read-only token). First run of this PR may still curl; a master run after merge seeds the keys for later PRs. Look at Measure apothecary cache for restore vs curl times.

Test

  • bash scripts/ci/tests/test-apothecary-cache-key.sh (fixture: gcc 14 remap, x64 alias, VS ARM64 extra zips, emscripten default 32, empty key when no digests)
  • bash -n on the new scripts
  • Live GitHub API keys against openframeworks/apothecary latest

Restore libs/download from Actions cache using one key per platform/arch:
apothecary-<tag>-<platform>-<arch>-<gcc>-<sha16 of asset digests>.

download_libs.sh already skips curl when the local archive size matches, so a
hit avoids re-fetching the same latest packages on every job.

Exact key only (no restore-keys prefix). Linux gcc 14 remaps to the published
gcc10 archives. VS ARM64 keys include the x64 + arm64ec zips that
download_libs.sh also installs.

Measure restore vs GitHub release curl before treating this as a win:
.github/workflows/measure-apothecary-cache.yml (linux ~7 MB, osx ~88 MB).
Restore from Actions cache is sometimes slower than curl.
@danoli3
danoli3 marked this pull request as draft November 5, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant