Release tags' runs no longer save to the cache what only their own tag could read, and the CLI's release builds in a pinned image from a cache kept on main. scripts/sccache.sh sets SCCACHE_GHA_RW_MODE=READ_ONLY when GITHUB_REF_TYPE is tag, for the server and the steps after it: against a stand-in for the cache service, sccache 0.18.0 on a branch's run made a CreateCacheEntry call and on a tag's run made none, and looked its compile up in both. The CLI's release ran in ghcr.io/rust-cross/cargo-zigbuild:latest; it now runs in that image pinned by digest (sha256:d8313491ec57…, Rust 1.93.0, zig 0.16.0), reads the crates entry CI saves from the job's ~/.cargo/registry instead of an empty Docker volume, hands what it wrote as root back to the job's user, and restores target/zig from cli-release-<os>-<image digest>-<deps>, which the Caches workflow builds whole when the key is new (ci-cache.mjs keys gives the digest as zigbuild). Built in that image on an 8-core machine, the five platforms took 162 s from nothing and 103 s from the cache, with only the g1t crate compiled again for each; the entry is 1.3 GB, 463 MB compressed. Tests hold the tag rule in sccache.sh, the CLI's entry to the image's digest and to being built whole, and the builder's flags; CONTRIBUTING.md and the Actions guide say both.
10 files+208−200/10 viewed
| 15 | 15 | # desktop-linux the desktop release's Linux build | |
| 16 | 16 | # desktop-windows its Windows cross build, cargo-xwin, the MSVC CRT and | |
| 17 | 17 | # apt's downloads for it | |
| 18 | + | # cli the CLI release's build of every platform, in the | |
| 19 | + | # cargo-zigbuild image | |
| 18 | 20 | # | |
| 19 | 21 | # A job runs only when its key is missing (a new lockfile, a new toolchain, | |
| 20 | 22 | # or an expired entry), and builds what the release builds. Nothing waits for | |
| ⋯ | |||
| 51 | 53 | runner: ${{ steps.missing.outputs.runner }} | |
| 52 | 54 | desktop-linux: ${{ steps.desktop-linux.outputs.cache-hit != 'true' }} | |
| 53 | 55 | desktop-windows: ${{ steps.desktop-windows.outputs.cache-hit != 'true' || steps.xwin.outputs.cache-hit != 'true' || steps.apt-windows.outputs.cache-hit != 'true' }} | |
| 56 | + | cli: ${{ steps.cli.outputs.cache-hit != 'true' }} | |
| 54 | 57 | steps: | |
| 55 | 58 | - uses: actions/checkout@v5 | |
| 56 | 59 | - name: Cache keys | |
| ⋯ | |||
| 135 | 138 | path: ~/.cache/apt/tauri-windows | |
| 136 | 139 | key: apt-tauri-windows-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }} | |
| 137 | 140 | lookup-only: true | |
| 141 | + | - id: cli | |
| 142 | + | uses: actions/cache/restore@v4 | |
| 143 | + | with: | |
| 144 | + | path: | | |
| 145 | + | target/zig | |
| 146 | + | !target/zig/**/incremental | |
| 147 | + | key: cli-release-${{ runner.os }}-${{ steps.keys.outputs.zigbuild }}-${{ steps.keys.outputs.cargo }} | |
| 148 | + | lookup-only: true | |
| 138 | 149 | # The runner's targets to build: those whose build is missing, or all | |
| 139 | 150 | # of them when zig is (each saves it, the first to finish wins). | |
| 140 | 151 | - name: The runner's targets to build | |
| ⋯ | |||
| 352 | 363 | # What the CRT and SDK came to, for the cache's 2 GB an entry may hold. | |
| 353 | 364 | - name: How large the MSVC CRT is | |
| 354 | 365 | run: du -sh ~/.cache/cargo-xwin || true | |
| 366 | + | ||
| 367 | + | # cli-release.yml's build of every platform, in the cargo-zigbuild image | |
| 368 | + | # (scripts/cli-release.mjs). Built whole whenever its key is new (a new | |
| 369 | + | # image or new dependencies), never started from an older entry: what | |
| 370 | + | # zigbuild built cannot be listed by this machine's Cargo to prune it, so | |
| 371 | + | # nothing old is carried in to begin with. | |
| 372 | + | cli: | |
| 373 | + | name: The CLI's release build | |
| 374 | + | needs: look | |
| 375 | + | if: ${{ needs.look.outputs.cli == 'true' }} | |
| 376 | + | # The builder runs in Docker, which g1t's own machines give each job. | |
| 377 | + | runs-on: g1t-4core | |
| 378 | + | timeout-minutes: 60 | |
| 379 | + | steps: | |
| 380 | + | - uses: actions/checkout@v5 | |
| 381 | + | - name: Cache keys | |
| 382 | + | id: keys | |
| 383 | + | run: node scripts/deploy/ci-cache.mjs keys | |
| 384 | + | - name: Cache crates | |
| 385 | + | uses: actions/cache/restore@v4 | |
| 386 | + | with: | |
| 387 | + | path: | | |
| 388 | + | ~/.cargo/registry/index | |
| 389 | + | ~/.cargo/registry/cache | |
| 390 | + | key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }} | |
| 391 | + | restore-keys: cargo-crates-${{ runner.os }}- | |
| 392 | + | - name: Cache the CLI's build | |
| 393 | + | uses: actions/cache@v4 | |
| 394 | + | with: | |
| 395 | + | path: | | |
| 396 | + | target/zig | |
| 397 | + | !target/zig/**/incremental | |
| 398 | + | key: cli-release-${{ runner.os }}-${{ steps.keys.outputs.zigbuild }}-${{ steps.keys.outputs.cargo }} | |
| 399 | + | - name: Build | |
| 400 | + | run: node scripts/cli-release.mjs build | |
| 401 | + | - name: How large the build is | |
| 402 | + | run: du -sh target/zig || true | |
| 8 | 8 | # work, and its header says what a release holds. It is signed with the | |
| 9 | 9 | # runner's release key (RUNNER_RELEASE_KEY, RUNNER_RELEASE_PUBLIC_KEY). | |
| 10 | 10 | # | |
| 11 | − | # Every platform builds in one job, in the rust-cross/cargo-zigbuild image: | |
| 12 | − | # the macOS builds link Apple's Security framework (the keychain), and that | |
| 13 | − | # image carries its SDK. It runs on g1t's own machines (g1t-4core), whose | |
| 14 | − | # jobs have a Docker Engine of their own. | |
| 11 | + | # Every platform builds in one job, in the rust-cross/cargo-zigbuild image, | |
| 12 | + | # pinned by digest in scripts/cli-release.mjs: the macOS builds link Apple's | |
| 13 | + | # Security framework (the keychain), and that image carries its SDK. It | |
| 14 | + | # runs on g1t's own machines (g1t-4core), whose jobs have a Docker Engine of | |
| 15 | + | # their own. | |
| 15 | 16 | name: CLI release | |
| 16 | 17 | ||
| 17 | 18 | on: | |
| ⋯ | |||
| 42 | 43 | run: | | |
| 43 | 44 | version="$(sed -n 's/^version = "\(.*\)"/\1/p' crates/g1t/Cargo.toml | head -1)" | |
| 44 | 45 | [ "cli-v$version" = "${GITHUB_REF_NAME}" ] || { echo "::error::The tag is ${GITHUB_REF_NAME}, but crates/g1t is $version"; exit 1; } | |
| 46 | + | # zigbuild: the builder image's digest, whose Rust the build uses; | |
| 47 | + | # cargo: Cargo.lock's crates.io and git packages, not the workspace's | |
| 48 | + | # own versions (scripts/deploy/ci-cache.mjs). | |
| 49 | + | - name: Cache keys | |
| 50 | + | id: keys | |
| 51 | + | run: node scripts/deploy/ci-cache.mjs keys | |
| 52 | + | # Every crate the lockfile names, as CI's Rust job saves them on main. | |
| 53 | + | # The builder reads them from this job's ~/.cargo/registry. | |
| 54 | + | - name: Cache crates | |
| 55 | + | uses: actions/cache/restore@v4 | |
| 56 | + | with: | |
| 57 | + | path: | | |
| 58 | + | ~/.cargo/registry/index | |
| 59 | + | ~/.cargo/registry/cache | |
| 60 | + | key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }} | |
| 61 | + | restore-keys: cargo-crates-${{ runner.os }}- | |
| 62 | + | # Every platform's build. A tag's run restores only from its own tag | |
| 63 | + | # and from main, so the Caches workflow (caches.yml) builds and saves | |
| 64 | + | # this on main whenever the key is new; a release compiles the CLI's | |
| 65 | + | # own crates again, and saves nothing. | |
| 66 | + | - name: Cache the CLI's build | |
| 67 | + | uses: actions/cache/restore@v4 | |
| 68 | + | with: | |
| 69 | + | path: | | |
| 70 | + | target/zig | |
| 71 | + | !target/zig/**/incremental | |
| 72 | + | key: cli-release-${{ runner.os }}-${{ steps.keys.outputs.zigbuild }}-${{ steps.keys.outputs.cargo }} | |
| 73 | + | restore-keys: cli-release-${{ runner.os }}-${{ steps.keys.outputs.zigbuild }}- | |
| 45 | 74 | - name: Build | |
| 46 | 75 | run: node scripts/cli-release.mjs build | |
| 47 | 76 | - uses: actions/upload-artifact@v4 | |
| 189 | 189 | - **One set of paths per entry.** An entry's version is the hash of its | |
| 190 | 190 | paths, so every step that names it, in every workflow, names the same | |
| 191 | 191 | ones. `crates/actions/tests/repository_workflows.rs` holds all of this. | |
| 192 | + | - **sccache only reads on a tag** (`scripts/sccache.sh` sets | |
| 193 | + | `SCCACHE_GHA_RW_MODE=READ_ONLY` when `GITHUB_REF_TYPE` is `tag`): a | |
| 194 | + | release's run finds what `main`'s runs compiled, and saves nothing only | |
| 195 | + | its own tag could read. | |
| 196 | + | - **Images a build runs in are pinned by digest.** The CLI's release | |
| 197 | + | builds in `cargo-zigbuild` (`BUILDER` in `scripts/cli-release.mjs`), and | |
| 198 | + | its cache is keyed by that digest (`zigbuild` from `ci-cache.mjs keys`) | |
| 199 | + | rather than by this machine's `rustc`. | |
| 192 | 200 | ||
| 193 | 201 | | Entry | Key | Saved by | Restored by | | |
| 194 | 202 | | --- | --- | --- | --- | | |
| ⋯ | |||
| 203 | 211 | | apt for Tauri | `apt-tauri-<os>-<scripts/apt-install.sh>` | CI desktop | the desktop release | | |
| 204 | 212 | | The runner release | `runner-release-<os>-<target>-<rust>-<deps>`, zig | Caches (`runner`) | the runner release | | |
| 205 | 213 | | The desktop release | `desktop-release-linux-…`, `desktop-release-windows-…`, `xwin-…`, `apt-tauri-windows-…` | Caches (`desktop-linux`, `desktop-windows`) | the desktop release | | |
| 214 | + | | The CLI release | `cli-release-<os>-<zigbuild>-<deps>` (`target/zig`, every platform) | Caches (`cli`), built whole | the CLI release | | |
| 206 | 215 | ||
| 207 | 216 | To see whether a change helped, read a run's cache steps: `Cache restored | |
| 208 | 217 | from key` names the entry (the key itself is a hit; another is a start), | |
| 209 | 218 | `Cache Size` and the step's time say what it cost, and the post step says | |
| 210 | 219 | whether it saved. `sccache's hits and misses` is on each Rust job's summary. | |
| 220 | + | `g1t cache usage` says what the repository's entries hold against its | |
| 221 | + | 10 GiB, and `g1t cache list --sort size-in-bytes` which take the most. | |
| 211 | 222 | ||
| 212 | 223 | ## Speed | |
| 213 | 224 | ||
| 1054 | 1054 | | What is compiled each time | What rustc links: binaries, `cdylib` crates, proc macros, build scripts and test harnesses. | | |
| 1055 | 1055 | | Its entries | One per compiled crate, often thousands for a workspace, each a few KB to a few MB. They count toward the repository's 10 GiB like any other entry, and the ones not restored for 7 days are deleted. | | |
| 1056 | 1056 | | Branches | A pull request reads what the default branch saved. Run the workflow on pushes to the default branch too, as above, or each pull request's first run starts with an empty cache. | | |
| 1057 | + | | Releases from tags | A tag's run reads the default branch's entries, but what it saves only that tag would ever read. Set `SCCACHE_GHA_RW_MODE: READ_ONLY` on a tag's run (`if [ "$GITHUB_REF_TYPE" = tag ]`) so it saves nothing: it still looks every crate up. | | |
| 1057 | 1058 | | Starting over | Set `SCCACHE_GHA_VERSION` to any new value. A new sccache release starts over by itself. | | |
| 1058 | 1059 | | If the cache cannot be reached | sccache refuses to start. Set `SCCACHE_IGNORE_SERVER_IO_ERROR: "1"` to have rustc run without it instead. | | |
| 1059 | 1060 |
| 344 | 344 | "apt-tauri-windows-", | |
| 345 | 345 | "xwin-", | |
| 346 | 346 | "zig-0.13.0-zigbuild-", | |
| 347 | + | "cli-release-", | |
| 347 | 348 | ] { | |
| 348 | 349 | assert!(seen.contains_key(family), "{family}: {:?}", seen.keys()); | |
| 349 | 350 | } | |
| ⋯ | |||
| 403 | 404 | ); | |
| 404 | 405 | let target = paths(&step).iter().any(|p| p.contains("target")); | |
| 405 | 406 | if target { | |
| 406 | − | assert!(key.contains("steps.keys.outputs.rust"), "{file} {}: {key}", job.id); | |
| 407 | + | // The toolchain that built it: this machine's rustc, or for | |
| 408 | + | // the CLI's release, the cargo-zigbuild image it builds in. | |
| 409 | + | let toolchain = if paths(&step).iter().any(|p| p.starts_with("target/zig")) { | |
| 410 | + | "steps.keys.outputs.zigbuild" | |
| 411 | + | } else { | |
| 412 | + | "steps.keys.outputs.rust" | |
| 413 | + | }; | |
| 414 | + | assert!(key.contains(toolchain), "{file} {}: {key}", job.id); | |
| 407 | 415 | // A restore key never reaches another toolchain's entries. | |
| 408 | 416 | for prefix in step.raw["with"]["restore-keys"] | |
| 409 | 417 | .as_str() | |
| ⋯ | |||
| 411 | 419 | .lines() | |
| 412 | 420 | .filter(|l| !l.trim().is_empty()) | |
| 413 | 421 | { | |
| 414 | − | assert!(prefix.contains("steps.keys.outputs.rust"), "{file} {}: {prefix}", job.id); | |
| 422 | + | assert!(prefix.contains(toolchain), "{file} {}: {prefix}", job.id); | |
| 415 | 423 | } | |
| 416 | 424 | } | |
| 417 | 425 | if key.contains("steps.keys.outputs.") { | |
| ⋯ | |||
| 463 | 471 | "{file} {id}" | |
| 464 | 472 | ); | |
| 465 | 473 | } | |
| 474 | + | // The CLI's build is made in the builder image, whose Cargo the host | |
| 475 | + | // cannot ask what it built: so it is never started from an older entry, | |
| 476 | + | // and has nothing old to prune. | |
| 477 | + | let caches = read("caches.yml"); | |
| 478 | + | let cli = caches.jobs.iter().find(|j| j.id == "cli").unwrap(); | |
| 479 | + | let build = cli | |
| 480 | + | .steps | |
| 481 | + | .iter() | |
| 482 | + | .find(|s| s.raw["with"]["key"].as_str().is_some_and(|k| k.starts_with("cli-release-"))) | |
| 483 | + | .unwrap(); | |
| 484 | + | assert!(saves(build)); | |
| 485 | + | assert!(build.raw["with"].get("restore-keys").is_none()); | |
| 466 | 486 | let ci = read("ci.yml"); | |
| 467 | 487 | let rust = ci.jobs.iter().find(|j| j.id == "rust").unwrap(); | |
| 468 | 488 | assert_eq!( | |
| ⋯ | |||
| 976 | 996 | assert!(script.contains("sha256sum -c")); | |
| 977 | 997 | assert!(script.contains("RUSTC_WRAPPER=")); | |
| 978 | 998 | assert!(script.contains("GITHUB_STEP_SUMMARY")); | |
| 999 | + | // A tag's run reads main's entries and saves none, which only its own | |
| 1000 | + | // tag could read; the mode reaches the steps after it too. | |
| 1001 | + | assert!(script.contains("if [ \"${GITHUB_REF_TYPE:-}\" = tag ]; then\n export SCCACHE_GHA_RW_MODE=READ_ONLY")); | |
| 1002 | + | assert!(script.contains("echo \"SCCACHE_GHA_RW_MODE=$SCCACHE_GHA_RW_MODE\"")); | |
| 1003 | + | let tagged = read("runner-release.yml"); | |
| 1004 | + | assert!(tagged.trigger("push").unwrap().tags.allows("runner-v0.4.0")); | |
| 979 | 1005 | } | |
| 980 | 1006 | ||
| 981 | 1007 | #[test] | |
| 28 | 28 | import { spawnSync } from "node:child_process"; | |
| 29 | 29 | import { createHash, createPrivateKey } from "node:crypto"; | |
| 30 | 30 | import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; | |
| 31 | + | import { homedir } from "node:os"; | |
| 31 | 32 | import { dirname, join } from "node:path"; | |
| 32 | 33 | import { fileURLToPath } from "node:url"; | |
| 33 | 34 | ||
| ⋯ | |||
| 35 | 36 | ||
| 36 | 37 | const ROOT = join(dirname(fileURLToPath(import.meta.url)), ".."); | |
| 37 | 38 | const BUCKET = "g1t-downloads"; | |
| 38 | − | const BUILDER = "ghcr.io/rust-cross/cargo-zigbuild:latest"; | |
| 39 | + | // Pinned by digest, so a release builds with the toolchain (Rust 1.93.0, | |
| 40 | + | // zig 0.16.0) its cache was built with, and `latest` moving never changes a | |
| 41 | + | // release under it. To move on: `docker pull ghcr.io/rust-cross/cargo-zigbuild:latest` | |
| 42 | + | // and take the digest from `docker image inspect --format '{{json .RepoDigests}}'`. | |
| 43 | + | // The Caches workflow keys the build's cache by it (ci-cache.mjs keys: | |
| 44 | + | // zigbuild), so a new digest starts a new entry. | |
| 45 | + | export const BUILDER = "ghcr.io/rust-cross/cargo-zigbuild@sha256:d8313491ec5798de0633fdc1c5753761bff79967bea69076020dc78121b2cca8"; | |
| 39 | 46 | export const TARGETS = { | |
| 40 | 47 | "linux-x64": { triple: "x86_64-unknown-linux-musl", file: "g1t-linux-x64" }, | |
| 41 | 48 | "linux-arm64": { triple: "aarch64-unknown-linux-musl", file: "g1t-linux-arm64" }, | |
| ⋯ | |||
| 78 | 85 | return body; | |
| 79 | 86 | } | |
| 80 | 87 | ||
| 88 | + | /** | |
| 89 | + | * The builder container's flags. Its target is target/zig in the | |
| 90 | + | * checkout, which the workflows cache (cli-release-…). In a workflow job, | |
| 91 | + | * Cargo's registry is the job's own ~/.cargo/registry, which the crates | |
| 92 | + | * entry (cargo-crates-…) restores; elsewhere a Docker volume keeps it. | |
| 93 | + | */ | |
| 94 | + | export function builderArgs({ root = ROOT, env = process.env, home = homedir() } = {}) { | |
| 95 | + | const registry = env.GITHUB_ACTIONS === "true" ? `${join(env.CARGO_HOME || join(home, ".cargo"), "registry").replaceAll("\\", "/")}` : "g1t-cargo-registry"; | |
| 96 | + | return [ | |
| 97 | + | "-e", "CARGO_TARGET_DIR=/src/target/zig", | |
| 98 | + | "-v", `${root.replaceAll("\\", "/")}:/src`, | |
| 99 | + | "-v", `${registry}:/usr/local/cargo/registry`, | |
| 100 | + | "-w", "/src", BUILDER, | |
| 101 | + | ]; | |
| 102 | + | } | |
| 103 | + | ||
| 104 | + | /** | |
| 105 | + | * What the container runs. It runs as root, so on a host with users what | |
| 106 | + | * it wrote is handed back to the one running this, for the cache to save | |
| 107 | + | * and prune. | |
| 108 | + | */ | |
| 109 | + | export function builderScript(triples, owner = process.getuid ? `${process.getuid()}:${process.getgid()}` : null) { | |
| 110 | + | const build = `set -e; for t in ${triples}; do rustup target add $t >/dev/null 2>&1; cargo zigbuild --release --locked --package g1t --target $t; done`; | |
| 111 | + | return owner ? `trap 'chown -R ${owner} /src/target/zig /usr/local/cargo/registry' EXIT; ${build}` : build; | |
| 112 | + | } | |
| 113 | + | ||
| 81 | 114 | function build() { | |
| 82 | 115 | const v = version(); | |
| 83 | 116 | const dir = outDir(v); | |
| ⋯ | |||
| 85 | 118 | const triples = Object.values(TARGETS).map((t) => t.triple).join(" "); | |
| 86 | 119 | // One container builds every platform; its target folder is kept apart | |
| 87 | 120 | // from the host's so the two never mix. | |
| 88 | − | run("docker", [ | |
| 89 | − | "run", "--rm", | |
| 90 | − | "-e", "CARGO_TARGET_DIR=/src/target/zig", | |
| 91 | − | "-v", `${ROOT.replaceAll("\\", "/")}:/src`, | |
| 92 | − | "-v", "g1t-cargo-registry:/usr/local/cargo/registry", | |
| 93 | − | "-w", "/src", BUILDER, "sh", "-c", | |
| 94 | − | `set -e; for t in ${triples}; do rustup target add $t >/dev/null 2>&1; cargo zigbuild --release --locked --package g1t --target $t; done`, | |
| 95 | − | ], { env: { ...process.env, MSYS_NO_PATHCONV: "1" } }); | |
| 121 | + | run("docker", ["run", "--rm", ...builderArgs(), "sh", "-c", builderScript(triples)], { env: { ...process.env, MSYS_NO_PATHCONV: "1" } }); | |
| 96 | 122 | for (const target of Object.values(TARGETS)) { | |
| 97 | 123 | const built = join(ROOT, "target", "zig", target.triple, "release", target.exe ? "g1t.exe" : "g1t"); | |
| 98 | 124 | writeFileSync(join(dir, target.file), readFileSync(built)); | |
| 6 | 6 | import { join } from "node:path"; | |
| 7 | 7 | import { test } from "node:test"; | |
| 8 | 8 | ||
| 9 | − | import { TARGETS, manifest, sha256, version } from "./cli-release.mjs"; | |
| 9 | + | import { BUILDER, TARGETS, builderArgs, builderScript, manifest, sha256, version } from "./cli-release.mjs"; | |
| 10 | 10 | import { keygen, signBytes, verifyBytes } from "./runner-release.mjs"; | |
| 11 | 11 | ||
| 12 | + | test("a workflow's build uses the job's Cargo registry, which the crates cache restores", () => { | |
| 13 | + | const job = builderArgs({ root: "/home/runner/work/repo", env: { GITHUB_ACTIONS: "true" }, home: "/home/node" }); | |
| 14 | + | assert.ok(job.includes("/home/node/.cargo/registry:/usr/local/cargo/registry"), job.join(" ")); | |
| 15 | + | assert.ok(job.includes("/home/runner/work/repo:/src")); | |
| 16 | + | assert.equal(job.at(-1), BUILDER); | |
| 17 | + | // Elsewhere a Docker volume keeps it. | |
| 18 | + | assert.ok(builderArgs({ root: "/x", env: {}, home: "/h" }).includes("g1t-cargo-registry:/usr/local/cargo/registry")); | |
| 19 | + | // What the container wrote as root goes back to the user, even when the build fails. | |
| 20 | + | const script = builderScript("x86_64-unknown-linux-musl", "1000:1000"); | |
| 21 | + | assert.match(script, /^trap 'chown -R 1000:1000 \/src\/target\/zig \/usr\/local\/cargo\/registry' EXIT; set -e;/); | |
| 22 | + | assert.ok(!builderScript("t", null).includes("chown")); | |
| 23 | + | }); | |
| 24 | + | ||
| 12 | 25 | test("the version is crates/g1t's", () => { | |
| 13 | 26 | assert.match(version(), /^\d+\.\d+\.\d+/); | |
| 14 | 27 | }); |
| 18 | 18 | // the week before this were version bumps, and each saved every | |
| 19 | 19 | // Rust cache again (the test build's 0.7 to 1.7 GB each time). | |
| 20 | 20 | // desktop the same for apps/desktop/src-tauri/Cargo.lock. | |
| 21 | + | // zigbuild the first 12 hex characters of the cargo-zigbuild image's | |
| 22 | + | // digest (scripts/cli-release.mjs): the CLI's release builds in | |
| 23 | + | // it, with its own Rust, so its cache is keyed by the image | |
| 24 | + | // rather than by this machine's rustc. | |
| 21 | 25 | // | |
| 22 | 26 | // prune runs Cargo again with --message-format=json once the build is done | |
| 23 | 27 | // (everything is fresh, so it only lists what it built) and deletes, from | |
| ⋯ | |||
| 35 | 39 | import { basename, join } from "node:path"; | |
| 36 | 40 | import { fileURLToPath } from "node:url"; | |
| 37 | 41 | ||
| 42 | + | import { BUILDER } from "../cli-release.mjs"; | |
| 43 | + | ||
| 38 | 44 | const ROOT = fileURLToPath(new URL("../..", import.meta.url)); | |
| 39 | 45 | ||
| 40 | 46 | /** The `[[package]]` blocks of a Cargo.lock that come from a registry or git: not the workspace's own. */ | |
| ⋯ | |||
| 52 | 58 | return createHash("sha256").update(externalPackages(lock).join("\n\n")).digest("hex").slice(0, 16); | |
| 53 | 59 | } | |
| 54 | 60 | ||
| 61 | + | /** 12 hex characters of an image reference's digest (`name@sha256:…`); null for a tag, which moves. */ | |
| 62 | + | export function imageDigest(reference) { | |
| 63 | + | return reference.match(/@sha256:([0-9a-f]{64})$/)?.[1].slice(0, 12) ?? null; | |
| 64 | + | } | |
| 65 | + | ||
| 55 | 66 | /** `1.99.0` from `rustc 1.99.0 (90b35a623 2026-09-30)`; null if it is not that. */ | |
| 56 | 67 | export function rustVersion(text) { | |
| 57 | 68 | return text.match(/^rustc (\S+)/)?.[1] ?? null; | |
| ⋯ | |||
| 61 | 72 | const rustc = spawnSync("rustc", ["-V"], { encoding: "utf8" }); | |
| 62 | 73 | const rust = rustVersion(rustc.stdout ?? ""); | |
| 63 | 74 | if (!rust) throw new Error(`rustc -V said ${JSON.stringify(rustc.stdout ?? rustc.error?.message)}`); | |
| 64 | − | const outputs = { rust }; | |
| 75 | + | const zigbuild = imageDigest(BUILDER); | |
| 76 | + | if (!zigbuild) throw new Error(`scripts/cli-release.mjs builds in ${BUILDER}, which is not pinned by digest`); | |
| 77 | + | const outputs = { rust, zigbuild }; | |
| 65 | 78 | for (const [name, path] of [ | |
| 66 | 79 | ["cargo", "Cargo.lock"], | |
| 67 | 80 | ["desktop", "apps/desktop/src-tauri/Cargo.lock"], | |
| 3 | 3 | import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; | |
| 4 | 4 | import { tmpdir } from "node:os"; | |
| 5 | 5 | import { join } from "node:path"; | |
| 6 | − | import { depsHash, externalPackages, rustVersion, stale, unitHash, usedHashes } from "./ci-cache.mjs"; | |
| 6 | + | import { depsHash, externalPackages, imageDigest, rustVersion, stale, unitHash, usedHashes } from "./ci-cache.mjs"; | |
| 7 | + | import { BUILDER } from "../cli-release.mjs"; | |
| 7 | 8 | ||
| 8 | 9 | const LOCK = (runner, serde) => `# This file is automatically @generated by Cargo. | |
| 9 | 10 | version = 4 | |
| ⋯ | |||
| 35 | 36 | assert.equal(rustVersion(""), null); | |
| 36 | 37 | }); | |
| 37 | 38 | ||
| 39 | + | test("the CLI's builder image is pinned, and keys its cache by its digest", () => { | |
| 40 | + | assert.match(imageDigest(BUILDER), /^[0-9a-f]{12}$/); | |
| 41 | + | assert.equal(imageDigest(`ghcr.io/rust-cross/cargo-zigbuild@sha256:${"ab".repeat(32)}`), "abababababab"); | |
| 42 | + | assert.equal(imageDigest("ghcr.io/rust-cross/cargo-zigbuild:latest"), null); | |
| 43 | + | }); | |
| 44 | + | ||
| 38 | 45 | test("a unit's hash is read from its files and folders", () => { | |
| 39 | 46 | assert.equal(unitHash("libserde-0123456789abcdef.rlib"), "0123456789abcdef"); | |
| 40 | 47 | assert.equal(unitHash("serde-0123456789abcdef"), "0123456789abcdef"); | |
| 15 | 15 | # install downloads the pinned release and checks its sha256, starts the | |
| 16 | 16 | # server (which reads and writes a check entry in the cache), and only then | |
| 17 | 17 | # sets RUSTC_WRAPPER for the steps after it. If anything there fails, the | |
| 18 | − | # job builds as it did without sccache, and the step says why. | |
| 18 | + | # job builds as it did without sccache, and the step says why. A release | |
| 19 | + | # tag's run only reads: a run restores from its own ref, then its base, then | |
| 20 | + | # main, so what a tag saved no later run would read | |
| 21 | + | # (docs.g1t.sh/guides/actions/#the-cache); it reads what main's runs saved, | |
| 22 | + | # the Caches workflow's among them. | |
| 19 | 23 | # stats prints the server's hits and misses, and adds them to the job's | |
| 20 | 24 | # summary. | |
| 21 | 25 | # | |
| ⋯ | |||
| 40 | 44 | # sccache cannot cache incremental builds; a fresh checkout gains nothing | |
| 41 | 45 | # from them anyway. | |
| 42 | 46 | export CARGO_INCREMENTAL=0 | |
| 47 | + | # Read only, on a tag: sccache's GHA backend then makes no CreateCacheEntry | |
| 48 | + | # call, and still looks every compile up. | |
| 49 | + | if [ "${GITHUB_REF_TYPE:-}" = tag ]; then | |
| 50 | + | export SCCACHE_GHA_RW_MODE=READ_ONLY | |
| 51 | + | fi | |
| 43 | 52 | ||
| 44 | 53 | warn() { echo "::warning title=sccache::$1, so this job builds without it"; } | |
| 45 | 54 | ||
| ⋯ | |||
| 75 | 84 | echo "SCCACHE_IDLE_TIMEOUT=$SCCACHE_IDLE_TIMEOUT" | |
| 76 | 85 | echo "SCCACHE_IGNORE_SERVER_IO_ERROR=$SCCACHE_IGNORE_SERVER_IO_ERROR" | |
| 77 | 86 | echo "CARGO_INCREMENTAL=$CARGO_INCREMENTAL" | |
| 87 | + | if [ -n "${SCCACHE_GHA_RW_MODE:-}" ]; then echo "SCCACHE_GHA_RW_MODE=$SCCACHE_GHA_RW_MODE"; fi | |
| 78 | 88 | } >> "$GITHUB_ENV" | |
| 79 | − | echo "sccache ${VERSION}: rustc goes through it from here on, cached in this repository's Actions cache." | |
| 89 | + | if [ -n "${SCCACHE_GHA_RW_MODE:-}" ]; then | |
| 90 | + | echo "sccache ${VERSION}: rustc goes through it from here on, reading this repository's Actions cache; a tag's run saves nothing to it." | |
| 91 | + | else | |
| 92 | + | echo "sccache ${VERSION}: rustc goes through it from here on, cached in this repository's Actions cache." | |
| 93 | + | fi | |
| 80 | 94 | } | |
| 81 | 95 | ||
| 82 | 96 | stats() { | |
Comments
Sign in to comment.