Skip to content

Where people and agents ship software together. The open-source git platform for the whole job: issues, agents, checks and deploys to the edge.

g1t.sh

Commit

g1t's own workflows cache by one scheme: main saves, pull requests and release tags restore, and Rust caches are keyed by the toolchain and the dependencies rather than by Cargo.lock. Measured over the last two days of runs: 25 of the 31 changes to Cargo.lock in a week were a release's version bump, and each saved every Rust cache again; CI's test build, restored from older entries and saved again each time, grew from 671 MB to 1,687 MB in a day (its restore from 15 s to a minute, toward the 2 GB an entry may hold), and with its superseded copies filled the repository's 10 GiB; release tags never read another tag's entries, so 10 of 14 desktop Linux builds, 12 of 13 Windows builds and most runner builds found nothing. Now scripts/deploy/ci-cache.mjs gives each job rustc's version and a hash of Cargo.lock's crates.io and git packages, and prunes a restored Cargo target to what the build used before it is saved; CI's tests build with line tables only; CI's Rust job fetches every crate and alone saves them, with the index; the runner's image build keeps target/release, where its build scripts and proc macros go (87 crates were compiled again on every deploy); apt's downloads for Tauri are cached (scripts/apt-install.sh, 110 s of every desktop job); cargo-xwin is pinned and kept with the MSVC CRT. The new Caches workflow looks every key up on each push to main and builds only what is missing: npm's downloads (moved from Deploy), the runner release per target and the desktop release's Linux and Windows builds, which the release workflows now only restore. Tests hold every entry to one set of paths across workflows, every restored entry to a saver on main, releases to restoring only, Rust keys to the toolchain and dependencies, and pruning before saving; CONTRIBUTING.md says how it fits together, and the Actions guide says how a tag's run restores and that a lookup counts as a use.

syntaqxcommitted Parent930d294Browse files
12 files+1176−1770/12 viewed
+354−0
1+# Keeps, on main, the cache entries that only other workflows' runs would
2+# otherwise make. A run restores from its own ref, then its pull request's
3+# base branch, then main (docs.g1t.sh/guides/actions/#the-cache): a release
4+# tag's run never sees another tag's entries, and before this every release
5+# built its Rust from the start (desktop: 10 of 14 Linux and 12 of 13
6+# Windows builds found nothing). So what a release restores is saved here.
7+#
8+# look looks every key up without downloading anything; a
9+# lookup counts as a use, so a warm entry is never taken
10+# for an unused one, and an entry past its 28 days is
11+# simply made again
12+# npm npm's downloads for the whole lockfile, which every npm
13+# install in every workflow starts from
14+# runner the runner release's build, for each target
15+# desktop-linux the desktop release's Linux build
16+# desktop-windows its Windows cross build, cargo-xwin, the MSVC CRT and
17+# apt's downloads for it
18+#
19+# A job runs only when its key is missing (a new lockfile, a new toolchain,
20+# or an expired entry), and builds what the release builds. Nothing waits for
21+# this workflow, and a failed build only leaves the cache as it was. CI
22+# keeps the rest of main's caches (ci.yml); CONTRIBUTING.md ("How g1t's own
23+# CI caches") has the whole scheme, and the paths and keys here must stay
24+# the same as the steps that restore them (repository_workflows.rs checks).
25+name: Caches
26+
27+on:
28+ push:
29+ branches: [main]
30+ workflow_dispatch:
31+
32+permissions:
33+ contents: read
34+
35+# A build half done saves nothing, so one is never cut off; the next run
36+# waits, and finds the key saved.
37+concurrency:
38+ group: caches-${{ github.ref }}
39+ cancel-in-progress: false
40+
41+env:
42+ CARGO_TERM_COLOR: never
43+
44+jobs:
45+ look:
46+ name: What is missing
47+ runs-on: ubuntu-latest
48+ timeout-minutes: 10
49+ outputs:
50+ npm: ${{ steps.npm.outputs.cache-hit != 'true' }}
51+ runner: ${{ steps.missing.outputs.runner }}
52+ desktop-linux: ${{ steps.desktop-linux.outputs.cache-hit != 'true' }}
53+ desktop-windows: ${{ steps.desktop-windows.outputs.cache-hit != 'true' || steps.xwin.outputs.cache-hit != 'true' || steps.apt-windows.outputs.cache-hit != 'true' }}
54+ steps:
55+ - uses: actions/checkout@v5
56+ - name: Cache keys
57+ id: keys
58+ run: node scripts/deploy/ci-cache.mjs keys
59+ - id: npm
60+ uses: actions/cache/restore@v4
61+ with:
62+ path: ~/.npm
63+ key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
64+ lookup-only: true
65+ - id: zig
66+ uses: actions/cache/restore@v4
67+ with:
68+ path: |
69+ ~/.local/lib/python3*/site-packages/ziglang
70+ ~/.local/bin
71+ ~/.cargo/bin/cargo-zigbuild
72+ key: zig-0.13.0-zigbuild-${{ runner.os }}-1
73+ lookup-only: true
74+ - id: runner-linux-x64
75+ uses: actions/cache/restore@v4
76+ with:
77+ path: |
78+ target
79+ !target/runner-release
80+ !target/**/incremental
81+ key: runner-release-${{ runner.os }}-linux-x64-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
82+ lookup-only: true
83+ - id: runner-linux-arm64
84+ uses: actions/cache/restore@v4
85+ with:
86+ path: |
87+ target
88+ !target/runner-release
89+ !target/**/incremental
90+ key: runner-release-${{ runner.os }}-linux-arm64-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
91+ lookup-only: true
92+ - id: runner-windows-x64
93+ uses: actions/cache/restore@v4
94+ with:
95+ path: |
96+ target
97+ !target/runner-release
98+ !target/**/incremental
99+ key: runner-release-${{ runner.os }}-windows-x64-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
100+ lookup-only: true
101+ - id: desktop-linux
102+ uses: actions/cache/restore@v4
103+ with:
104+ path: |
105+ ~/.cargo/registry/index
106+ ~/.cargo/registry/cache
107+ apps/desktop/src-tauri/target/release
108+ !apps/desktop/src-tauri/target/release/bundle
109+ !apps/desktop/src-tauri/target/**/incremental
110+ key: desktop-release-linux-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
111+ lookup-only: true
112+ - id: desktop-windows
113+ uses: actions/cache/restore@v4
114+ with:
115+ path: |
116+ ~/.cargo/registry/index
117+ ~/.cargo/registry/cache
118+ apps/desktop/src-tauri/target/release
119+ apps/desktop/src-tauri/target/x86_64-pc-windows-msvc/release
120+ !apps/desktop/src-tauri/target/**/bundle
121+ !apps/desktop/src-tauri/target/**/incremental
122+ key: desktop-release-windows-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
123+ lookup-only: true
124+ - id: xwin
125+ uses: actions/cache/restore@v4
126+ with:
127+ path: |
128+ ~/.cargo/bin/cargo-xwin
129+ ~/.cache/cargo-xwin
130+ key: xwin-${{ runner.os }}-0.23.1
131+ lookup-only: true
132+ - id: apt-windows
133+ uses: actions/cache/restore@v4
134+ with:
135+ path: ~/.cache/apt/tauri-windows
136+ key: apt-tauri-windows-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
137+ lookup-only: true
138+ # The runner's targets to build: those whose build is missing, or all
139+ # of them when zig is (each saves it, the first to finish wins).
140+ - name: The runner's targets to build
141+ id: missing
142+ env:
143+ ZIG: ${{ steps.zig.outputs.cache-hit }}
144+ LINUX_X64: ${{ steps.runner-linux-x64.outputs.cache-hit }}
145+ LINUX_ARM64: ${{ steps.runner-linux-arm64.outputs.cache-hit }}
146+ WINDOWS_X64: ${{ steps.runner-windows-x64.outputs.cache-hit }}
147+ # Each with its Rust triple, as scripts/runner-release.mjs names it.
148+ run: |
149+ targets=()
150+ [ "$ZIG" = true ] && [ "$LINUX_X64" = true ] || targets+=('{"target":"linux-x64","triple":"x86_64-unknown-linux-musl"}')
151+ [ "$ZIG" = true ] && [ "$LINUX_ARM64" = true ] || targets+=('{"target":"linux-arm64","triple":"aarch64-unknown-linux-musl"}')
152+ [ "$ZIG" = true ] && [ "$WINDOWS_X64" = true ] || targets+=('{"target":"windows-x64","triple":"x86_64-pc-windows-gnu"}')
153+ list="[$(IFS=,; echo "${targets[*]}")]"
154+ echo "runner=$list" >> "$GITHUB_OUTPUT"
155+ echo "The runner's targets to build: $list"
156+
157+ # npm's downloads for the whole lockfile, saved once for each lockfile.
158+ # Before, every job saved its own part under one key, and the first to
159+ # finish (Wrangler alone, 73 MB) stood in for the rest: each stage then
160+ # downloaded hundreds of MB through the runner's egress, 50 MB tarballs
161+ # among them. Starts from the last lockfile's, with scripts off.
162+ npm:
163+ name: npm's downloads
164+ needs: look
165+ if: ${{ needs.look.outputs.npm == 'true' }}
166+ runs-on: ubuntu-latest
167+ timeout-minutes: 20
168+ steps:
169+ - uses: actions/checkout@v5
170+ - name: Start from the last lockfile's
171+ uses: actions/cache/restore@v4
172+ with:
173+ path: ~/.npm
174+ key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
175+ restore-keys: npm-full-${{ runner.os }}-
176+ # A failed download leaves the cache as it was, and the run green.
177+ - name: Download every package
178+ id: download
179+ continue-on-error: true
180+ run: node scripts/deploy/npm-ci.mjs --prefer-offline --ignore-scripts --no-audit --no-fund
181+ - name: Save them
182+ if: ${{ steps.download.outcome == 'success' }}
183+ uses: actions/cache/save@v4
184+ with:
185+ path: ~/.npm
186+ key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
187+
188+ # The steps of runner-release.yml's binaries job that fill its cache.
189+ runner:
190+ name: The runner's build for ${{ matrix.target }}
191+ needs: look
192+ if: ${{ needs.look.outputs.runner != '[]' }}
193+ runs-on: ubuntu-latest
194+ timeout-minutes: 60
195+ strategy:
196+ fail-fast: false
197+ matrix:
198+ include: ${{ fromJSON(needs.look.outputs.runner) }}
199+ env:
200+ G1T_RUNNER_RELEASE_KEY: ${{ vars.RUNNER_RELEASE_PUBLIC_KEY }}
201+ steps:
202+ - uses: actions/checkout@v5
203+ - name: Cache zig and cargo-zigbuild
204+ id: zig
205+ uses: actions/cache@v4
206+ with:
207+ path: |
208+ ~/.local/lib/python3*/site-packages/ziglang
209+ ~/.local/bin
210+ ~/.cargo/bin/cargo-zigbuild
211+ key: zig-0.13.0-zigbuild-${{ runner.os }}-1
212+ - name: Install zig and cargo-zigbuild
213+ if: steps.zig.outputs.cache-hit != 'true'
214+ run: |
215+ pip install --user --break-system-packages ziglang==0.13.0
216+ cargo install --locked cargo-zigbuild
217+ - name: zig on the PATH
218+ run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
219+ - name: Cache keys
220+ id: keys
221+ run: node scripts/deploy/ci-cache.mjs keys
222+ - name: Cache crates
223+ uses: actions/cache/restore@v4
224+ with:
225+ path: |
226+ ~/.cargo/registry/index
227+ ~/.cargo/registry/cache
228+ key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }}
229+ restore-keys: cargo-crates-${{ runner.os }}-
230+ - name: Cache the runner's build
231+ id: runner-build
232+ uses: actions/cache@v4
233+ with:
234+ path: |
235+ target
236+ !target/runner-release
237+ !target/**/incremental
238+ key: runner-release-${{ runner.os }}-${{ matrix.target }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
239+ restore-keys: runner-release-${{ runner.os }}-${{ matrix.target }}-${{ steps.keys.outputs.rust }}-
240+ - name: sccache
241+ run: bash scripts/sccache.sh install
242+ - name: Build
243+ run: node scripts/runner-release.mjs build --targets ${{ matrix.target }}
244+ - name: Only this build's files are saved
245+ if: ${{ steps.runner-build.outputs.cache-hit != 'true' }}
246+ run: node scripts/deploy/ci-cache.mjs prune target/release target/${{ matrix.triple }}/release -- zigbuild --release --locked --package g1t-runner --target ${{ matrix.triple }}
247+ - name: sccache's hits and misses
248+ if: ${{ always() }}
249+ run: bash scripts/sccache.sh stats
250+
251+ # desktop-release.yml's Linux build, without its installers: `tauri
252+ # build --no-bundle` compiles exactly what `tauri build` does, and needs
253+ # no signing key.
254+ desktop-linux:
255+ name: The desktop app's Linux build
256+ needs: look
257+ if: ${{ needs.look.outputs.desktop-linux == 'true' }}
258+ # The larger machine: linking Tauri on the small one dies with a bus error.
259+ runs-on: g1t-4core
260+ timeout-minutes: 60
261+ steps:
262+ - uses: actions/checkout@v5
263+ - name: Cache keys
264+ id: keys
265+ run: node scripts/deploy/ci-cache.mjs keys
266+ # Saved by CI's desktop job; only restored here.
267+ - name: Cache apt's downloads
268+ uses: actions/cache/restore@v4
269+ with:
270+ path: ~/.cache/apt/tauri
271+ key: apt-tauri-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
272+ - name: What Tauri needs on Linux
273+ run: bash scripts/apt-install.sh tauri
274+ - name: Cache the release build
275+ uses: actions/cache@v4
276+ with:
277+ path: |
278+ ~/.cargo/registry/index
279+ ~/.cargo/registry/cache
280+ apps/desktop/src-tauri/target/release
281+ !apps/desktop/src-tauri/target/release/bundle
282+ !apps/desktop/src-tauri/target/**/incremental
283+ key: desktop-release-linux-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
284+ restore-keys: desktop-release-linux-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
285+ - name: Cache npm's downloads
286+ uses: actions/cache/restore@v4
287+ with:
288+ path: ~/.npm
289+ key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
290+ restore-keys: npm-full-${{ runner.os }}-
291+ - name: Install
292+ run: node scripts/deploy/npm-ci.mjs --prefer-offline --no-audit --no-fund
293+ - name: Build
294+ working-directory: apps/desktop
295+ run: npx tauri build --ci --no-bundle
296+
297+ # desktop-release.yml's Windows cross build, without its installer.
298+ desktop-windows:
299+ name: The desktop app's Windows build
300+ needs: look
301+ if: ${{ needs.look.outputs.desktop-windows == 'true' && vars.DESKTOP_WINDOWS_FROM_LINUX == 'true' && vars.DESKTOP_WINDOWS_RUNNER != 'true' }}
302+ runs-on: g1t-4core
303+ timeout-minutes: 60
304+ steps:
305+ - uses: actions/checkout@v5
306+ - name: Cache keys
307+ id: keys
308+ run: node scripts/deploy/ci-cache.mjs keys
309+ - name: Cache apt's downloads
310+ uses: actions/cache@v4
311+ with:
312+ path: ~/.cache/apt/tauri-windows
313+ key: apt-tauri-windows-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
314+ - name: What the cross build needs
315+ run: bash scripts/desktop-cross-tools.sh
316+ - name: Cache cargo-xwin and the MSVC CRT
317+ id: xwin
318+ uses: actions/cache@v4
319+ with:
320+ path: |
321+ ~/.cargo/bin/cargo-xwin
322+ ~/.cache/cargo-xwin
323+ key: xwin-${{ runner.os }}-0.23.1
324+ - name: cargo-xwin
325+ if: ${{ steps.xwin.outputs.cache-hit != 'true' }}
326+ run: cargo install --locked cargo-xwin@0.23.1
327+ - name: Cache the release build
328+ uses: actions/cache@v4
329+ with:
330+ path: |
331+ ~/.cargo/registry/index
332+ ~/.cargo/registry/cache
333+ apps/desktop/src-tauri/target/release
334+ apps/desktop/src-tauri/target/x86_64-pc-windows-msvc/release
335+ !apps/desktop/src-tauri/target/**/bundle
336+ !apps/desktop/src-tauri/target/**/incremental
337+ key: desktop-release-windows-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
338+ restore-keys: desktop-release-windows-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
339+ - name: Cache npm's downloads
340+ uses: actions/cache/restore@v4
341+ with:
342+ path: ~/.npm
343+ key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
344+ restore-keys: npm-full-${{ runner.os }}-
345+ - name: Install
346+ run: node scripts/deploy/npm-ci.mjs --prefer-offline --no-audit --no-fund
347+ - name: Build
348+ working-directory: apps/desktop
349+ run: |
350+ rustup target add x86_64-pc-windows-msvc
351+ npx tauri build --ci --runner cargo-xwin --target x86_64-pc-windows-msvc --no-bundle
352+ # What the CRT and SDK came to, for the cache's 2 GB an entry may hold.
353+ - name: How large the MSVC CRT is
354+ run: du -sh ~/.cache/cargo-xwin || true
+66−23
88 # deploy and ops scripts, and the deploy manifest
99 # build the site, sudo and the docs build as they deploy
1010 #
11−# On a push to main only `rust` runs, to keep main's caches current: a pull
12−# request's run restores from main's cache, never from another pull
13−# request's, so without it each pull request would start from nothing.
11+# On a push to main `rust` and `desktop` run, keeping main's caches current:
12+# a pull request's run restores from main's cache, never from another pull
13+# request's, so without them each pull request would start from nothing.
14+# What each cache holds, its key and which job saves it: CONTRIBUTING.md,
15+# "How g1t's own CI caches". The Caches workflow (caches.yml) keeps the
16+# ones only releases build.
1417 name: CI
1518
1619 on:
3740 name: Rust
3841 runs-on: g1t-4core
3942 timeout-minutes: 45
43+ env:
44+ # Line tables, not full debug info: a panic still names its file and
45+ # line, and the test build is a fraction of the size to link, keep
46+ # and restore (at full debug info its cache entry reached 1.7 GB of
47+ # the 2 GB an entry may hold).
48+ CARGO_PROFILE_DEV_DEBUG: line-tables-only
4049 steps:
4150 - uses: actions/checkout@v5
51+ # rust: the toolchain; cargo: Cargo.lock's crates.io and git packages,
52+ # not the workspace's own versions (scripts/deploy/ci-cache.mjs).
53+ - name: Cache keys
54+ id: keys
55+ run: node scripts/deploy/ci-cache.mjs keys
56+ # Every crate the lockfile names, for every target (the fetch below):
57+ # saved by this job alone, and only restored by Deploy and the
58+ # releases, which each download part of it.
4259 - name: Cache crates
4360 uses: actions/cache@v4
4461 with:
45− path: ~/.cargo/registry/cache
46− key: cargo-crates-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
62+ path: |
63+ ~/.cargo/registry/index
64+ ~/.cargo/registry/cache
65+ key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }}
4766 restore-keys: cargo-crates-${{ runner.os }}-
67+ - name: Every crate the lockfile names
68+ run: cargo fetch --locked
69+ # The compiled dependencies and workspace crates. Saved when the
70+ # dependencies or the toolchain change; otherwise the last entry for
71+ # this toolchain is a start, and only this build's files go into the
72+ # new one (the prune step below).
4873 - name: Cache the test build
74+ id: test-build
4975 uses: actions/cache@v4
5076 with:
5177 path: |
5278 target/debug
5379 !target/debug/incremental
54− key: cargo-test-${{ runner.os }}-${{ hashFiles('Cargo.lock', 'services/runner/base.json') }}
55− restore-keys: cargo-test-${{ runner.os }}-
56− # The workspace's library crates, which Cargo compiles again on every
57− # checkout, come back from the repository's Actions cache when their
58− # inputs did not change (scripts/sccache.sh). Test harnesses and
59− # Workers' own crates are linked, and still compiled.
80+ key: cargo-test-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
81+ restore-keys: cargo-test-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
82+ # The workspace's library crates that Cargo compiles again come back
83+ # from the repository's Actions cache when their inputs did not change
84+ # (scripts/sccache.sh). Test harnesses and Workers' own crates are
85+ # linked, and still compiled.
6086 - name: sccache
6187 run: bash scripts/sccache.sh install
6288 - name: Tests
6389 run: cargo test --workspace --locked --quiet
90+ # A restored target keeps the old versions of whatever changed beside
91+ # the new ones; only what this build used goes into the new entry.
92+ - name: Only this build's files are saved
93+ if: ${{ steps.test-build.outputs.cache-hit != 'true' }}
94+ run: node scripts/deploy/ci-cache.mjs prune target/debug -- test --workspace --locked --no-run
6495 - name: sccache's hits and misses
6596 if: ${{ always() }}
6697 run: bash scripts/sccache.sh stats
127158 timeout-minutes: 30
128159 steps:
129160 - uses: actions/checkout@v5
161+ - name: Cache keys
162+ id: keys
163+ run: node scripts/deploy/ci-cache.mjs keys
164+ # apt's lists and downloads (scripts/apt-install.sh): 110 s of every
165+ # run before, mostly fetching 335 MB. Saved here, on main; the desktop
166+ # release's Linux build restores the same entry.
167+ - name: Cache apt's downloads
168+ uses: actions/cache@v4
169+ with:
170+ path: ~/.cache/apt/tauri
171+ key: apt-tauri-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
130172 - name: What Tauri needs on Linux
131− # Tried three times: the mirror has answered with a mismatched package before.
132− run: |
133− ok=0
134− for try in 1 2 3; do
135− if sudo apt-get update && sudo apt-get install -y libwebkit2gtk-4.1-dev build-essential libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev; then ok=1; break; fi
136− sudo apt-get clean; sleep 15
137− done
138− [ "$ok" = 1 ]
139− - uses: actions/cache@v4
173+ run: bash scripts/apt-install.sh tauri
174+ - name: Cache the desktop app's build
175+ id: desktop-build
176+ uses: actions/cache@v4
140177 with:
141178 path: |
142− ~/.cargo/registry
143− apps/desktop/src-tauri/target
144− key: desktop-check-${{ hashFiles('apps/desktop/src-tauri/Cargo.lock') }}
179+ ~/.cargo/registry/index
180+ ~/.cargo/registry/cache
181+ apps/desktop/src-tauri/target/debug
182+ !apps/desktop/src-tauri/target/debug/incremental
183+ key: desktop-check-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
184+ restore-keys: desktop-check-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
145185 # Outside the workspace on purpose (Cargo.toml says why): checked and
146186 # tested here, built into installers by desktop-release.yml.
147187 - name: Check and test
148188 run: cargo test --locked --manifest-path apps/desktop/src-tauri/Cargo.toml --quiet
189+ - name: Only this build's files are saved
190+ if: ${{ steps.desktop-build.outputs.cache-hit != 'true' }}
191+ run: node scripts/deploy/ci-cache.mjs prune apps/desktop/src-tauri/target/debug -- test --locked --manifest-path apps/desktop/src-tauri/Cargo.toml --no-run
149192 - name: The release scripts' tests
150193 run: node --test scripts/desktop-release.test.mjs scripts/mobile-release.test.mjs scripts/cli-release.test.mjs
+31−56
7070 timeout-minutes: 20
7171 steps:
7272 - uses: actions/checkout@v5
73− # npm's downloads for the whole lockfile, which the npm-cache job
74− # keeps: an install reads them first and asks the registry only for
73+ # npm's downloads for the whole lockfile, which the Caches workflow
74+ # keeps (caches.yml): an install reads them first and asks the registry only for
7575 # what changed, so a dropped connection to it rarely stops the job.
7676 # Only restored: a job that installs part of the lockfile never
7777 # saves, or its part would stand in for the whole under that key.
8787 run: node scripts/deploy.mjs manifest --check
8888 - name: The deploy tool's tests
8989 run: npm run test:deploy
90−
91− # npm's downloads for the whole lockfile, saved once for each lockfile
92− # on main, which every install of this and the other workflows starts
93− # from. Before, every job saved its own part under one key, and the
94− # first to finish (Wrangler alone, 73 MB) stood in for the rest: each
95− # stage then downloaded hundreds of MB through the runner's egress,
96− # 50 MB tarballs among them. Nothing waits for this job, and a failed
97− # download only leaves the cache as it was.
98− npm-cache:
99− name: npm's downloads
100− runs-on: ubuntu-latest
101− timeout-minutes: 20
102− steps:
103− - uses: actions/checkout@v5
104− - name: Look for this lockfile's downloads
105− id: saved
106− uses: actions/cache/restore@v4
107− with:
108− path: ~/.npm
109− key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
110− lookup-only: true
111− - name: Start from the last lockfile's
112− if: ${{ steps.saved.outputs.cache-hit != 'true' }}
113− uses: actions/cache/restore@v4
114− with:
115− path: ~/.npm
116− key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
117− restore-keys: npm-full-${{ runner.os }}-
118− - name: Download every package
119− id: download
120− if: ${{ steps.saved.outputs.cache-hit != 'true' }}
121− continue-on-error: true
122− run: node scripts/deploy/npm-ci.mjs --prefer-offline --ignore-scripts --no-audit --no-fund
123− - name: Save them
124− if: ${{ steps.saved.outputs.cache-hit != 'true' && steps.download.outcome == 'success' }}
125− uses: actions/cache/save@v4
126− with:
127− path: ~/.npm
128− key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
12990
13091 # Runs beside check: nothing deploys until both have succeeded.
13192 plan:
231192 with:
232193 path: ~/.cache/worker-build
233194 key: worker-build-tools-${{ runner.os }}-${{ hashFiles('scripts/build-rust-worker.mjs') }}
195+ # rust: the toolchain; cargo: Cargo.lock's crates.io and git packages,
196+ # not the workspace's own versions (scripts/deploy/ci-cache.mjs).
197+ - name: Cache keys
198+ id: keys
199+ if: ${{ matrix.rust || matrix.image }}
200+ run: node scripts/deploy/ci-cache.mjs keys
201+ # Every crate the lockfile names, as CI's Rust job saves them on
202+ # main. Only restored: a stage downloads the crates of its own units,
203+ # and saving those would stand in for the whole under that key.
234204 - name: Cache crates
235− if: ${{ matrix.rust }}
236− uses: actions/cache@v4
205+ if: ${{ matrix.rust || matrix.image }}
206+ uses: actions/cache/restore@v4
237207 with:
238− path: ~/.cargo/registry/cache
239− key: cargo-crates-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
208+ path: |
209+ ~/.cargo/registry/index
210+ ~/.cargo/registry/cache
211+ key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }}
240212 restore-keys: cargo-crates-${{ runner.os }}-
241213 # The compiled dependencies of this job's units, for wasm32 and the
242214 # build scripts and proc macros they run. Cargo calls rustc again for
243215 # the workspace's own crates whatever is cached (a checkout's sources
244216 # are newer); sccache, below, answers those calls. So an entry is
245− # saved only when the dependencies change: a new Cargo.lock, or a new
246− # base image (base.json names its Rust). Otherwise the nearest earlier
247− # entry, of any group, is a start.
217+ # saved only when the dependencies or the toolchain change, not when
218+ # a release bumps a workspace crate's version; otherwise this group's
219+ # last entry for the toolchain is a start, else any group's.
248220 - name: Cache the Cargo target
249221 if: ${{ matrix.rust }}
250222 uses: actions/cache@v4
254226 target/wasm32-unknown-unknown/release
255227 !target/**/incremental
256228 !target/**/*.wasm
257− key: cargo-target-${{ runner.os }}-${{ matrix.group }}-${{ hashFiles('Cargo.lock', 'services/runner/base.json') }}
229+ key: cargo-workers-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ matrix.group }}-${{ steps.keys.outputs.cargo }}
258230 restore-keys: |
259− cargo-target-${{ runner.os }}-${{ matrix.group }}-
260− cargo-target-${{ runner.os }}-
231+ cargo-workers-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ matrix.group }}-
232+ cargo-workers-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
261233 # The runner's image: its binary, built natively for musl (the base
262234 # has musl-gcc; the target is added here), with its Cargo target kept
263− # between runs. The image itself is built and pushed with the job's
264− # own Docker Engine (scripts/deploy/image.mjs).
235+ # between runs: the musl target's, and target/release, where Cargo
236+ # puts the build scripts and proc macros it runs on this machine
237+ # (without it, 87 crates were compiled again on every deploy). The
238+ # image itself is built and pushed with the job's own Docker Engine
239+ # (scripts/deploy/image.mjs).
265240 - name: Rust for the runner
266241 if: ${{ matrix.image }}
267242 run: rustup target add x86_64-unknown-linux-musl
270245 uses: actions/cache@v4
271246 with:
272247 path: |
273− ~/.cargo/registry/cache
248+ target/release
274249 target/x86_64-unknown-linux-musl/release
275250 !target/**/incremental
276− key: runner-musl-${{ runner.os }}-${{ hashFiles('Cargo.lock', 'services/runner/base.json') }}
277− restore-keys: runner-musl-${{ runner.os }}-
251+ key: runner-musl-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
252+ restore-keys: runner-musl-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
278253 # Every rustc call that makes a library (the workspace's crates,
279254 # which Cargo compiles again on every checkout, and any dependency
280255 # not restored above) is looked up by its inputs in the repository's
+70−54
7575 timeout-minutes: 60 # the most g1t's own machines run a job
7676 steps:
7777 - uses: actions/checkout@v5
78+ # rust: the toolchain; desktop: the app's Cargo.lock's crates.io and
79+ # git packages, not its own version (scripts/deploy/ci-cache.mjs).
80+ - name: Cache keys
81+ id: keys
82+ run: node scripts/deploy/ci-cache.mjs keys
83+ # apt's lists and downloads, as CI's desktop job saves them on main.
84+ - name: Cache apt's downloads
85+ uses: actions/cache/restore@v4
86+ with:
87+ path: ~/.cache/apt/tauri
88+ key: apt-tauri-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
7889 - name: What Tauri needs on Linux
79− # Tried three times: the mirror has answered with a mismatched package before.
80− run: |
81− ok=0
82− for try in 1 2 3; do
83− if sudo apt-get update && sudo apt-get install -y libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev; then ok=1; break; fi
84− sudo apt-get clean; sleep 15
85− done
86− [ "$ok" = 1 ]
87− - uses: actions/cache@v4
90+ run: bash scripts/apt-install.sh tauri
91+ # The app's release build. A tag's run restores only from its own tag
92+ # and from main, never from the last release's, so the Caches
93+ # workflow (caches.yml) builds and saves this on main whenever the
94+ # key is new. Only restored here.
95+ - name: Cache the release build
96+ uses: actions/cache/restore@v4
8897 with:
8998 path: |
90− ~/.cargo/registry
91− apps/desktop/src-tauri/target
92− key: desktop-release-linux-${{ hashFiles('apps/desktop/src-tauri/Cargo.lock') }}
93− # npm's downloads, kept between runs: an install reads them first
94− # and asks the registry only for what changed, so a dropped
95− # connection to it rarely stops the job.
99+ ~/.cargo/registry/index
100+ ~/.cargo/registry/cache
101+ apps/desktop/src-tauri/target/release
102+ !apps/desktop/src-tauri/target/release/bundle
103+ !apps/desktop/src-tauri/target/**/incremental
104+ key: desktop-release-linux-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
105+ restore-keys: desktop-release-linux-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
106+ # npm's downloads for the whole lockfile, which the Caches workflow
107+ # keeps on main: an install reads them first and asks the registry
108+ # only for what changed. Only restored: a tag's entries are read by
109+ # nothing else.
96110 - name: Cache npm's downloads
97− uses: actions/cache@v4
111+ uses: actions/cache/restore@v4
98112 with:
99113 path: ~/.npm
100114 key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
165179 timeout-minutes: 60 # the most g1t's own machines run a job
166180 steps:
167181 - uses: actions/checkout@v5
182+ - name: Cache keys
183+ id: keys
184+ run: node scripts/deploy/ci-cache.mjs keys
185+ # apt's lists and downloads, which the Caches workflow keeps on main.
186+ - name: Cache apt's downloads
187+ uses: actions/cache/restore@v4
188+ with:
189+ path: ~/.cache/apt/tauri-windows
190+ key: apt-tauri-windows-${{ runner.os }}-${{ hashFiles('scripts/apt-install.sh') }}
168191 - name: What the cross build needs
169192 # The tray's Linux library too: Tauri's bundler looks for it on the
170193 # machine it runs on, whatever the target, and stops without it.
171194 run: |
172− ok=0
173− for try in 1 2 3; do
174− if sudo apt-get update && sudo apt-get install -y build-essential curl wget file libssl-dev libayatana-appindicator3-dev nsis clang lld llvm; then ok=1; break; fi
175− sudo apt-get clean; sleep 15
176− done
177− [ "$ok" = 1 ]
178− # cargo-xwin compiles C (ring's, among others) with clang-cl and
179− # links with lld-link and llvm-lib. Each is a mode of a tool
180− # Debian does install, chosen by the name it is run as: clang-cl
181− # is clang, lld-link is lld, and llvm-lib is llvm-ar. Debian ships
182− # none of the three names, so each is linked to its tool.
183− for pair in clang-cl:clang lld-link:lld llvm-lib:llvm-ar; do
184− name="${pair%%:*}" tool="${pair##*:}"
185− command -v "$name" >/dev/null && continue
186− found="$(command -v "$tool" || ls /usr/lib/llvm-*/bin/"$tool" 2>/dev/null | sort -V | tail -1)"
187− [ -n "$found" ] || { echo "::error::$tool is not installed, so there is no $name"; exit 1; }
188− sudo ln -sf "$(readlink -f "$found")" "/usr/local/bin/$name"
189− echo "$name: $(readlink -f "$found")"
190− done
191− # Tauri's installer script includes Win\RestartManager.nsh, which
192− # NSIS ships from 3.10 on; Debian's is 3.08. The header stands
193− # alone (defines and System calls), so it is added from NSIS 3.11's
194− # source, checked against its SHA-256.
195− rm_nsh=/usr/share/nsis/Include/Win/RestartManager.nsh
196− if [ ! -f "$rm_nsh" ]; then
197− curl -fsSL --retry 3 -o /tmp/RestartManager.nsh https://raw.githubusercontent.com/kichik/nsis/v311/Include/Win/RestartManager.nsh
198− echo "67149978412da9b283e2e1abe48a360093df9caf6363373e2cabe463978b7c0e /tmp/RestartManager.nsh" | sha256sum -c -
199− sudo install -m 0644 /tmp/RestartManager.nsh "$rm_nsh"
200− fi
201− cargo install --locked cargo-xwin
202− - uses: actions/cache@v4
195+ bash scripts/desktop-cross-tools.sh
196+ # cargo-xwin, pinned, and the MSVC CRT and Windows SDK it downloads on
197+ # first use: 88 s compiling it and 59 s downloading them in every
198+ # release before. The Caches workflow keeps them on main.
199+ - name: Cache cargo-xwin and the MSVC CRT
200+ id: xwin
201+ uses: actions/cache/restore@v4
202+ with:
203+ path: |
204+ ~/.cargo/bin/cargo-xwin
205+ ~/.cache/cargo-xwin
206+ key: xwin-${{ runner.os }}-0.23.1
207+ - name: cargo-xwin
208+ if: ${{ steps.xwin.outputs.cache-hit != 'true' }}
209+ run: cargo install --locked cargo-xwin@0.23.1
210+ # The app's Windows build, which the Caches workflow keeps on main.
211+ - name: Cache the release build
212+ uses: actions/cache/restore@v4
203213 with:
204214 path: |
205− ~/.cargo/registry
206− apps/desktop/src-tauri/target
207− key: desktop-release-windows-${{ hashFiles('apps/desktop/src-tauri/Cargo.lock') }}
208− # npm's downloads, kept between runs: an install reads them first
209− # and asks the registry only for what changed, so a dropped
210− # connection to it rarely stops the job.
215+ ~/.cargo/registry/index
216+ ~/.cargo/registry/cache
217+ apps/desktop/src-tauri/target/release
218+ apps/desktop/src-tauri/target/x86_64-pc-windows-msvc/release
219+ !apps/desktop/src-tauri/target/**/bundle
220+ !apps/desktop/src-tauri/target/**/incremental
221+ key: desktop-release-windows-${{ runner.os }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.desktop }}
222+ restore-keys: desktop-release-windows-${{ runner.os }}-${{ steps.keys.outputs.rust }}-
223+ # npm's downloads for the whole lockfile, which the Caches workflow
224+ # keeps on main: an install reads them first and asks the registry
225+ # only for what changed. Only restored: a tag's entries are read by
226+ # nothing else.
211227 - name: Cache npm's downloads
212− uses: actions/cache@v4
228+ uses: actions/cache/restore@v4
213229 with:
214230 path: ~/.npm
215231 key: npm-full-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
+31−16
4545 fail-fast: true
4646 matrix:
4747 target: [linux-x64, linux-arm64, windows-x64]
48+ env:
49+ G1T_RUNNER_RELEASE_KEY: ${{ vars.RUNNER_RELEASE_PUBLIC_KEY }}
4850 steps:
4951 - uses: actions/checkout@v5
5052 - name: The tag names this version
5254 run: |
5355 version="$(sed -n 's/^version = "\(.*\)"/\1/p' crates/runner/Cargo.toml | head -1)"
5456 [ "runner-v$version" = "${GITHUB_REF_NAME}" ] || { echo "::error::The tag is ${GITHUB_REF_NAME}, but crates/runner is $version"; exit 1; }
55− # zig and cargo-zigbuild, kept between releases: installing them took
56− # four minutes of every one. Change the key to take newer versions.
57+ # zig and cargo-zigbuild, kept on main by the Caches workflow
58+ # (caches.yml): installing them took four minutes of every release.
59+ # Change the key there and here to take newer versions.
5760 - name: Cache zig and cargo-zigbuild
5861 id: zig
59− uses: actions/cache@v4
62+ uses: actions/cache/restore@v4
6063 with:
6164 path: |
6265 ~/.local/lib/python3*/site-packages/ziglang
7174 cargo install --locked cargo-zigbuild
7275 - name: zig on the PATH
7376 run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
74− # Each target's build, kept between releases; a release after a
75− # Cargo.lock change starts from the last one's rather than from nothing.
76− - uses: actions/cache@v4
77+ # rust: the toolchain; cargo: Cargo.lock's crates.io and git packages,
78+ # not the workspace's own versions (scripts/deploy/ci-cache.mjs).
79+ - name: Cache keys
80+ id: keys
81+ run: node scripts/deploy/ci-cache.mjs keys
82+ # Every crate the lockfile names, as CI's Rust job saves them on main.
83+ - name: Cache crates
84+ uses: actions/cache/restore@v4
7785 with:
7886 path: |
79− ~/.cargo/registry
87+ ~/.cargo/registry/index
88+ ~/.cargo/registry/cache
89+ key: cargo-crates-${{ runner.os }}-${{ steps.keys.outputs.cargo }}
90+ restore-keys: cargo-crates-${{ runner.os }}-
91+ # Each target's build. A tag's run restores only from its own tag and
92+ # from main, never from the last release's tag, so the Caches workflow
93+ # (caches.yml) builds and saves this on main whenever the key is new;
94+ # a release finds it whole, compiles the workspace's crates that
95+ # changed, and saves nothing.
96+ - name: Cache the runner's build
97+ uses: actions/cache/restore@v4
98+ with:
99+ path: |
80100 target
81− key: runner-release-${{ matrix.target }}-${{ hashFiles('Cargo.lock') }}
82− restore-keys: runner-release-${{ matrix.target }}-
101+ !target/runner-release
102+ !target/**/incremental
103+ key: runner-release-${{ runner.os }}-${{ matrix.target }}-${{ steps.keys.outputs.rust }}-${{ steps.keys.outputs.cargo }}
104+ restore-keys: runner-release-${{ runner.os }}-${{ matrix.target }}-${{ steps.keys.outputs.rust }}-
83105 # Crates whose inputs did not change come back from the Actions cache
84106 # instead of being compiled (scripts/sccache.sh); without it, the job
85107 # builds as before.
86108 - name: sccache
87109 run: bash scripts/sccache.sh install
88− # The cache above brings back the last release's target/runner-release
89− # too: left there, it would go out with this one's, and the jobs after
90− # would find two of each file.
91− - name: Only this release's files
92− run: rm -rf target/runner-release
93110 - name: Build Linux and Windows
94111 # The macOS targets need Apple's SDK to link (the Security and
95112 # CoreFoundation frameworks), which zig does not carry, so they are
96113 # left out until this builds in the rust-cross/cargo-zigbuild image
97114 # as the CLI's release does (scripts/cli-release.mjs). A Mac runner
98115 # keeps the version it has, since the manifest names no file for it.
99− env:
100− G1T_RUNNER_RELEASE_KEY: ${{ vars.RUNNER_RELEASE_PUBLIC_KEY }}
101116 run: node scripts/runner-release.mjs build --targets ${{ matrix.target }}
102117 - name: sccache's hits
103118 if: always()
+45−0
164164 `/-/notify`, notify's phone push) deploys with the site and services.
165165 The guide is [The phone app](https://docs.g1t.sh/guides/mobile/).
166166
167+## How g1t's own CI caches
168+
169+g1t's workflows cache with `actions/cache` against g1t's own cache
170+([the Actions guide](https://docs.g1t.sh/guides/actions/#the-cache) has its
171+rules). Three of them shape everything here: a run restores from its own
172+ref, its pull request's base, then `main`, so a release tag never reads
173+another tag's entries; a key is written once; and a repository holds
174+10 GiB, past which the entries restored longest ago go. So:
175+
176+- **`main` saves; pull requests and tags restore.** CI's Rust and desktop
177+ jobs keep their caches on each push to `main`, and the Caches workflow
178+ (`.g1t/workflows/caches.yml`) builds and saves what only releases use,
179+ when its key is missing. Release workflows only restore.
180+- **Rust caches are keyed by the toolchain and the dependencies**, from
181+ `node scripts/deploy/ci-cache.mjs keys`: `rustc -V`, and a hash of the
182+ packages `Cargo.lock` takes from crates.io and git. A release bumping a
183+ workspace crate's version changes neither. A restore key never crosses
184+ toolchains.
185+- **A Cargo target restored from an older entry is pruned before it is
186+ saved** (`ci-cache.mjs prune`), to what this build used.
187+- **A partial install never saves a whole-install key.** npm's downloads
188+ and the crates are saved only by jobs that fetch all of them.
189+- **One set of paths per entry.** An entry's version is the hash of its
190+ paths, so every step that names it, in every workflow, names the same
191+ ones. `crates/actions/tests/repository_workflows.rs` holds all of this.
192+
193+| Entry | Key | Saved by | Restored by |
194+| --- | --- | --- | --- |
195+| npm's downloads | `npm-full-<os>-<package-lock.json>` | Caches (`npm`) | every npm install |
196+| Crates and their index | `cargo-crates-<os>-<deps>` | CI Rust (`cargo fetch` of the whole lockfile) | Deploy, the runner release |
197+| The test build | `cargo-test-<os>-<rust>-<deps>` | CI Rust | CI Rust on pull requests |
198+| Workers' builds | `cargo-workers-<os>-<rust>-<group>-<deps>` | Deploy | Deploy |
199+| The runner's image build | `runner-musl-<os>-<rust>-<deps>` | Deploy | Deploy |
200+| worker-build and its tools | `worker-build-…`, `worker-build-tools-…` | Deploy | Deploy |
201+| Compiled crates | sccache's, one per crate | every Rust job | every Rust job |
202+| The desktop app's check | `desktop-check-<os>-<rust>-<desktop deps>` | CI desktop | CI desktop on pull requests |
203+| apt for Tauri | `apt-tauri-<os>-<scripts/apt-install.sh>` | CI desktop | the desktop release |
204+| The runner release | `runner-release-<os>-<target>-<rust>-<deps>`, zig | Caches (`runner`) | the runner release |
205+| The desktop release | `desktop-release-linux-…`, `desktop-release-windows-…`, `xwin-…`, `apt-tauri-windows-…` | Caches (`desktop-linux`, `desktop-windows`) | the desktop release |
206+
207+To see whether a change helped, read a run's cache steps: `Cache restored
208+from key` names the entry (the key itself is a hit; another is a start),
209+`Cache Size` and the step's time say what it cost, and the post step says
210+whether it saved. `sccache's hits and misses` is on each Rust job's summary.
211+
167212 ## Speed
168213
169214 Pages are a few rounds of service calls, and each round costs a trip to
+11−7
793793 | One entry | Up to 2 GiB, compressed. A larger one is not saved, and the job goes on. |
794794 | A repository's entries | Up to 10 GiB together. Saving past it removes the entries restored longest ago. |
795795 | A free workspace | Its cache and [artifacts](#artifacts), in every repository, up to 2 GiB together. Past it, a save is refused (the job goes on) until old entries expire or artifacts are deleted; on the g1t plan there is no such cap. |
796−| How long | Until it has not been restored for 7 days, and at most 28 days after it was saved. |
797−| Which branch | An entry belongs to the branch, tag or pull request whose run saved it. A run restores from its own, then from the branch its pull request merges into, then from the default branch. |
798−| Keys | Written once on each branch: saving under a key that exists there does nothing. On each branch in turn, a restore finds its `key` exactly, else the newest entry whose key starts with one of its `restore-keys`. |
796+| How long | Until it has not been restored for 7 days, and at most 28 days after it was saved. A lookup (`lookup-only: true`) counts as a restore. |
797+| Which branch | An entry belongs to the branch, tag or pull request whose run saved it. A run restores from its own, then from the branch its pull request merges into, then from the default branch. A tag's run reads its own tag's entries and the default branch's, never another tag's. |
798+| Keys | Written once on each branch: saving under a key that exists there does nothing, and the step says the key is already cached. On each branch in turn, a restore finds its `key` exactly, else the newest entry whose key starts with its `key`, else the newest whose key starts with each of its `restore-keys` in order. |
799799 | `path` | Files and folders; globs, `**` included; `~/` is the home folder; a line starting with `!` leaves matching paths out. The same key saved for other paths is another entry. |
800800 | Compression | zstd. |
801801
802802 So a feature branch can read what `main` saved, but `main` never reads
803803 what a feature branch saved, and a pull request from outside the
804804 repository saves where nothing else ever reads it: nobody can plant an
805−entry that the default branch's builds restore. Actions that cache through the
805+entry that the default branch's builds restore. For the same reason, a
806+release built from a tag finds only what the default branch saved: have
807+a workflow on the default branch save what your releases restore. Actions that cache through the
806808 toolkit, such as `actions/setup-node` with `cache: npm`, follow the same
807809 rules.
808810
888890 it, from its paths and compression, as on GitHub: an entry `setup-node`
889891 saved is restored by `actions/cache` with the same key and paths, and
890892 the other way round.
891−- An entry the toolkit sends whole, which it does below 128 MB, is not
892− saved when it is over 100 MB, the most g1t takes in one request. The
893− step warns and the job goes on.
893+- Against any server but github.com, the toolkit uses the cache's older
894+ protocol, which sends an entry in 32 MB chunks, so an entry up to the
895+ cache's 2 GiB is saved. A client of the newer protocol that sends an
896+ entry whole in one request is held to 100 MB there; past it, the step
897+ warns and the job goes on.
894898 - The toolkit's artifact library refuses to run against any server but
895899 github.com, so an action that uploads artifacts with it directly fails
896900 with its own message. `actions/upload-artifact`,
+244−21
187187 assert_eq!(concurrency.cancel_in_progress, json!(false));
188188 assert_eq!(
189189 deploy.job_order(),
190− [
191− "check",
192− "npm-cache",
193− "plan",
194− "migrate",
195− "core",
196− "edge",
197− "front",
198− "smoke"
199− ]
200− );
201− // Nothing waits for npm's downloads to be saved.
202− assert!(
203− deploy
204− .jobs
205− .iter()
206− .all(|job| !job.needs.iter().any(|n| n == "npm-cache"))
190+ ["check", "plan", "migrate", "core", "edge", "front", "smoke"]
207191 );
208192 // The stages share their steps (a YAML alias), and read the token only
209193 // where they deploy.
274258 }
275259 }
276260 }
277− // main's whole cache is kept by the deploy workflow, beside its check.
278− assert!(
279− saving.contains(&"deploy.yml npm-cache".to_owned()),
280− "{saving:?}"
261+ // main's whole cache is kept by the Caches workflow.
262+ assert!(saving.contains(&"caches.yml npm".to_owned()), "{saving:?}");
263+}
264+
265+/// Every `actions/cache` step of every workflow: (file, job, step).
266+fn cache_steps() -> Vec<(String, workflow::Job, workflow::Step)> {
267+ let mut out = Vec::new();
268+ let mut names: Vec<String> = std::fs::read_dir(workflows_dir())
269+ .unwrap()
270+ .map(|e| e.unwrap().file_name().to_string_lossy().to_string())
271+ .filter(|n| n.ends_with(".yml"))
272+ .collect();
273+ names.sort();
274+ for name in names {
275+ for job in read(&name).jobs {
276+ for step in &job.steps {
277+ if step
278+ .uses
279+ .as_deref()
280+ .is_some_and(|u| u.starts_with("actions/cache"))
281+ {
282+ out.push((name.clone(), job.clone(), step.clone()));
283+ }
284+ }
285+ }
286+ }
287+ out
288+}
289+
290+/// What a cache step's key starts with before its first expression: the
291+/// entry it is, whatever its inputs (`cargo-test-`, `apt-tauri-windows-`).
292+fn family(key: &str) -> String {
293+ key.split("${{").next().unwrap_or_default().to_owned()
294+}
295+
296+fn saves(step: &workflow::Step) -> bool {
297+ let uses = step.uses.as_deref().unwrap_or_default();
298+ (uses.starts_with("actions/cache@") || uses.starts_with("actions/cache/save@"))
299+ && step.raw["with"]["lookup-only"] != json!(true)
300+}
301+
302+fn paths(step: &workflow::Step) -> Vec<String> {
303+ let text = step.raw["with"]["path"].as_str().unwrap_or_default();
304+ text.lines()
305+ .map(str::trim)
306+ .filter(|l| !l.is_empty())
307+ .map(str::to_owned)
308+ .collect()
309+}
310+
311+#[test]
312+fn each_cache_has_one_set_of_paths_wherever_it_is_used() {
313+ // An entry's version is the hash of its paths: a step that names the
314+ // same key with other paths finds nothing, silently. So every step of
315+ // an entry, in every workflow, names the same paths.
316+ let mut seen: std::collections::BTreeMap<String, (Vec<String>, String)> = Default::default();
317+ for (file, job, step) in cache_steps() {
318+ let key = step.raw["with"]["key"].as_str().unwrap_or_default();
319+ let family = family(key);
320+ assert!(!family.is_empty(), "{file} {}: a key starts with its name", job.id);
321+ let here = format!("{file} {}", job.id);
322+ match seen.get(&family) {
323+ Some((first, at)) => assert_eq!(
324+ &paths(&step),
325+ first,
326+ "{here} names other paths for {family} than {at}"
327+ ),
328+ None => {
329+ seen.insert(family, (paths(&step), here));
330+ }
331+ }
332+ }
333+ for family in [
334+ "npm-full-",
335+ "cargo-crates-",
336+ "cargo-test-",
337+ "cargo-workers-",
338+ "runner-musl-",
339+ "runner-release-",
340+ "desktop-check-",
341+ "desktop-release-linux-",
342+ "desktop-release-windows-",
343+ "apt-tauri-",
344+ "apt-tauri-windows-",
345+ "xwin-",
346+ "zig-0.13.0-zigbuild-",
347+ ] {
348+ assert!(seen.contains_key(family), "{family}: {:?}", seen.keys());
349+ }
350+}
351+
352+#[test]
353+fn main_saves_every_cache_that_pull_requests_and_releases_restore() {
354+ // A run restores from its own ref, its pull request's base and main;
355+ // a release tag's run never reads another tag's entries. So every
356+ // entry any step restores is saved by a workflow that runs on pushes
357+ // to main, and the release workflows only restore.
358+ let on_main = ["ci.yml", "caches.yml", "deploy.yml"];
359+ for file in on_main {
360+ let push = read(file).trigger("push").cloned().unwrap();
361+ assert!(push.branches.allows("main"), "{file}");
362+ }
363+ let releases = [
364+ "cli-release.yml",
365+ "desktop-release.yml",
366+ "mobile-build.yml",
367+ "mobile-release.yml",
368+ "runner-release.yml",
369+ ];
370+ let steps = cache_steps();
371+ let saved_on_main: std::collections::BTreeSet<String> = steps
372+ .iter()
373+ .filter(|(file, _, step)| on_main.contains(&file.as_str()) && saves(step))
374+ .map(|(_, _, step)| family(step.raw["with"]["key"].as_str().unwrap_or_default()))
375+ .collect();
376+ for (file, job, step) in &steps {
377+ let family = family(step.raw["with"]["key"].as_str().unwrap_or_default());
378+ // The conformance run restores what its own first job saved.
379+ if family == "conformance-" {
380+ continue;
381+ }
382+ assert!(
383+ saved_on_main.contains(&family),
384+ "{file} {}: nothing on main saves {family}",
385+ job.id
386+ );
387+ if releases.contains(&file.as_str()) {
388+ assert!(!saves(step), "{file} {}: a release saves {family}, which only its own tag would read", job.id);
389+ }
390+ }
391+}
392+
393+#[test]
394+fn rust_caches_are_keyed_by_the_toolchain_and_the_dependencies() {
395+ // Not by Cargo.lock as a whole: 25 of its 31 changes in a week were a
396+ // release's version bump, and each saved every Rust cache again.
397+ for (file, job, step) in cache_steps() {
398+ let key = step.raw["with"]["key"].as_str().unwrap_or_default();
399+ assert!(
400+ !key.contains("Cargo.lock") && !key.contains("base.json"),
401+ "{file} {}: {key}",
402+ job.id
403+ );
404+ let target = paths(&step).iter().any(|p| p.contains("target"));
405+ if target {
406+ assert!(key.contains("steps.keys.outputs.rust"), "{file} {}: {key}", job.id);
407+ // A restore key never reaches another toolchain's entries.
408+ for prefix in step.raw["with"]["restore-keys"]
409+ .as_str()
410+ .unwrap_or_default()
411+ .lines()
412+ .filter(|l| !l.trim().is_empty())
413+ {
414+ assert!(prefix.contains("steps.keys.outputs.rust"), "{file} {}: {prefix}", job.id);
415+ }
416+ }
417+ if key.contains("steps.keys.outputs.") {
418+ let at = job.steps.iter().position(|s| s == &step).unwrap();
419+ let keys = job
420+ .steps
421+ .iter()
422+ .position(|s| s.id.as_deref() == Some("keys"))
423+ .unwrap_or_else(|| panic!("{file} {}: no keys step", job.id));
424+ assert!(keys < at, "{file} {}", job.id);
425+ assert_eq!(
426+ job.steps[keys].run.as_deref(),
427+ Some("node scripts/deploy/ci-cache.mjs keys")
428+ );
429+ }
430+ }
431+}
432+
433+#[test]
434+fn a_cargo_target_restored_from_an_older_entry_is_pruned_before_it_is_saved() {
435+ // Otherwise the old versions of what changed ride along forever: CI's
436+ // test build grew from 671 MB to 1,687 MB in a day.
437+ for (file, id, cache) in [
438+ ("ci.yml", "rust", "test-build"),
439+ ("ci.yml", "desktop", "desktop-build"),
440+ ("caches.yml", "runner", "runner-build"),
441+ ] {
442+ let workflow = read(file);
443+ let job = workflow.jobs.iter().find(|j| j.id == id).unwrap();
444+ let at = job
445+ .steps
446+ .iter()
447+ .position(|s| s.id.as_deref() == Some(cache))
448+ .unwrap();
449+ assert!(saves(&job.steps[at]), "{file} {id}");
450+ let prune = job
451+ .steps
452+ .iter()
453+ .position(|s| {
454+ s.run
455+ .as_deref()
456+ .is_some_and(|r| r.starts_with("node scripts/deploy/ci-cache.mjs prune "))
457+ })
458+ .unwrap_or_else(|| panic!("{file} {id}: no prune"));
459+ assert!(at < prune, "{file} {id}");
460+ assert_eq!(
461+ job.steps[prune].condition.as_deref().map(str::trim),
462+ Some(format!("${{{{ steps.{cache}.outputs.cache-hit != 'true' }}}}").as_str()),
463+ "{file} {id}"
464+ );
465+ }
466+ let ci = read("ci.yml");
467+ let rust = ci.jobs.iter().find(|j| j.id == "rust").unwrap();
468+ assert_eq!(
469+ rust.raw["env"]["CARGO_PROFILE_DEV_DEBUG"],
470+ json!("line-tables-only")
281471 );
282472 }
283473
284474 #[test]
475+fn the_caches_workflow_builds_only_what_is_missing() {
476+ let caches = read("caches.yml");
477+ let push = caches.trigger("push").unwrap();
478+ assert!(push.branches.allows("main"));
479+ assert!(!push.branches.allows("feature"));
480+ // Never cut off halfway: a build half done saves nothing.
481+ assert_eq!(
482+ caches.concurrency.as_ref().unwrap().cancel_in_progress,
483+ json!(false)
484+ );
485+ let look = caches.jobs.iter().find(|j| j.id == "look").unwrap();
486+ // Looked up, not downloaded.
487+ for step in &look.steps {
488+ if step.uses.as_deref().is_some_and(|u| u.starts_with("actions/cache")) {
489+ assert_eq!(step.uses.as_deref(), Some("actions/cache/restore@v4"));
490+ assert_eq!(step.raw["with"]["lookup-only"], json!(true), "{:?}", step.id);
491+ }
492+ }
493+ for job in &caches.jobs {
494+ if job.id != "look" {
495+ assert_eq!(job.needs, ["look"], "{}", job.id);
496+ assert!(
497+ job.condition
498+ .as_deref()
499+ .is_some_and(|c| c.contains("needs.look.outputs.")),
500+ "{}",
501+ job.id
502+ );
503+ }
504+ }
505+}
506+
507+#[test]
285508 fn deploy_stages_follow_one_another() {
286509 let deploy = read("deploy.yml");
287510 let all = plan_outputs(
+57−0
1+#!/usr/bin/env bash
2+# Installs a named set of Debian packages, from the job's cache when it has
3+# them. CI's desktop job spent 110 s of its 175 s here on every run: 21 s
4+# on `apt-get update`, 35 s downloading 335 MB, the rest unpacking.
5+#
6+# bash scripts/apt-install.sh tauri # Tauri's build on Linux
7+# bash scripts/apt-install.sh tauri-windows # its Windows cross build
8+#
9+# apt's package lists and downloaded .debs are kept in ~/.cache/apt/<set>,
10+# which the workflow restores and saves with actions/cache under
11+# apt-<os>-<set>-<hash of this file>, so changing a set here starts a new
12+# entry. With them restored, the install runs offline (--no-download); when
13+# that fails (a list out of date for a package the image already has newer,
14+# say) or nothing was restored, it updates the lists and downloads as
15+# before, tried three times, since the mirror has answered with a
16+# mismatched package before. An entry lives at most 28 days, so the lists
17+# are never older than that.
18+set -euo pipefail
19+
20+set_name="${1:-}"
21+case "$set_name" in
22+ tauri)
23+ packages=(libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libayatana-appindicator3-dev librsvg2-dev)
24+ ;;
25+ tauri-windows)
26+ packages=(build-essential curl wget file libssl-dev libayatana-appindicator3-dev nsis clang lld llvm)
27+ ;;
28+ *)
29+ echo "usage: bash scripts/apt-install.sh tauri|tauri-windows" >&2
30+ exit 2
31+ ;;
32+esac
33+
34+dir="$HOME/.cache/apt/$set_name"
35+mkdir -p "$dir/archives/partial" "$dir/lists/partial"
36+apt=(sudo apt-get -o "Dir::Cache::Archives=$dir/archives" -o "Dir::State::Lists=$dir/lists" -y)
37+
38+# The cache's files go back into it as this user, and apt leaves its
39+# partial folders readable by root alone.
40+mine() { sudo chown -R "$(id -u):$(id -g)" "$dir"; }
41+trap mine EXIT
42+
43+if compgen -G "$dir/lists/*_Packages*" >/dev/null && "${apt[@]}" install --no-download "${packages[@]}"; then
44+ echo "Installed ${#packages[@]} packages from the cache."
45+ exit 0
46+fi
47+
48+for try in 1 2 3; do
49+ if "${apt[@]}" update && "${apt[@]}" install "${packages[@]}"; then
50+ # Only what this install used is kept.
51+ "${apt[@]}" autoclean >/dev/null 2>&1 || true
52+ exit 0
53+ fi
54+ sudo rm -rf "$dir/archives/partial/"*
55+ sleep 15
56+done
57+exit 1
+161−0
1+// The keys of the Rust caches in g1t's own workflows, and the pruning of a
2+// Cargo target before it is saved. CONTRIBUTING.md ("How g1t's own CI
3+// caches") has the whole scheme.
4+//
5+// node scripts/deploy/ci-cache.mjs keys
6+// node scripts/deploy/ci-cache.mjs prune <profile dir>... -- <cargo args>
7+//
8+// keys writes to the step's outputs (GITHUB_OUTPUT):
9+//
10+// rust rustc's version, such as 1.99.0. Everything Cargo compiles
11+// depends on it, so a new toolchain starts a new line of entries
12+// rather than restoring the old one's, whose files it would
13+// only carry along unused.
14+// cargo a hash of the packages Cargo.lock takes from crates.io and git,
15+// leaving out the workspace's own: a release bumps a crate's
16+// version, which changes Cargo.lock and nothing that is cached.
17+// Keyed by the whole file, 25 of the 31 changes to Cargo.lock in
18+// the week before this were version bumps, and each saved every
19+// Rust cache again (the test build's 0.7 to 1.7 GB each time).
20+// desktop the same for apps/desktop/src-tauri/Cargo.lock.
21+//
22+// prune runs Cargo again with --message-format=json once the build is done
23+// (everything is fresh, so it only lists what it built) and deletes, from
24+// each profile dir's deps/, build/ and .fingerprint/, the files and folders
25+// of units this build did not use. A target restored from an older entry
26+// (a restore key) keeps the old versions of whatever changed beside the new
27+// ones, and saving it carried them on forever: CI's test build grew from
28+// 671 MB to 1,687 MB in a day, two thirds of the way to the 2 GB an entry
29+// may hold, and its restore from 15 s to a minute. Anything unexpected
30+// (Cargo fails, lists nothing) prunes nothing.
31+
32+import { spawnSync } from "node:child_process";
33+import { createHash } from "node:crypto";
34+import { appendFileSync, existsSync, readFileSync, readdirSync, rmSync, statSync } from "node:fs";
35+import { basename, join } from "node:path";
36+import { fileURLToPath } from "node:url";
37+
38+const ROOT = fileURLToPath(new URL("../..", import.meta.url));
39+
40+/** The `[[package]]` blocks of a Cargo.lock that come from a registry or git: not the workspace's own. */
41+export function externalPackages(lock) {
42+ return lock
43+ .split(/^\[\[package\]\]\s*$/m)
44+ .slice(1)
45+ .map((block) => block.trim())
46+ .filter((block) => /^source = /m.test(block))
47+ .sort();
48+}
49+
50+/** 16 hex characters of the SHA-256 of a Cargo.lock's external packages. */
51+export function depsHash(lock) {
52+ return createHash("sha256").update(externalPackages(lock).join("\n\n")).digest("hex").slice(0, 16);
53+}
54+
55+/** `1.99.0` from `rustc 1.99.0 (90b35a623 2026-09-30)`; null if it is not that. */
56+export function rustVersion(text) {
57+ return text.match(/^rustc (\S+)/)?.[1] ?? null;
58+}
59+
60+function keys() {
61+ const rustc = spawnSync("rustc", ["-V"], { encoding: "utf8" });
62+ const rust = rustVersion(rustc.stdout ?? "");
63+ if (!rust) throw new Error(`rustc -V said ${JSON.stringify(rustc.stdout ?? rustc.error?.message)}`);
64+ const outputs = { rust };
65+ for (const [name, path] of [
66+ ["cargo", "Cargo.lock"],
67+ ["desktop", "apps/desktop/src-tauri/Cargo.lock"],
68+ ]) {
69+ const file = join(ROOT, path);
70+ if (existsSync(file)) outputs[name] = depsHash(readFileSync(file, "utf8"));
71+ }
72+ const lines = Object.entries(outputs).map(([k, v]) => `${k}=${v}`);
73+ console.log(lines.join("\n"));
74+ if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `${lines.join("\n")}\n`);
75+}
76+
77+/** The 16-hex-character hash Cargo puts on a unit's files and folders (`libserde-0123456789abcdef.rlib`), or null. */
78+export function unitHash(name) {
79+ return name.match(/-([0-9a-f]{16})(?:\.[^/]*)?$/)?.[1] ?? null;
80+}
81+
82+/** The hashes of every unit in Cargo's JSON messages: its files, and a build script's out dir. */
83+export function usedHashes(messages) {
84+ const used = new Set();
85+ for (const line of messages.split("\n")) {
86+ if (!line.startsWith("{")) continue;
87+ let m;
88+ try {
89+ m = JSON.parse(line);
90+ } catch {
91+ continue;
92+ }
93+ const paths = [...(m.filenames ?? []), ...(m.executable ? [m.executable] : [])];
94+ if (m.reason === "build-script-executed" && m.out_dir) paths.push(m.out_dir.replace(/[\\/]out$/, ""));
95+ for (const path of paths) {
96+ // A unit's own folder (build/foo-<hash>/build-script-build) or file (deps/libfoo-<hash>.rlib).
97+ for (const part of path.split(/[\\/]/).slice(-2)) {
98+ const hash = unitHash(part);
99+ if (hash) used.add(hash);
100+ }
101+ }
102+ }
103+ return used;
104+}
105+
106+/** What to delete from a profile dir: deps/ files and build/, .fingerprint/ folders of units not in `used`. */
107+export function stale(dir, used) {
108+ const out = [];
109+ for (const sub of ["deps", "build", ".fingerprint"]) {
110+ const at = join(dir, sub);
111+ if (!existsSync(at)) continue;
112+ for (const name of readdirSync(at)) {
113+ const hash = unitHash(name);
114+ if (hash && !used.has(hash)) out.push(join(at, name));
115+ }
116+ }
117+ return out;
118+}
119+
120+function bytes(path) {
121+ const s = statSync(path, { throwIfNoEntry: false });
122+ if (!s) return 0;
123+ if (!s.isDirectory()) return s.size;
124+ return readdirSync(path).reduce((sum, name) => sum + bytes(join(path, name)), 0);
125+}
126+
127+function prune(args) {
128+ const split = args.indexOf("--");
129+ const dirs = split < 0 ? [] : args.slice(0, split);
130+ const cargo = split < 0 ? [] : args.slice(split + 1);
131+ if (!dirs.length || !cargo.length) throw new Error("usage: ci-cache.mjs prune <profile dir>... -- <cargo args>");
132+ const listed = spawnSync("cargo", [...cargo, "--message-format=json"], {
133+ cwd: ROOT,
134+ encoding: "utf8",
135+ maxBuffer: 1 << 30,
136+ stdio: ["ignore", "pipe", "inherit"],
137+ });
138+ const used = listed.status === 0 ? usedHashes(listed.stdout) : new Set();
139+ if (!used.size) {
140+ console.log(`::warning title=Cargo cache::cargo ${cargo.join(" ")} did not list what it built, so nothing is pruned`);
141+ return;
142+ }
143+ for (const dir of dirs) {
144+ const before = bytes(join(ROOT, dir));
145+ const gone = stale(join(ROOT, dir), used);
146+ for (const path of gone) rmSync(path, { recursive: true, force: true });
147+ const after = bytes(join(ROOT, dir));
148+ const mb = (n) => `${Math.round(n / 1_048_576)} MB`;
149+ console.log(`${dir}: ${gone.length} files and folders of units this build did not use, ${mb(before - after)}; ${mb(after)} left`);
150+ }
151+}
152+
153+if (process.argv[1] && basename(process.argv[1]) === "ci-cache.mjs") {
154+ const [command, ...rest] = process.argv.slice(2);
155+ if (command === "keys") keys();
156+ else if (command === "prune") prune(rest);
157+ else {
158+ console.error("usage: node scripts/deploy/ci-cache.mjs keys | prune <profile dir>... -- <cargo args>");
159+ process.exit(2);
160+ }
161+}
+74−0
1+import { test } from "node:test";
2+import assert from "node:assert/strict";
3+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
4+import { tmpdir } from "node:os";
5+import { join } from "node:path";
6+import { depsHash, externalPackages, rustVersion, stale, unitHash, usedHashes } from "./ci-cache.mjs";
7+
8+const LOCK = (runner, serde) => `# This file is automatically @generated by Cargo.
9+version = 4
10+
11+[[package]]
12+name = "g1t-runner"
13+version = "${runner}"
14+dependencies = [
15+ "serde",
16+]
17+
18+[[package]]
19+name = "serde"
20+version = "${serde}"
21+source = "registry+https://github.com/rust-lang/crates.io-index"
22+checksum = "abc"
23+`;
24+
25+test("a release's version bump keeps the Rust caches' key, a new dependency changes it", () => {
26+ assert.deepEqual(externalPackages(LOCK("0.3.3", "1.0.200")).length, 1);
27+ assert.equal(depsHash(LOCK("0.3.3", "1.0.200")), depsHash(LOCK("0.3.4", "1.0.200")));
28+ assert.notEqual(depsHash(LOCK("0.3.3", "1.0.200")), depsHash(LOCK("0.3.3", "1.0.201")));
29+ assert.match(depsHash(LOCK("0.3.3", "1.0.200")), /^[0-9a-f]{16}$/);
30+});
31+
32+test("the toolchain is read from rustc -V", () => {
33+ assert.equal(rustVersion("rustc 1.99.0 (90b35a623 2026-09-30)\n"), "1.99.0");
34+ assert.equal(rustVersion("rustc 1.100.0-nightly (abc 2026-10-01)"), "1.100.0-nightly");
35+ assert.equal(rustVersion(""), null);
36+});
37+
38+test("a unit's hash is read from its files and folders", () => {
39+ assert.equal(unitHash("libserde-0123456789abcdef.rlib"), "0123456789abcdef");
40+ assert.equal(unitHash("serde-0123456789abcdef"), "0123456789abcdef");
41+ assert.equal(unitHash("g1t_actions-0123456789abcdef.d"), "0123456789abcdef");
42+ assert.equal(unitHash("repository_workflows-0123456789abcdef"), "0123456789abcdef");
43+ assert.equal(unitHash("g1t"), null);
44+ assert.equal(unitHash("libg1t.rlib"), null);
45+});
46+
47+test("only the units the build did not use are pruned", () => {
48+ const dir = mkdtempSync(join(tmpdir(), "ci-cache-"));
49+ try {
50+ const [used, old, script, run] = ["1111111111111111", "2222222222222222", "3333333333333333", "4444444444444444"];
51+ for (const sub of ["deps", "build", ".fingerprint"]) mkdirSync(join(dir, sub), { recursive: true });
52+ for (const f of [`libserde-${used}.rlib`, `libserde-${used}.rmeta`, `serde-${used}.d`, `libserde-${old}.rlib`, `serde-${old}.d`, `tests-${used}`]) {
53+ writeFileSync(join(dir, "deps", f), "x");
54+ }
55+ for (const d of [`serde-${script}`, `serde-${run}`, `serde-${old}`]) mkdirSync(join(dir, "build", d));
56+ for (const d of [`serde-${used}`, `serde-${script}`, `serde-${run}`, `serde-${old}`]) mkdirSync(join(dir, ".fingerprint", d));
57+ const messages = [
58+ " Compiling serde",
59+ JSON.stringify({ reason: "compiler-artifact", filenames: [join(dir, "build", `serde-${script}`, "build-script-build")], executable: null }),
60+ JSON.stringify({ reason: "build-script-executed", out_dir: join(dir, "build", `serde-${run}`, "out") }),
61+ JSON.stringify({ reason: "compiler-artifact", filenames: [join(dir, "deps", `libserde-${used}.rlib`), join(dir, "deps", `libserde-${used}.rmeta`)] }),
62+ JSON.stringify({ reason: "compiler-artifact", filenames: [], executable: join(dir, "deps", `tests-${used}`) }),
63+ JSON.stringify({ reason: "build-finished", success: true }),
64+ ].join("\n");
65+ const hashes = usedHashes(messages);
66+ assert.deepEqual([...hashes].sort(), [used, script, run].sort());
67+ const gone = stale(dir, hashes).map((p) => p.slice(dir.length + 1).replaceAll("\\", "/")).sort();
68+ assert.deepEqual(gone, [`.fingerprint/serde-${old}`, `build/serde-${old}`, `deps/libserde-${old}.rlib`, `deps/serde-${old}.d`]);
69+ // Nothing listed: nothing would be kept, so the caller prunes nothing.
70+ assert.equal(usedHashes("error: could not compile").size, 0);
71+ } finally {
72+ rmSync(dir, { recursive: true, force: true });
73+ }
74+});
+32−0
1+#!/usr/bin/env bash
2+# What the desktop app's Windows cross build needs on g1t's Linux machines,
3+# besides cargo-xwin: Debian's packages (scripts/apt-install.sh, from the
4+# job's cache when it has them), the names cargo-xwin runs LLVM's tools by,
5+# and an NSIS header Debian lacks. desktop-release.yml's Windows build and
6+# caches.yml, which keeps its cache warm on main, both run it.
7+set -euo pipefail
8+
9+bash "$(dirname "$0")/apt-install.sh" tauri-windows
10+# cargo-xwin compiles C (ring's, among others) with clang-cl and
11+# links with lld-link and llvm-lib. Each is a mode of a tool
12+# Debian does install, chosen by the name it is run as: clang-cl
13+# is clang, lld-link is lld, and llvm-lib is llvm-ar. Debian ships
14+# none of the three names, so each is linked to its tool.
15+for pair in clang-cl:clang lld-link:lld llvm-lib:llvm-ar; do
16+ name="${pair%%:*}" tool="${pair##*:}"
17+ command -v "$name" >/dev/null && continue
18+ found="$(command -v "$tool" || ls /usr/lib/llvm-*/bin/"$tool" 2>/dev/null | sort -V | tail -1)"
19+ [ -n "$found" ] || { echo "::error::$tool is not installed, so there is no $name"; exit 1; }
20+ sudo ln -sf "$(readlink -f "$found")" "/usr/local/bin/$name"
21+ echo "$name: $(readlink -f "$found")"
22+done
23+# Tauri's installer script includes Win\RestartManager.nsh, which
24+# NSIS ships from 3.10 on; Debian's is 3.08. The header stands
25+# alone (defines and System calls), so it is added from NSIS 3.11's
26+# source, checked against its SHA-256.
27+rm_nsh=/usr/share/nsis/Include/Win/RestartManager.nsh
28+if [ ! -f "$rm_nsh" ]; then
29+ curl -fsSL --retry 3 -o /tmp/RestartManager.nsh https://raw.githubusercontent.com/kichik/nsis/v311/Include/Win/RestartManager.nsh
30+ echo "67149978412da9b283e2e1abe48a360093df9caf6363373e2cabe463978b7c0e /tmp/RestartManager.nsh" | sha256sum -c -
31+ sudo install -m 0644 /tmp/RestartManager.nsh "$rm_nsh"
32+fi