Skip to content
Sign up

flagon-io/g1t

public
0

Commit

A deploy makes the runner's image in the registry, without Docker: it reads the base's manifest and config from Cloudflare's registry, uploads one gzipped layer holding the runner binary at /usr/local/bin/g1t-runner with mode 0755, and writes a config (the entrypoint g1t-runner, the base's command cleared, as the Dockerfile's ENTRYPOINT does) and a manifest that name the base's layers and that one, under the runner's tag, for linux/amd64 (scripts/deploy/registry.mjs, the registry's own upload calls with the credentials Wrangler gives). Deploy #256 ran out of disk because docker build pulled the 4.5 GB base into the job's own Docker Engine, compressed and unpacked, on a 20 GB sandbox whose own filesystem is that same base; now nothing of the base is downloaded and the job's Docker Engine is never started. Docker is asked for only when it is needed: a dry run (build, or image --no-push) still builds services/runner/Dockerfile with it, and so does a deploy whose registry step fails, logging df -h and docker system df before and after. The image job removes npm's downloads, which it only restores, once its install has read them, and logs the disk before and after the deploy. Tests cover the layer's tar, the config and manifest, the calls against a registry kept in memory (the base's layers are never fetched), the token flow, and that the Dockerfile says the same path, mode and entrypoint; the image was also checked against a local registry, where it runs and has the same config as a docker build of the Dockerfile.

syntaqxcommitted Parent576db88Browse files
7 files+727−310/7 viewed
+26−5
2222 # Containers write), the variable CLOUDFLARE_ACCOUNT_ID, and
2323 # api.cloudflare.com among the project's workflow-only domains for
2424 # deploy.yml in production (Settings, Guardrails), and
25−# registry.cloudflare.com there too, to find, pull and push the runner's
26−# image. A job that must build that image (the `runner-image` group) does
27−# so with its own Docker Engine, on a larger machine. Optionally, the
25+# registry.cloudflare.com there too, to find the runner's image and make
26+# it. A job that must make that image (the `runner-image` group) builds the
27+# runner's binary on a larger machine and makes the image in the registry
28+# with no Docker: one layer with the binary on the base's layers, pulling
29+# none of the base (scripts/deploy/registry.mjs). Optionally, the
2830 # secret STATUS_DEPLOY_TOKEN (the status Worker's secret of the same name)
2931 # and status.g1t.sh among the same workflow-only domains, so status.g1t.sh
3032 # hears each deploy start and finish.
260262 cargo-target-${{ runner.os }}-
261263 # The runner's image: its binary, built natively for musl (the base
262264 # has musl-gcc; the target is added here), with its Cargo target kept
263− # between runs. The image itself is built and pushed with the job's
264− # own Docker Engine (scripts/deploy/image.mjs).
265+ # between runs. The image itself is made in the registry, without
266+ # Docker: the base's layers and one with the binary
267+ # (scripts/deploy/registry.mjs). Building it with the job's Docker
268+ # Engine, only if that fails, pulls the 4.5 GB base onto the job's
269+ # disk.
265270 - name: Rust for the runner
266271 if: ${{ matrix.image }}
267272 run: rustup target add x86_64-unknown-linux-musl
299304 restore-keys: npm-full-${{ runner.os }}-
300305 - name: Install
301306 run: node scripts/deploy.mjs install --only "${{ matrix.units }}"
307+ # The runner's image job: npm's downloads are only restored here, and
308+ # the install has read them, so they go, leaving the disk to the
309+ # runner's build (and to Docker, should the image need it). What the
310+ # disk holds is logged before and after, so a job that runs out of
311+ # room says what took it.
312+ - name: Room for the runner's image
313+ if: ${{ matrix.image }}
314+ run: |
315+ rm -rf ~/.npm
316+ df -h / "$HOME" /tmp
317+ du -sh "$HOME"/.cargo target node_modules 2>/dev/null || true
302318 - name: Deploy ${{ matrix.units }}
303319 env:
304320 CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
306322 # are not drafted as incidents. Optional: without it, nothing is sent.
307323 STATUS_DEPLOY_TOKEN: ${{ secrets.STATUS_DEPLOY_TOKEN }}
308324 run: node scripts/deploy.mjs deploy --only "${{ matrix.units }}" --force --no-migrations --concurrency 2
325+ - name: The disk after the runner's image
326+ if: ${{ always() && matrix.image }}
327+ run: |
328+ df -h / "$HOME" /tmp
329+ du -sh "$HOME"/.cargo target node_modules 2>/dev/null || true
309330 # Hits and misses, on the log and in the run's summary.
310331 - name: sccache's hits and misses
311332 if: ${{ always() && (matrix.rust || matrix.image) }}
+10−2
8080 node scripts/deploy.mjs manifest --check # the manifest's problems
8181 node scripts/deploy.mjs doctor # secrets each unit lacks
8282 node scripts/deploy.mjs build-base # build and push the runner's base image (Docker)
83−node scripts/deploy.mjs image # build and push the runner's image (Docker)
83+node scripts/deploy.mjs image # make the runner's image in the registry (no Docker)
8484 ```
8585
8686 | Flag | What it does |
288288 `services/runner/base.json` records the base that was pushed;
289289 `npm run test:deploy` fails while it and the base's folder disagree. The
290290 runner's image is tagged with a hash of what it is built from, so a deploy
291−builds it only when the registry does not already have it. When nothing
291+makes it only when the registry does not already have it. It is made in
292+the registry, without Docker: the deploy reads the base's manifest,
293+uploads one layer holding the binary, and writes a manifest that names the
294+base's layers and that one, with the entrypoint `services/runner/Dockerfile`
295+gives it. Nothing of the base is downloaded, so the deploy job needs no
296+room for it. If that fails and Docker runs where the deploy does, the
297+image is built from `services/runner/Dockerfile` and pushed instead, and
298+the log shows how full the disk was before and after. A dry run
299+(`build`, or `image --no-push`) builds it with Docker. When nothing
292300 the image is built from changed, the deploy leaves running sandboxes
293301 alone. When it did, Cloudflare replaces sandboxes on the new image but
294302 leaves any that started within the last two hours
+45−16
1212 // node scripts/deploy.mjs doctor which units lack their secrets
1313 // node scripts/deploy.mjs install --only a,b npm ci of just what those units need (CI)
1414 // node scripts/deploy.mjs build-base build and push the runner's base image (Docker)
15−// node scripts/deploy.mjs image build and push the runner's image for this checkout (Docker)
15+// node scripts/deploy.mjs image make the runner's image for this checkout in the registry (--no-push: build it with Docker)
1616 //
1717 // Flags: --all (every unit, changed or not), --only a,b, --skip a,b,
1818 // --force, --concurrency N (default 4), --stage core (one stage),
2525 import { tmpdir } from "node:os";
2626 import { join } from "node:path";
2727
28−import { OUT_DIR } from "./build-runner.mjs";
28+import { BINARY, OUT_DIR } from "./build-runner.mjs";
2929 import { ensureWorkerBuild } from "./build-rust-worker.mjs";
3030 import {
3131 annotation,
5151 baseVersions,
5252 buildBase,
5353 buildRunnerImage,
54+ diskReport,
5455 dockerHas,
5556 dockerLogin,
5657 imageSize,
58+ makeRunnerImage,
5759 pushImage,
5860 readBaseLock,
5961 registryHas,
149151
150152 const seconds = (ms) => `${(ms / 1000).toFixed(1)}s`;
151153
154+/** `fn`, called the first time it is wanted and then remembered. */
155+const once = (fn) => {
156+ let answer = null;
157+ return () => (answer ??= fn());
158+};
159+
152160 /** Reads what each unit runs and what its database is waiting for. */
153161 async function survey(units, opts) {
154162 const live = {};
242250 /**
243251 * The runner's image for this checkout, by registry reference: already in
244252 * the registry (built by an earlier deploy, `deploy.mjs image`, or on
245− * another machine), or built and pushed here, which needs Docker. A dry
246− * run builds it locally and pushes nothing. Returns { ref, note }.
253+ * another machine), or made here. Pushing, it is made in the registry
254+ * (scripts/deploy/registry.mjs: the base's layers and one with the
255+ * binary, nothing pulled), which needs no Docker and no room for the
256+ * base; if that fails and Docker answers, it is built and pushed with
257+ * Docker. A dry run builds it with Docker and pushes nothing. `docker` is
258+ * asked only when needed: asking starts a job's Docker Engine. Returns
259+ * { ref, note }.
247260 */
248261 async function runnerImage(unit, { dryRun, docker, rebuildImage, rebuildBase }, out) {
249262 let base = baseState(unit);
253266 `${unit.image.base.context} has changed since ${unit.image.base.lock} was written${base.lock ? "" : " (the base has never been built)"}. Build and push the base, and commit ${unit.image.base.lock}: node scripts/deploy.mjs build-base`,
254267 );
255268 }
256− if (!docker) throw new Error("--rebuild-base needs Docker.");
269+ if (!(await docker())) throw new Error("--rebuild-base needs Docker.");
257270 await newBase(unit, { push: !dryRun, onLine: out.line });
258271 base = baseState(unit);
259272 }
261274 const ref = runnerRef(unit);
262275 const tag = ref.split(":").pop();
263276 if (!rebuildImage) {
264− if (dryRun && docker && (await dockerHas(ref))) return { ref, note: `image ${tag} already built here` };
277+ if (dryRun && (await docker()) && (await dockerHas(ref))) return { ref, note: `image ${tag} already built here` };
265278 if (!dryRun) {
266279 try {
267280 if (await registryHas(ref)) return { ref, note: `image ${tag} already in the registry` };
270283 }
271284 }
272285 }
273− if (!docker) {
274− throw new Error(
275− `its image ${tag} is not in the registry, and Docker does not answer here. Build and push it where Docker runs (a g1t Actions job, or a machine with Docker: node scripts/deploy.mjs image), then run this again.`,
276− );
286+ if (dryRun && !(await docker())) {
287+ throw new Error(`its image ${tag} is not built here, and Docker does not answer here: a dry run builds it with Docker.`);
277288 }
278289 const started = Date.now();
279290 const binary = await exec(process.execPath, [join(ROOT, "scripts/build-runner.mjs")], { onLine: out.line });
280291 if (binary.code !== 0) throw new Error(`the runner binary did not build:\n${lastLines(binary.out)}`);
281− if (!dryRun) await dockerLogin({ push: true });
282− await buildRunnerImage(unit, { ref, base: base.ref, binaryDir: OUT_DIR, onLine: out.line });
283− if (!dryRun) await pushImage(ref, { onLine: out.line });
284− return { ref, note: `image ${tag} ${dryRun ? "built" : "built and pushed"} in ${seconds(Date.now() - started)}` };
292+ if (!dryRun) {
293+ try {
294+ await makeRunnerImage({ ref, base: base.ref, binary: BINARY, onLine: out.line });
295+ return { ref, note: `image ${tag} made in the registry in ${seconds(Date.now() - started)}` };
296+ } catch (error) {
297+ if (!(await docker())) {
298+ throw new Error(`its image ${tag} could not be made in the registry (${error.message ?? error}), and Docker does not answer here to build it instead.`);
299+ }
300+ out.line(`could not make ${tag} in the registry (${error.message ?? error}); building it with Docker instead`);
301+ }
302+ }
303+ // With Docker: the base is pulled and unpacked on this disk, so what the
304+ // disk held is logged before and after, and when it fails.
305+ await diskReport(out.line, "before the build");
306+ try {
307+ if (!dryRun) await dockerLogin({ push: true });
308+ await buildRunnerImage(unit, { ref, base: base.ref, binaryDir: OUT_DIR, onLine: out.line });
309+ if (!dryRun) await pushImage(ref, { onLine: out.line });
310+ } finally {
311+ await diskReport(out.line, "after the build");
312+ }
313+ return { ref, note: `image ${tag} ${dryRun ? "built" : "built and pushed"} with Docker in ${seconds(Date.now() - started)}` };
285314 }
286315
287316 /** Builds the base (and pushes it unless `push` is false), and writes its lock. */
423452 }
424453
425454 if (touched.some((u) => u.kind === "rust-worker")) ensureWorkerBuild();
426− const docker = touched.some((u) => u.image) ? await dockerAvailable() : false;
455+ const docker = once(dockerAvailable);
427456 const context = { head, subject: git.subject(), dirty, dryRun, docker, rebuildImage: opts.rebuildImage, rebuildBase: opts.rebuildBase };
428457 // A deploy run by hand shows on the repository's Deployments page; a
429458 // dirty tree is not a commit anyone can look at, so it is left out.
564593 case "image": {
565594 const unit = imageUnit(stack, opts);
566595 const out = unitLogger(`${unit.id}-image`);
567− const docker = await dockerAvailable();
596+ const docker = once(dockerAvailable);
568597 const image = await runnerImage(unit, { dryRun: Boolean(opts.noPush), docker, rebuildImage: opts.rebuildImage, rebuildBase: opts.rebuildBase }, out);
569598 out.save();
570599 console.log(`${image.ref}\n${image.note}`);
+40−5
99 // base plus the runner binary (scripts/build-runner.mjs). Its tag
1010 // is a hash of everything it is built from, so the same source
1111 // always names the same image, and an image already in the
12−// registry is never built again.
12+// registry is never made again.
1313 //
14−// Both live in one repository of Cloudflare's registry, so pushing the
15−// runner uploads only its one new layer. `wrangler deploy` is given the
16−// runner's registry reference (a generated config), so a deploy never
17−// builds an image itself.
14+// Both live in one repository of Cloudflare's registry. A deploy makes the
15+// runner's image there (registry.mjs): it uploads one layer with the
16+// binary and a manifest naming the base's layers and that one, and pulls
17+// nothing, so it needs neither Docker nor room for the base. Building it
18+// with Docker is for dry runs, and the fallback. `wrangler deploy` is
19+// given the runner's registry reference (a generated config), so a deploy
20+// never builds an image itself.
1821
1922 import { createHash } from "node:crypto";
2023 import { execFileSync } from "node:child_process";
2326
2427 import { ACCOUNT_ID, exec, jsonFrom, lastLines, wrangler } from "./cloudflare.mjs";
2528 import { parseCargoLock } from "./lockfiles.mjs";
29+import { appendRunnerImage } from "./registry.mjs";
2630 import { ROOT, parseJsonc } from "./stack.mjs";
2731
2832 export const REGISTRY = "registry.cloudflare.com";
259263 export const PLAIN_IMAGE = ["--platform", "linux/amd64", "--provenance=false", "--sbom=false"];
260264
261265 /**
266+ * Makes the runner's image in the registry, without Docker: the base's
267+ * layers, named by its manifest, and one layer with the binary at `binary`
268+ * (scripts/deploy/registry.mjs). What deploys use; the Docker build below
269+ * is for a dry run, and the fallback. Returns its digest.
270+ */
271+export async function makeRunnerImage({ ref, base, binary, onLine = () => {}, credentials = () => registryCredentials({ push: true }) }) {
272+ return appendRunnerImage({ ref, base, binary: readFileSync(binary), credentials, onLine });
273+}
274+
275+/**
276+ * Logs how full the disk is, and what Docker keeps on it: before and after
277+ * a Docker build, so a build that runs out of room says what took it.
278+ * Never fails; on a machine without `df`, it logs what it can.
279+ */
280+export async function diskReport(onLine, when, { run = exec } = {}) {
281+ if (process.platform === "win32") return;
282+ onLine(`disk ${when}:`);
283+ for (const [command, args] of [
284+ ["df", ["-h", "/", "/tmp", process.env.HOME ?? "/root"]],
285+ ["docker", ["system", "df"]],
286+ ]) {
287+ try {
288+ const done = await run(command, args);
289+ for (const line of done.out.trim().split(/\r?\n/)) onLine(` ${line}`);
290+ } catch (error) {
291+ onLine(` ${command}: ${error.message ?? error}`);
292+ }
293+ }
294+}
295+
296+/**
262297 * Builds the runner's image: the base and the binary, from a build
263298 * context holding only the binary (target/runner-image).
264299 */
+290−0
1+// The runner's image made in the registry, with no Docker: the base's
2+// manifest and config are read from Cloudflare's registry, one layer
3+// holding the runner binary is uploaded beside the base's layers, and a
4+// new config and manifest that name the base's layers and that one are
5+// written under the runner's tag. Nothing of the base is downloaded, so a
6+// deploy job needs no room for the 4.5 GB base, which a `docker build`
7+// pulls and unpacks on the same disk the job runs on.
8+//
9+// What it makes is what services/runner/Dockerfile builds on the same
10+// base: the binary at /usr/local/bin/g1t-runner with mode 0755 (COPY
11+// --chmod=0755), ENTRYPOINT ["g1t-runner"] with the base's CMD cleared
12+// (as a Dockerfile's ENTRYPOINT clears an inherited one), for linux/amd64.
13+// A test holds the Dockerfile to these.
14+//
15+// The calls are the OCI distribution API (docs.docker.com/registry/spec/api/),
16+// as `docker push` makes them: HEAD a blob to skip one already there; POST
17+// to start an upload, PATCH its bytes, PUT with its digest to finish; PUT
18+// the manifest by tag.
19+
20+import { createHash } from "node:crypto";
21+import { gzipSync } from "node:zlib";
22+
23+export const RUNNER_BINARY_PATH = "usr/local/bin/g1t-runner";
24+export const RUNNER_BINARY_MODE = 0o755;
25+export const RUNNER_ENTRYPOINT = ["g1t-runner"];
26+
27+export const OCI_MANIFEST = "application/vnd.oci.image.manifest.v1+json";
28+export const DOCKER_MANIFEST = "application/vnd.docker.distribution.manifest.v2+json";
29+const OCI_INDEX = "application/vnd.oci.image.index.v1+json";
30+const DOCKER_LIST = "application/vnd.docker.distribution.manifest.list.v2+json";
31+const ACCEPT = [OCI_MANIFEST, DOCKER_MANIFEST, OCI_INDEX, DOCKER_LIST].join(", ");
32+
33+/** Each manifest type's gzipped layer type. */
34+const LAYER_TYPE = {
35+ [OCI_MANIFEST]: "application/vnd.oci.image.layer.v1.tar+gzip",
36+ [DOCKER_MANIFEST]: "application/vnd.docker.image.rootfs.diff.tar.gzip",
37+};
38+
39+export const sha256 = (bytes) => `sha256:${createHash("sha256").update(bytes).digest("hex")}`;
40+
41+/** A tar header field: octal, zero-padded, ending in NUL. */
42+function octal(value, width) {
43+ return `${value.toString(8).padStart(width - 1, "0")}\0`;
44+}
45+
46+/**
47+ * A tar archive (ustar) of one regular file, owned by root. Only the file:
48+ * its folders are the base's, so a layer that names them would replace
49+ * their owner, mode and time with its own.
50+ */
51+export function tarOf(path, data, { mode = 0o644, mtime = 0 } = {}) {
52+ if (Buffer.byteLength(path) > 100) throw new Error(`a path this layer cannot hold: ${path}`);
53+ const header = Buffer.alloc(512);
54+ header.write(path, 0, 100, "utf8");
55+ header.write(octal(mode, 8), 100, 8, "ascii");
56+ header.write(octal(0, 8), 108, 8, "ascii"); // uid
57+ header.write(octal(0, 8), 116, 8, "ascii"); // gid
58+ header.write(octal(data.length, 12), 124, 12, "ascii");
59+ header.write(octal(Math.floor(mtime), 12), 136, 12, "ascii");
60+ header.write(" ", 148, 8, "ascii"); // the checksum, as spaces while it is summed
61+ header.write("0", 156, 1, "ascii"); // a regular file
62+ header.write("ustar\0", 257, 6, "ascii");
63+ header.write("00", 263, 2, "ascii");
64+ header.write("root", 265, 32, "ascii");
65+ header.write("root", 297, 32, "ascii");
66+ let sum = 0;
67+ for (const byte of header) sum += byte;
68+ header.write(`${sum.toString(8).padStart(6, "0")}\0 `, 148, 8, "ascii");
69+ const padding = Buffer.alloc((512 - (data.length % 512)) % 512);
70+ return Buffer.concat([header, data, padding, Buffer.alloc(1024)]);
71+}
72+
73+/**
74+ * The runner's layer: the binary at its path and mode, gzipped. Returns
75+ * the compressed bytes with their digest and size, and the uncompressed
76+ * tar's digest (the config's diff_id).
77+ */
78+export function runnerLayer(binary, { mtime = 0 } = {}) {
79+ const tar = tarOf(RUNNER_BINARY_PATH, binary, { mode: RUNNER_BINARY_MODE, mtime });
80+ const bytes = gzipSync(tar, { level: 9 });
81+ return { bytes, digest: sha256(bytes), size: bytes.length, diffId: sha256(tar) };
82+}
83+
84+/**
85+ * The runner image's config from the base's: one more layer, the
86+ * entrypoint, and the history a Dockerfile build would add.
87+ */
88+export function runnerConfig(base, { diffId, created }) {
89+ if (base.os !== "linux" || base.architecture !== "amd64") {
90+ throw new Error(`the base is ${base.os}/${base.architecture}, not linux/amd64`);
91+ }
92+ const config = structuredClone(base);
93+ config.created = created;
94+ config.config = { ...(config.config ?? {}), Entrypoint: [...RUNNER_ENTRYPOINT] };
95+ // An ENTRYPOINT clears the CMD a Dockerfile inherited.
96+ delete config.config.Cmd;
97+ config.rootfs = { type: "layers", diff_ids: [...(base.rootfs?.diff_ids ?? []), diffId] };
98+ config.history = [
99+ ...(base.history ?? []),
100+ { created, created_by: `COPY --chmod=0755 g1t-runner /${RUNNER_BINARY_PATH} # g1t deploy`, comment: "scripts/deploy/registry.mjs" },
101+ { created, created_by: `ENTRYPOINT ${JSON.stringify(RUNNER_ENTRYPOINT)}`, comment: "scripts/deploy/registry.mjs", empty_layer: true },
102+ ];
103+ return config;
104+}
105+
106+/** The runner image's manifest: the base's layers, then the runner's. */
107+export function runnerManifest(base, { config, layer }) {
108+ const type = manifestType(base);
109+ if (!LAYER_TYPE[type]) throw new Error(`a manifest of a type this cannot extend: ${type}`);
110+ const manifest = {
111+ schemaVersion: 2,
112+ mediaType: type,
113+ config: { mediaType: base.config.mediaType, digest: config.digest, size: config.size },
114+ layers: [...base.layers, { mediaType: LAYER_TYPE[type], digest: layer.digest, size: layer.size }],
115+ };
116+ return manifest;
117+}
118+
119+/** A manifest's type: its own, or (OCI lets it go unsaid) its config's. */
120+export function manifestType(manifest) {
121+ return manifest.mediaType ?? (manifest.config?.mediaType === "application/vnd.oci.image.config.v1+json" ? OCI_MANIFEST : DOCKER_MANIFEST);
122+}
123+
124+/** A `WWW-Authenticate: Bearer realm=...,service=...,scope=...` challenge, or null. */
125+export function bearerChallenge(header) {
126+ if (!header || !/^bearer\s/i.test(header)) return null;
127+ const params = Object.fromEntries([...header.slice(7).matchAll(/(\w+)="([^"]*)"/g)].map((m) => [m[1].toLowerCase(), m[2]]));
128+ return params.realm ? params : null;
129+}
130+
131+/** `host/name:tag`, `host/name@digest` or `host/name:tag@digest`, split. */
132+export function parseRef(ref) {
133+ const match = /^([^/]+)\/(.+?)(?::([^:/@]+))?(?:@(sha256:[0-9a-f]{64}))?$/.exec(ref);
134+ if (!match || (!match[3] && !match[4])) throw new Error(`not an image reference with a tag or digest: ${ref}`);
135+ const [, host, name, tag, digest] = match;
136+ return { host, name, tag: tag ?? null, digest: digest ?? null };
137+}
138+
139+/**
140+ * A client for one repository: requests carry Basic auth from
141+ * `credentials` (none when it returns null), or the bearer token a
142+ * registry that asks for one gives for them; paths are under /v2/<name>/.
143+ */
144+export function repositoryClient(host, name, { credentials, fetchImpl = fetch, scheme = "https" }) {
145+ const root = `${scheme}://${host}`;
146+ let bearer = null;
147+ const basic = async () => {
148+ const auth = await credentials();
149+ return auth ? `Basic ${Buffer.from(`${auth.username}:${auth.password}`).toString("base64")}` : null;
150+ };
151+ const once = async (method, url, headers, body) => {
152+ const authorization = bearer ? `Bearer ${bearer}` : await basic();
153+ return fetchImpl(new URL(url, root).toString(), { method, headers: authorization ? { ...headers, authorization } : headers, body });
154+ };
155+ const send = async (method, url, { headers = {}, body } = {}) => {
156+ const response = await once(method, url, headers, body);
157+ const challenge = response.status === 401 ? bearerChallenge(response.headers.get("www-authenticate")) : null;
158+ if (!challenge) return response;
159+ // The token flow: the realm trades the credentials for a token.
160+ const realm = new URL(challenge.realm);
161+ if (challenge.service) realm.searchParams.set("service", challenge.service);
162+ realm.searchParams.set("scope", challenge.scope ?? `repository:${name}:pull,push`);
163+ const authorization = await basic();
164+ const got = await fetchImpl(realm.toString(), { method: "GET", headers: authorization ? { authorization } : {} });
165+ if (!got.ok) return response;
166+ const json = await got.json();
167+ bearer = json.token ?? json.access_token ?? null;
168+ return bearer ? once(method, url, headers, body) : response;
169+ };
170+ const at = (path) => `${root}/v2/${name}/${path}`;
171+ const fail = async (what, response) => {
172+ let detail = "";
173+ try {
174+ detail = (await response.text()).slice(0, 500);
175+ } catch {}
176+ return new Error(`${what}: the registry answered ${response.status}${detail ? ` ${detail}` : ""}`);
177+ };
178+ return {
179+ name,
180+ async manifest(reference) {
181+ const response = await send("GET", at(`manifests/${reference}`), { headers: { accept: ACCEPT } });
182+ if (!response.ok) throw await fail(`reading ${name} ${reference}`, response);
183+ const bytes = Buffer.from(await response.arrayBuffer());
184+ return { bytes, digest: sha256(bytes), json: JSON.parse(bytes.toString("utf8")) };
185+ },
186+ async blob(digest) {
187+ const response = await send("GET", at(`blobs/${digest}`));
188+ if (!response.ok) throw await fail(`reading blob ${digest}`, response);
189+ const bytes = Buffer.from(await response.arrayBuffer());
190+ if (sha256(bytes) !== digest) throw new Error(`blob ${digest} came back as ${sha256(bytes)}`);
191+ return bytes;
192+ },
193+ async hasBlob(digest) {
194+ const response = await send("HEAD", at(`blobs/${digest}`));
195+ if (response.status === 404) return false;
196+ if (!response.ok) throw await fail(`asking for blob ${digest}`, response);
197+ return true;
198+ },
199+ /** Uploads `bytes` unless the repository has them. Returns whether it did. */
200+ async putBlob(bytes, digest = sha256(bytes)) {
201+ if (await this.hasBlob(digest)) return false;
202+ const started = await send("POST", at("blobs/uploads/"));
203+ if (started.status !== 202) throw await fail(`starting an upload of ${digest}`, started);
204+ const location = started.headers.get("location");
205+ if (!location) throw new Error(`the registry started an upload of ${digest} without saying where`);
206+ const patched = await send("PATCH", location, {
207+ headers: { "content-type": "application/octet-stream" },
208+ body: bytes,
209+ });
210+ if (patched.status !== 202) throw await fail(`uploading ${digest}`, patched);
211+ const done = new URL(patched.headers.get("location") ?? location, root);
212+ done.searchParams.set("digest", digest);
213+ const put = await send("PUT", done.toString());
214+ if (put.status !== 201) throw await fail(`finishing the upload of ${digest}`, put);
215+ return true;
216+ },
217+ async putManifest(tag, bytes, type) {
218+ const response = await send("PUT", at(`manifests/${tag}`), { headers: { "content-type": type }, body: bytes });
219+ if (response.status !== 201) throw await fail(`writing ${name}:${tag}`, response);
220+ return response.headers.get("docker-content-digest");
221+ },
222+ };
223+}
224+
225+/**
226+ * Makes the runner's image `ref` in the registry from the base `base` (by
227+ * digest, as base.json records it) and the binary's bytes. Returns the new
228+ * manifest's digest. The base must be in the same registry; its layers
229+ * are named, not copied, so it must be in the same repository too.
230+ */
231+export async function appendRunnerImage({
232+ ref,
233+ base,
234+ binary,
235+ credentials,
236+ fetchImpl = fetch,
237+ scheme = "https",
238+ created = new Date().toISOString(),
239+ onLine = () => {},
240+ attempts = 3,
241+ wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms)),
242+}) {
243+ const target = parseRef(ref);
244+ const from = parseRef(base);
245+ if (!target.tag) throw new Error(`the runner's image needs a tag: ${ref}`);
246+ if (from.host !== target.host || from.name !== target.name) {
247+ throw new Error(`the base ${base} is not in the runner's repository ${target.host}/${target.name}`);
248+ }
249+ const client = repositoryClient(target.host, target.name, { credentials, fetchImpl, scheme });
250+
251+ let manifest = await client.manifest(from.digest ?? from.tag);
252+ if (from.digest && manifest.digest !== from.digest) throw new Error(`the base's manifest came back as ${manifest.digest}, not ${from.digest}`);
253+ const listed = manifest.json.mediaType;
254+ if (listed === OCI_INDEX || listed === DOCKER_LIST) {
255+ // An index: its linux/amd64 image, as `docker build --platform linux/amd64` would pick.
256+ const entry = manifest.json.manifests.find((m) => m.platform?.os === "linux" && m.platform?.architecture === "amd64");
257+ if (!entry) throw new Error(`the base ${base} has no linux/amd64 image`);
258+ manifest = await client.manifest(entry.digest);
259+ }
260+ const baseConfig = JSON.parse((await client.blob(manifest.json.config.digest)).toString("utf8"));
261+ // Every layer the new manifest names must be there.
262+ for (const layer of manifest.json.layers) {
263+ if (!(await client.hasBlob(layer.digest))) throw new Error(`the base's layer ${layer.digest} is not in ${target.name}`);
264+ }
265+
266+ const layer = runnerLayer(binary, { mtime: Date.parse(created) / 1000 });
267+ const configBytes = Buffer.from(JSON.stringify(runnerConfig(baseConfig, { diffId: layer.diffId, created })));
268+ const config = { digest: sha256(configBytes), size: configBytes.length };
269+ const type = manifestType(manifest.json);
270+ const manifestBytes = Buffer.from(JSON.stringify(runnerManifest(manifest.json, { config, layer })));
271+ const digest = sha256(manifestBytes);
272+
273+ onLine(`${target.tag}: the binary's layer ${layer.digest.slice(7, 19)}, ${(layer.size / 1e6).toFixed(1)} MB, on ${manifest.json.layers.length} layers of the base`);
274+ for (let attempt = 1; ; attempt++) {
275+ try {
276+ if (await client.putBlob(layer.bytes, layer.digest)) onLine(`uploaded the binary's layer`);
277+ await client.putBlob(configBytes, config.digest);
278+ const answered = await client.putManifest(target.tag, manifestBytes, type);
279+ if (answered && answered !== digest) throw new Error(`the registry kept ${target.tag} as ${answered}, not ${digest}`);
280+ onLine(`${target.tag}: ${digest}`);
281+ return digest;
282+ } catch (error) {
283+ // Cloudflare's registry can answer "blob unknown" for a blob it has
284+ // just taken; asking again finds it there.
285+ if (attempt >= attempts) throw error;
286+ onLine(`making ${target.tag} in the registry did not finish (attempt ${attempt} of ${attempts}): ${error.message ?? error}; trying again`);
287+ await wait(2000 * attempt);
288+ }
289+ }
290+}
+309−0
1+// The runner's image made in the registry (registry.mjs): its layer,
2+// config and manifest, and the calls it makes, against a registry kept in
3+// memory.
4+import assert from "node:assert/strict";
5+import { readFileSync } from "node:fs";
6+import { join } from "node:path";
7+import { test } from "node:test";
8+import { gunzipSync } from "node:zlib";
9+
10+import {
11+ DOCKER_MANIFEST,
12+ OCI_MANIFEST,
13+ RUNNER_BINARY_MODE,
14+ RUNNER_BINARY_PATH,
15+ RUNNER_ENTRYPOINT,
16+ appendRunnerImage,
17+ bearerChallenge,
18+ parseRef,
19+ runnerConfig,
20+ runnerLayer,
21+ runnerManifest,
22+ sha256,
23+ tarOf,
24+} from "./registry.mjs";
25+import { ROOT } from "./stack.mjs";
26+
27+/** A tar's entries: name, mode, size, owner, type and bytes, checksums checked. */
28+function untar(tar) {
29+ const entries = [];
30+ for (let at = 0; at + 512 <= tar.length; ) {
31+ const header = tar.subarray(at, at + 512);
32+ if (header.every((b) => b === 0)) break;
33+ const field = (start, length) => header.subarray(start, start + length).toString("ascii").replace(/\0.*$/s, "").trim();
34+ const summed = [...header].reduce((sum, byte, i) => sum + (i >= 148 && i < 156 ? 32 : byte), 0);
35+ assert.equal(Number.parseInt(field(148, 8), 8), summed, "the header's checksum");
36+ assert.equal(field(257, 6), "ustar");
37+ const size = Number.parseInt(field(124, 12), 8);
38+ entries.push({
39+ name: field(0, 100),
40+ mode: Number.parseInt(field(100, 8), 8),
41+ uid: Number.parseInt(field(108, 8), 8),
42+ gid: Number.parseInt(field(116, 8), 8),
43+ mtime: Number.parseInt(field(136, 12), 8),
44+ type: field(156, 1),
45+ data: tar.subarray(at + 512, at + 512 + size),
46+ });
47+ at += 512 + Math.ceil(size / 512) * 512;
48+ }
49+ return entries;
50+}
51+
52+test("the runner's layer is the binary alone, at its path and mode, owned by root", () => {
53+ const binary = Buffer.from("\x7fELF a runner, of a length that is not a whole block");
54+ const layer = runnerLayer(binary, { mtime: 1_760_000_000 });
55+ const tar = gunzipSync(layer.bytes);
56+ assert.equal(tar.length % 512, 0);
57+ assert.equal(layer.digest, sha256(layer.bytes));
58+ assert.equal(layer.size, layer.bytes.length);
59+ assert.equal(layer.diffId, sha256(tar));
60+ const entries = untar(tar);
61+ assert.equal(entries.length, 1);
62+ const [entry] = entries;
63+ assert.deepEqual(
64+ { name: entry.name, mode: entry.mode, uid: entry.uid, gid: entry.gid, mtime: entry.mtime, type: entry.type },
65+ { name: "usr/local/bin/g1t-runner", mode: 0o755, uid: 0, gid: 0, mtime: 1_760_000_000, type: "0" },
66+ );
67+ assert.ok(entry.data.equals(binary));
68+ // The same bytes make the same layer.
69+ assert.equal(runnerLayer(binary, { mtime: 1_760_000_000 }).digest, layer.digest);
70+ // An exact number of blocks, and an empty file.
71+ assert.equal(untar(tarOf("a", Buffer.alloc(512, 1)))[0].data.length, 512);
72+ assert.equal(untar(tarOf("a", Buffer.alloc(0)))[0].data.length, 0);
73+});
74+
75+test("the layer and entrypoint are what services/runner/Dockerfile builds", () => {
76+ const dockerfile = readFileSync(join(ROOT, "services/runner/Dockerfile"), "utf8");
77+ const copy = /^COPY --chmod=(\d+) g1t-runner (\S+)$/m.exec(dockerfile);
78+ assert.ok(copy, "the Dockerfile copies the binary in");
79+ assert.equal(Number.parseInt(copy[1], 8), RUNNER_BINARY_MODE);
80+ assert.equal(copy[2], `/${RUNNER_BINARY_PATH}`);
81+ const entrypoint = /^ENTRYPOINT (.+)$/m.exec(dockerfile);
82+ assert.deepEqual(JSON.parse(entrypoint[1]), RUNNER_ENTRYPOINT);
83+ // Nothing else that would change the image.
84+ const instructions = dockerfile.split(/\r?\n/).filter((line) => line.trim() && !line.startsWith("#")).map((line) => line.split(" ")[0]);
85+ assert.deepEqual(instructions, ["ARG", "FROM", "COPY", "ENTRYPOINT"]);
86+});
87+
88+const BASE_CONFIG = {
89+ architecture: "amd64",
90+ os: "linux",
91+ created: "2026-10-08T19:50:00Z",
92+ config: { Env: ["PATH=/usr/local/bin:/usr/bin"], Cmd: ["bash"], WorkingDir: "/home/runner", User: "root" },
93+ rootfs: { type: "layers", diff_ids: ["sha256:" + "1".repeat(64), "sha256:" + "2".repeat(64)] },
94+ history: [{ created_by: "FROM debian" }, { created_by: "RUN apt-get install" }],
95+};
96+
97+test("the config is the base's, with the layer, the entrypoint and no inherited command", () => {
98+ const before = structuredClone(BASE_CONFIG);
99+ const diffId = "sha256:" + "3".repeat(64);
100+ const config = runnerConfig(BASE_CONFIG, { diffId, created: "2026-10-13T00:00:00.000Z" });
101+ assert.deepEqual(BASE_CONFIG, before, "the base's config is left as it was");
102+ assert.deepEqual(config.config.Entrypoint, ["g1t-runner"]);
103+ assert.equal(config.config.Cmd, undefined);
104+ assert.deepEqual(config.config.Env, BASE_CONFIG.config.Env);
105+ assert.equal(config.config.WorkingDir, "/home/runner");
106+ assert.deepEqual(config.rootfs.diff_ids, [...BASE_CONFIG.rootfs.diff_ids, diffId]);
107+ assert.equal(config.created, "2026-10-13T00:00:00.000Z");
108+ // History: one entry per layer, plus one that adds none.
109+ assert.equal(config.history.filter((h) => !h.empty_layer).length - BASE_CONFIG.history.length, 1);
110+ assert.equal(config.history.at(-1).empty_layer, true);
111+ assert.throws(() => runnerConfig({ ...BASE_CONFIG, architecture: "arm64" }, { diffId, created: "x" }), /linux\/amd64/);
112+});
113+
114+test("the manifest names the base's layers, then the binary's, in the base's format", () => {
115+ const layers = [
116+ { mediaType: "application/vnd.docker.image.rootfs.diff.tar.gzip", digest: "sha256:" + "a".repeat(64), size: 10 },
117+ { mediaType: "application/vnd.docker.image.rootfs.diff.tar.gzip", digest: "sha256:" + "b".repeat(64), size: 20 },
118+ ];
119+ const config = { digest: "sha256:" + "c".repeat(64), size: 5 };
120+ const layer = { digest: "sha256:" + "d".repeat(64), size: 7 };
121+ const docker = runnerManifest(
122+ { schemaVersion: 2, mediaType: DOCKER_MANIFEST, config: { mediaType: "application/vnd.docker.container.image.v1+json", digest: "x", size: 1 }, layers },
123+ { config, layer },
124+ );
125+ assert.deepEqual(docker, {
126+ schemaVersion: 2,
127+ mediaType: DOCKER_MANIFEST,
128+ config: { mediaType: "application/vnd.docker.container.image.v1+json", ...config },
129+ layers: [...layers, { mediaType: "application/vnd.docker.image.rootfs.diff.tar.gzip", ...layer }],
130+ });
131+ // OCI, with its type left unsaid.
132+ const oci = runnerManifest(
133+ { schemaVersion: 2, config: { mediaType: "application/vnd.oci.image.config.v1+json", digest: "x", size: 1 }, layers: [], annotations: { a: "b" } },
134+ { config, layer },
135+ );
136+ assert.equal(oci.mediaType, OCI_MANIFEST);
137+ assert.deepEqual(oci.layers, [{ mediaType: "application/vnd.oci.image.layer.v1.tar+gzip", ...layer }]);
138+ assert.equal(oci.annotations, undefined);
139+});
140+
141+test("image references and bearer challenges are read", () => {
142+ const digest = "sha256:" + "e".repeat(64);
143+ assert.deepEqual(parseRef("registry.cloudflare.com/acct/g1t-runner:abc"), { host: "registry.cloudflare.com", name: "acct/g1t-runner", tag: "abc", digest: null });
144+ assert.deepEqual(parseRef(`localhost:5000/g1t-runner:base-1@${digest}`), { host: "localhost:5000", name: "g1t-runner", tag: "base-1", digest });
145+ assert.deepEqual(parseRef(`r.example/x@${digest}`), { host: "r.example", name: "x", tag: null, digest });
146+ assert.throws(() => parseRef("r.example/x"));
147+ assert.deepEqual(bearerChallenge('Bearer realm="https://auth.example/token",service="registry",scope="repository:x:pull"'), {
148+ realm: "https://auth.example/token",
149+ service: "registry",
150+ scope: "repository:x:pull",
151+ });
152+ assert.equal(bearerChallenge('Basic realm="x"'), null);
153+ assert.equal(bearerChallenge(null), null);
154+});
155+
156+/** A registry in memory: one repository, Basic auth, uploads by POST, PATCH and PUT. */
157+function fakeRegistry({ name, auth = "Basic " + Buffer.from("u:p").toString("base64") }) {
158+ const blobs = new Map();
159+ const manifests = new Map();
160+ const uploads = new Map();
161+ const calls = [];
162+ let next = 0;
163+ const reply = (status, { headers = {}, body = null } = {}) => ({
164+ status,
165+ ok: status >= 200 && status < 300,
166+ headers: new Headers(headers),
167+ arrayBuffer: async () => (body ? body.buffer.slice(body.byteOffset, body.byteOffset + body.byteLength) : new ArrayBuffer(0)),
168+ text: async () => (body ? body.toString("utf8") : ""),
169+ json: async () => JSON.parse(body.toString("utf8")),
170+ });
171+ const fetchImpl = async (url, init) => {
172+ const { pathname, searchParams } = new URL(url);
173+ calls.push(`${init.method} ${pathname}`);
174+ if (init.headers.authorization !== auth) return reply(401);
175+ const prefix = `/v2/${name}/`;
176+ if (!pathname.startsWith(prefix)) return reply(404);
177+ const path = pathname.slice(prefix.length);
178+ let match;
179+ if ((match = /^manifests\/(.+)$/.exec(path))) {
180+ if (init.method === "PUT") {
181+ const bytes = Buffer.from(init.body);
182+ const digest = sha256(bytes);
183+ for (const ref of [...JSON.parse(bytes).layers, JSON.parse(bytes).config]) {
184+ if (!blobs.has(ref.digest)) return reply(400, { body: Buffer.from('{"errors":[{"code":"MANIFEST_BLOB_UNKNOWN"}]}') });
185+ }
186+ manifests.set(match[1], { bytes, type: init.headers["content-type"] });
187+ manifests.set(digest, { bytes, type: init.headers["content-type"] });
188+ return reply(201, { headers: { "docker-content-digest": digest } });
189+ }
190+ const found = manifests.get(match[1]);
191+ return found ? reply(200, { headers: { "content-type": found.type }, body: found.bytes }) : reply(404);
192+ }
193+ if ((match = /^blobs\/(sha256:[0-9a-f]{64})$/.exec(path))) {
194+ const found = blobs.get(match[1]);
195+ if (!found) return reply(404);
196+ return init.method === "HEAD" ? reply(200) : reply(200, { body: found });
197+ }
198+ if (path === "blobs/uploads/" && init.method === "POST") {
199+ const id = `u${next++}`;
200+ uploads.set(id, Buffer.alloc(0));
201+ return reply(202, { headers: { location: `${prefix}blobs/uploads/${id}` } });
202+ }
203+ if ((match = /^blobs\/uploads\/(u\d+)$/.exec(path))) {
204+ const id = match[1];
205+ if (init.method === "PATCH") {
206+ uploads.set(id, Buffer.concat([uploads.get(id), Buffer.from(init.body)]));
207+ return reply(202, { headers: { location: `${prefix}blobs/uploads/${id}?state=1` } });
208+ }
209+ if (init.method === "PUT") {
210+ const bytes = uploads.get(id);
211+ if (sha256(bytes) !== searchParams.get("digest")) return reply(400);
212+ blobs.set(searchParams.get("digest"), bytes);
213+ return reply(201);
214+ }
215+ }
216+ return reply(405);
217+ };
218+ return { blobs, manifests, calls, fetchImpl };
219+}
220+
221+test("the runner's image is made in the registry from the base's manifest and one new layer, pulling none of the base", async () => {
222+ const registry = fakeRegistry({ name: "acct/g1t-runner" });
223+ // The base, as `docker push` left it.
224+ const layerBlobs = [Buffer.from("base layer one"), Buffer.from("base layer two")];
225+ for (const blob of layerBlobs) registry.blobs.set(sha256(blob), blob);
226+ const configBytes = Buffer.from(JSON.stringify(BASE_CONFIG));
227+ registry.blobs.set(sha256(configBytes), configBytes);
228+ const baseManifest = Buffer.from(
229+ JSON.stringify({
230+ schemaVersion: 2,
231+ mediaType: OCI_MANIFEST,
232+ config: { mediaType: "application/vnd.oci.image.config.v1+json", digest: sha256(configBytes), size: configBytes.length },
233+ layers: layerBlobs.map((blob) => ({ mediaType: "application/vnd.oci.image.layer.v1.tar+gzip", digest: sha256(blob), size: blob.length })),
234+ }),
235+ );
236+ registry.manifests.set(sha256(baseManifest), { bytes: baseManifest, type: OCI_MANIFEST });
237+ const base = `registry.cloudflare.com/acct/g1t-runner:base-20261008-a314499af03b@${sha256(baseManifest)}`;
238+ const ref = "registry.cloudflare.com/acct/g1t-runner:0123456789abcdef";
239+ const binary = Buffer.from("the runner binary");
240+ const credentials = async () => ({ username: "u", password: "p" });
241+
242+ const digest = await appendRunnerImage({ ref, base, binary, credentials, fetchImpl: registry.fetchImpl, created: "2026-10-13T00:00:00.000Z" });
243+ const made = registry.manifests.get("0123456789abcdef");
244+ assert.equal(sha256(made.bytes), digest);
245+ assert.equal(made.type, OCI_MANIFEST);
246+ const manifest = JSON.parse(made.bytes);
247+ assert.deepEqual(manifest.layers.slice(0, 2).map((l) => l.digest), layerBlobs.map(sha256));
248+ assert.equal(manifest.layers.length, 3);
249+ const config = JSON.parse(registry.blobs.get(manifest.config.digest));
250+ assert.deepEqual(config.config.Entrypoint, ["g1t-runner"]);
251+ assert.equal(config.rootfs.diff_ids.at(-1), sha256(gunzipSync(registry.blobs.get(manifest.layers[2].digest))));
252+ assert.ok(untar(gunzipSync(registry.blobs.get(manifest.layers[2].digest)))[0].data.equals(binary));
253+ // The base's layers were asked about, never downloaded.
254+ for (const blob of layerBlobs) assert.ok(!registry.calls.includes(`GET /v2/acct/g1t-runner/blobs/${sha256(blob)}`));
255+
256+ // Made again: the layer is already there, so only the config and manifest are written.
257+ const before = registry.calls.length;
258+ await appendRunnerImage({ ref, base, binary, credentials, fetchImpl: registry.fetchImpl, created: "2026-10-13T00:00:00.000Z" });
259+ assert.ok(!registry.calls.slice(before).some((call) => call.startsWith("PATCH")));
260+});
261+
262+test("a base that is not where it should be, or not what base.json says, stops it", async () => {
263+ const registry = fakeRegistry({ name: "acct/g1t-runner" });
264+ const credentials = async () => ({ username: "u", password: "p" });
265+ const digest = "sha256:" + "f".repeat(64);
266+ const options = { binary: Buffer.from("x"), credentials, fetchImpl: registry.fetchImpl, attempts: 1 };
267+ await assert.rejects(
268+ appendRunnerImage({ ...options, ref: "registry.cloudflare.com/acct/g1t-runner:t", base: `registry.cloudflare.com/acct/other:b@${digest}` }),
269+ /not in the runner's repository/,
270+ );
271+ await assert.rejects(appendRunnerImage({ ...options, ref: "registry.cloudflare.com/acct/g1t-runner:t", base: `registry.cloudflare.com/acct/g1t-runner:b@${digest}` }), /404/);
272+ // Wrong credentials: the registry's answer is the error.
273+ await assert.rejects(
274+ appendRunnerImage({ ...options, credentials: async () => ({ username: "u", password: "wrong" }), ref: "registry.cloudflare.com/acct/g1t-runner:t", base: `registry.cloudflare.com/acct/g1t-runner:b@${digest}` }),
275+ /401/,
276+ );
277+});
278+
279+test("a registry that asks for a bearer token is given the one its realm trades the credentials for", async () => {
280+ const registry = fakeRegistry({ name: "acct/r", auth: "Bearer tok" });
281+ const inner = registry.fetchImpl;
282+ const asked = [];
283+ const fetchImpl = async (url, init) => {
284+ if (url.startsWith("https://auth.example/")) {
285+ asked.push([url, init.headers.authorization]);
286+ return { ok: true, status: 200, json: async () => ({ token: "tok" }) };
287+ }
288+ const answer = await inner(url, init);
289+ if (answer.status === 401) answer.headers.set("www-authenticate", 'Bearer realm="https://auth.example/token",service="r.example"');
290+ return answer;
291+ };
292+ const config = Buffer.from(JSON.stringify(BASE_CONFIG));
293+ registry.blobs.set(sha256(config), config);
294+ const manifest = Buffer.from(
295+ JSON.stringify({ schemaVersion: 2, mediaType: DOCKER_MANIFEST, config: { mediaType: "application/vnd.docker.container.image.v1+json", digest: sha256(config), size: config.length }, layers: [] }),
296+ );
297+ registry.manifests.set("base", { bytes: manifest, type: DOCKER_MANIFEST });
298+ await appendRunnerImage({
299+ ref: "r.example/acct/r:new",
300+ base: "r.example/acct/r:base",
301+ binary: Buffer.from("x"),
302+ credentials: async () => ({ username: "u", password: "p" }),
303+ fetchImpl,
304+ });
305+ assert.ok(registry.manifests.has("new"));
306+ assert.equal(asked.length, 1, "one token, kept");
307+ assert.equal(asked[0][0], "https://auth.example/token?service=r.example&scope=repository%3Aacct%2Fr%3Apull%2Cpush");
308+ assert.equal(asked[0][1], `Basic ${Buffer.from("u:p").toString("base64")}`);
309+});
+7−3
66 # runner, a static Linux binary built outside Docker by
77 # scripts/build-runner.mjs.
88 #
9−# Built by scripts/deploy.mjs, which passes BASE (the image recorded in
10−# services/runner/base.json) and uses a build context holding only the
11−# binary, so this takes seconds. docs/DEPLOYING.md explains the two images.
9+# Deploys do not build this file: scripts/deploy/registry.mjs makes the
10+# same image in the registry, adding one layer with the binary to the
11+# base's layers (the same path, mode and entrypoint; a test in
12+# scripts/deploy holds the two together), so a deploy job never pulls the
13+# base. scripts/deploy.mjs builds it with Docker for a dry run, or when
14+# that fails, passing BASE (the image recorded in services/runner/base.json)
15+# and a build context holding only the binary.
1216
1317 ARG BASE
1418 FROM ${BASE}