Skip to content
Sign up

flagon-io/g1t

public
0
584 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Checkout tries a failed fetch twice more, after 2 s and 5 s1#!/usr/bin/env node
2// Deploys g1t to Cloudflare from deploy/stack.jsonc: only what changed since
3// each Worker's live commit, migrations first, then stage by stage.
The docs folder is gone, and what it held lives where people read it: how a self-hosted g1t runs and how to deploy g1t to Cloudflare are pages on docs.g1t.sh under Run g1t yourself, and speed, rate limits and operating g1t.sh are sections of CONTRIBUTING.md; code that cited a file in docs/ now points to the page or section that covers it, or says what it means itself, and applied migrations and the runner images are left as they were.4// docs.g1t.sh/guides/deploy-to-cloudflare/ is the guide.
Checkout tries a failed fetch twice more, after 2 s and 5 s5//
6// node scripts/deploy.mjs plan what would deploy, and why (read-only)
7// node scripts/deploy.mjs deploy migrations, then every changed unit
8// node scripts/deploy.mjs deploy --only web,api just these (if changed; --force: anyway)
9// node scripts/deploy.mjs build --only events build as a deploy would, upload nothing
10// node scripts/deploy.mjs migrate pending D1 migrations only
11// node scripts/deploy.mjs manifest [--check] the resolved manifest, or its problems
12// node scripts/deploy.mjs doctor which units lack their secrets
13// node scripts/deploy.mjs install --only a,b npm ci of just what those units need (CI)
14// node scripts/deploy.mjs build-base build and push the runner's base image (Docker)
15// node scripts/deploy.mjs image build and push the runner's image for this checkout (Docker)
16//
17// Flags: --all (every unit, changed or not), --only a,b, --skip a,b,
18// --force, --concurrency N (default 4), --stage core (one stage),
19// --json (plan), --out FILE (plan), --no-migrations, --allow-dirty,
20// --rebuild-image, --rebuild-base, --since REV (Workers with no recorded
21// commit are taken to run REV); for build-base and image, --no-push, and
22// for build-base, --no-cache.
23
24import { appendFileSync, mkdirSync, writeFileSync } from "node:fs";
25import { tmpdir } from "node:os";
26import { join } from "node:path";
27
28import { OUT_DIR } from "./build-runner.mjs";
29import { ensureWorkerBuild } from "./build-rust-worker.mjs";
30import {
31 annotation,
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check32 apiAuth,
Checkout tries a failed fetch twice more, after 2 s and 5 s33 applyMigrations,
Merge main into rm-agents: mirroring removal, open signups, free-tier limits, the g1t CLI and runner fixes, with the agent removal kept. The CLI's table of operations is regenerated from the API without the agent, customize and AI credit operations; no CLI command used them.34 DEPLOY_RETRY_WAITS_MS,
Checkout tries a failed fetch twice more, after 2 s and 5 s35 dockerAvailable,
36 exec,
37 jsonFrom,
38 lastLines,
39 pendingMigrations,
40 readLive,
Merge main into rm-agents: mirroring removal, open signups, free-tier limits, the g1t CLI and runner fixes, with the agent removal kept. The CLI's table of operations is regenerated from the API without the agent, customize and AI credit operations; no CLI command used them.41 retryableDeploy,
Checkout tries a failed fetch twice more, after 2 s and 5 s42 versionFrom,
43 wrangler,
44 wranglerEnv,
Merge main into rm-agents: mirroring removal, open signups, free-tier limits, the g1t CLI and runner fixes, with the agent removal kept. The CLI's table of operations is regenerated from the API without the agent, customize and AI credit operations; no CLI command used them.45 wranglerError,
Checkout tries a failed fetch twice more, after 2 s and 5 s46} from "./deploy/cloudflare.mjs";
47import {
48 baseInputs,
49 baseState,
50 baseTag,
51 baseVersions,
52 buildBase,
53 buildRunnerImage,
54 dockerHas,
55 dockerLogin,
56 imageSize,
57 pushImage,
58 readBaseLock,
59 registryHas,
60 removeDeployConfig,
61 runnerRef,
62 writeBaseLock,
63 writeDeployConfig,
64} from "./deploy/image.mjs";
65import { decide, git, planJson, pool, table } from "./deploy/plan.mjs";
Deploying: a deploy by hand shows on the repository's Deployments page66import { reportDeployment } from "./deploy/report.mjs";
The artifacts service is services/artifacts, the Worker g1t-artifacts, bound as ARTIFACTS by the API, the site and the agents; its live rooms move to it with a Durable Object transfer from g1t-docs-service, and its database, bucket, indexes and queue keep their names. The git store's binding and settings are GITSTORE, its ops scripts gitstore-*, and workflow run artifacts keep their compatible API under run_artifacts modules. The deploy tool puts a Worker that has never deployed before the Workers in its stage that bind to it, and the deploy guide gives the cutover runbook.67import { ROOT, byStage, codeStages, findWranglerConfigs, npmCiArgs, npmWorkspace, pick, problems, resolvedStack, waves } from "./deploy/stack.mjs";
Merge status detection: first-byte speed probe, deploy windows, 4 of 5 with a re-check, check history, reminders68import { withDeployWindow } from "./deploy/status-window.mjs";
Checkout tries a failed fetch twice more, after 2 s and 5 s69
Deploying: never roll production back by accident70const USAGE = "usage: node scripts/deploy.mjs plan|deploy|build|migrate|manifest|doctor|install|build-base|image [--all] [--only a,b] [--skip a,b] [--force] [--rollback] [--concurrency N] [--stage S] [--json]";
Checkout tries a failed fetch twice more, after 2 s and 5 s71
72function parseArgs(argv) {
73 const opts = { command: argv[0], only: [], skip: [], concurrency: 4, force: false, all: false, json: false };
74 for (let i = 1; i < argv.length; i++) {
75 const arg = argv[i];
76 const [flag, inline] = arg.split(/=(.*)/s);
77 const value = () => inline ?? argv[++i];
78 if (flag === "--only") opts.only.push(value());
79 else if (flag === "--skip") opts.skip.push(value());
80 else if (flag === "--concurrency") opts.concurrency = Number(value());
81 else if (flag === "--stage") opts.stage = value();
82 else if (flag === "--all") opts.all = true;
83 else if (flag === "--force") opts.force = true;
Deploying: never roll production back by accident84 else if (flag === "--rollback") opts.rollback = true;
Checkout tries a failed fetch twice more, after 2 s and 5 s85 else if (flag === "--json") opts.json = true;
86 else if (flag === "--check") opts.check = true;
87 else if (flag === "--no-migrations") opts.noMigrations = true;
88 else if (flag === "--allow-dirty") opts.allowDirty = true;
89 else if (flag === "--rebuild-image") opts.rebuildImage = true;
90 else if (flag === "--rebuild-base") opts.rebuildBase = true;
91 else if (flag === "--no-push") opts.noPush = true;
92 else if (flag === "--no-cache") opts.noCache = true;
93 else if (flag === "--out") opts.out = value();
94 else if (flag === "--since") opts.since = value();
95 else if (flag === "--github-output") opts.githubOutput = true;
96 else throw new Error(`unknown flag ${arg}\n${USAGE}`);
97 }
98 if (!Number.isInteger(opts.concurrency) || opts.concurrency < 1) throw new Error("--concurrency is a whole number, 1 or more");
99 return opts;
100}
101
102/** The units a command works on: --only (or all), less --skip, in --stage. */
103function selected(stack, opts) {
104 let units = opts.only.length ? pick(stack, opts.only) : [...stack.units];
105 const skipped = pick(stack, opts.skip);
106 units = units.filter((u) => !skipped.includes(u));
107 if (opts.stage) {
108 if (!codeStages(stack).includes(opts.stage)) throw new Error(`--stage is one of ${codeStages(stack).join(", ")}`);
109 units = units.filter((u) => u.stage === opts.stage);
110 }
111 // Manifest order, whatever order they were named in.
112 return stack.units.filter((u) => units.includes(u));
113}
114
115const log = (...args) => console.error(...args);
116
117/**
118 * The plan for a workflow (.g1t/workflows/deploy.yml): job outputs in
119 * $GITHUB_OUTPUT, and the plan as a table in $GITHUB_STEP_SUMMARY.
120 */
121function writeGithubOutputs(data, p) {
122 const lines = [
123 `commit=${data.commit}`,
124 `migrate=${data.migrations.length > 0}`,
125 `migrate_units=${data.migrations.map((m) => m.unit).join(",")}`,
126 `deploying=${data.units.filter((u) => u.deploy).map((u) => u.unit).join(",")}`,
127 ];
128 for (const [stage, { jobs }] of Object.entries(data.stages)) {
129 // A matrix needs one entry; an empty stage's job is skipped by its `if`.
130 const include = jobs.length ? jobs : [{ group: "none", units: "" }];
131 lines.push(`has_${stage}=${jobs.length > 0}`, `${stage}=${JSON.stringify({ include })}`);
132 }
Deploying: a Plan that cannot read migrations says why133 if (data.migration_errors.length) {
134 // The units, then why for the first: one cause (a token, say) is usually all of them.
135 throw new Error(`Could not read pending migrations: ${data.migration_errors.map((e) => e.unit).join(", ")}
136${data.migration_errors[0].error}`);
137 }
Checkout tries a failed fetch twice more, after 2 s and 5 s138 if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `${lines.join("\n")}\n`);
139 else console.log(lines.join("\n"));
140 if (process.env.GITHUB_STEP_SUMMARY) {
141 const rows = p.decisions.map((d) => `| ${d.unit.id} | ${d.unit.stage} | ${d.deploy ? "deploy" : ""} | ${d.since ? d.since.slice(0, 12) : "?"} | ${d.reason.replaceAll("|", "\\|")} |`);
142 const pending = data.migrations.map((m) => `- ${m.database}: ${m.pending.join(", ")}`);
143 appendFileSync(
144 process.env.GITHUB_STEP_SUMMARY,
145 [`## Deploy plan for ${data.commit.slice(0, 12)}`, "", "| unit | stage | action | live | why |", "| --- | --- | --- | --- | --- |", ...rows, "", pending.length ? "### Pending migrations" : "", ...pending, ""].join("\n"),
146 );
147 }
148}
149
150const seconds = (ms) => `${(ms / 1000).toFixed(1)}s`;
151
152/** Reads what each unit runs and what its database is waiting for. */
153async function survey(units, opts) {
154 const live = {};
155 const migrations = {};
156 const tasks = [
157 ...(opts.noLive ? [] : units).map((unit) => async () => {
158 live[unit.id] = await readLive(unit);
159 }),
160 ...(opts.noMigrations ? [] : units.filter((u) => u.d1)).map((unit) => async () => {
161 migrations[unit.id] = await pendingMigrations(unit);
162 }),
163 ];
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check164 // Through the API every read starts at once (the requests themselves are
165 // held to API_CONCURRENCY); through Wrangler, a few processes at a time.
166 await pool(tasks, apiAuth() ? tasks.length : Math.max(8, opts.concurrency * 2), (task) => task());
Checkout tries a failed fetch twice more, after 2 s and 5 s167 return { live, migrations };
168}
169
170async function plan(stack, opts) {
171 const units = selected(stack, opts);
172 const head = git.head();
173 const { live, migrations } = await survey(units, opts);
174 // --since: what a Worker with no recorded commit is taken to run (once,
175 // to adopt Workers deployed before this tool), for example --since HEAD~3.
176 if (opts.since) {
177 const since = git.resolve(opts.since);
178 for (const unit of units) {
179 const found = live[unit.id];
180 if (found && !found.sha && !found.missing && !found.error) live[unit.id] = { ...found, sha: since, assumed: true };
181 }
182 }
Deploying: never roll production back by accident183 const decisions = decide(units, { live, head, force: opts.force || opts.all, rollback: opts.rollback });
Checkout tries a failed fetch twice more, after 2 s and 5 s184 return { head, units, live, migrations, decisions };
185}
186
187function buildPlan(stack, opts) {
188 const units = selected(stack, opts);
189 return {
190 head: git.head(),
191 units,
192 live: {},
193 migrations: {},
194 decisions: units.map((unit) => ({ unit, deploy: true, reason: "build", since: null, files: [], image: false })),
195 };
196}
197
198function printPlan(stack, { head, decisions, migrations, live }) {
199 console.log(`Deploying ${head.slice(0, 12)} (${git.subject()})\n`);
200 const rows = decisions.map((d) => [
201 d.unit.id,
202 d.unit.stage,
203 d.deploy ? "deploy" : "-",
204 d.since ? d.since.slice(0, 12) + (live[d.unit.id]?.assumed ? "*" : "") : "?",
205 d.unit.d1 ? (migrations[d.unit.id]?.error ? "error" : String(migrations[d.unit.id]?.pending?.length ?? "-")) : "",
206 d.reason + (d.image ? "; image rebuilds" : ""),
207 ]);
208 console.log(table(rows, ["unit", "stage", "action", "live", "migrations", "why"]));
209 if (decisions.some((d) => live[d.unit.id]?.assumed)) console.log("* taken from --since: no commit was recorded for it");
210 const pending = Object.entries(migrations).filter(([, m]) => m.pending?.length);
211 if (pending.length) {
212 console.log("\nPending migrations:");
213 for (const [id, m] of pending) console.log(` ${stack.units.find((u) => u.id === id).d1.database}: ${m.pending.join(", ")}`);
214 }
215 for (const [id, m] of Object.entries(migrations).filter(([, m]) => m.error)) {
216 console.log(`\nCould not list ${id}'s migrations:\n${m.error}`);
217 }
218 const deploying = decisions.filter((d) => d.deploy).map((d) => d.unit);
219 console.log(
220 deploying.length
221 ? `\n${deploying.length} to deploy: ${byStage(stack, deploying).map((g) => `${g.stage} (${g.units.map((u) => u.id).join(", ")})`).join(" -> ")}`
222 : "\nNothing to deploy.",
223 );
224}
225
226/** Output of one unit, prefixed, to the terminal and a log file. */
227function unitLogger(id) {
228 const dir = join(tmpdir(), "g1t-deploy");
229 mkdirSync(dir, { recursive: true });
230 const file = join(dir, `${id}.log`);
231 const lines = [];
232 return {
233 file,
234 line: (text) => {
235 lines.push(text);
236 if (text.trim()) log(`[${id}] ${text}`);
237 },
238 save: () => writeFileSync(file, `${lines.join("\n")}\n`),
239 };
240}
241
242/**
243 * The runner's image for this checkout, by registry reference: already in
244 * the registry (built by an earlier deploy, `deploy.mjs image`, or on
245 * another machine), or built and pushed here, which needs Docker. A dry
246 * run builds it locally and pushes nothing. Returns { ref, note }.
247 */
248async function runnerImage(unit, { dryRun, docker, rebuildImage, rebuildBase }, out) {
249 let base = baseState(unit);
250 if (!base.current || rebuildBase) {
251 if (!rebuildBase) {
252 throw new Error(
253 `${unit.image.base.context} has changed since ${unit.image.base.lock} was written${base.lock ? "" : " (the base has never been built)"}. Build and push the base, and commit ${unit.image.base.lock}: node scripts/deploy.mjs build-base`,
254 );
255 }
256 if (!docker) throw new Error("--rebuild-base needs Docker.");
257 await newBase(unit, { push: !dryRun, onLine: out.line });
258 base = baseState(unit);
259 }
260 if (!base.pushed && !dryRun) throw new Error(`${unit.image.base.lock} records a base that was never pushed: node scripts/deploy.mjs build-base`);
261 const ref = runnerRef(unit);
262 const tag = ref.split(":").pop();
263 if (!rebuildImage) {
264 if (dryRun && docker && (await dockerHas(ref))) return { ref, note: `image ${tag} already built here` };
265 if (!dryRun) {
266 try {
267 if (await registryHas(ref)) return { ref, note: `image ${tag} already in the registry` };
268 } catch (error) {
269 out.line(`could not ask the registry for ${tag}: ${error.message ?? error}`);
270 }
271 }
272 }
273 if (!docker) {
274 throw new Error(
Merge branch 'main' into actions-toolkit-oidc-artifacts275 `its image ${tag} is not in the registry, and Docker does not answer here. Build and push it where Docker runs (a g1t Actions job, or a machine with Docker: node scripts/deploy.mjs image), then run this again.`,
Checkout tries a failed fetch twice more, after 2 s and 5 s276 );
277 }
278 const started = Date.now();
279 const binary = await exec(process.execPath, [join(ROOT, "scripts/build-runner.mjs")], { onLine: out.line });
280 if (binary.code !== 0) throw new Error(`the runner binary did not build:\n${lastLines(binary.out)}`);
281 if (!dryRun) await dockerLogin({ push: true });
282 await buildRunnerImage(unit, { ref, base: base.ref, binaryDir: OUT_DIR, onLine: out.line });
283 if (!dryRun) await pushImage(ref, { onLine: out.line });
284 return { ref, note: `image ${tag} ${dryRun ? "built" : "built and pushed"} in ${seconds(Date.now() - started)}` };
285}
286
287/** Builds the base (and pushes it unless `push` is false), and writes its lock. */
288async function newBase(unit, { push = true, noCache = false, onLine = log } = {}) {
289 const started = Date.now();
290 const inputs = baseInputs(unit);
291 const tag = baseTag(inputs);
292 const previous = readBaseLock(unit);
293 // Logged in, the base it replaces is pulled as cache.
294 if (push || previous?.pushed) {
295 try {
296 await dockerLogin({ push });
297 } catch (error) {
298 if (push) throw error;
299 onLine(`not logged in to the registry, so no cache from it: ${error.message ?? error}`);
300 }
301 }
302 const ref = await buildBase(unit, { tag, previous: previous?.pushed ? previous.image : null, onLine, noCache });
303 const built = Date.now();
304 const [size, versions] = await Promise.all([imageSize(ref), baseVersions(ref)]);
305 const digest = push ? await pushImage(ref, { onLine }) : null;
306 const lock = { image: ref, digest, pushed: push, inputs, built_at: new Date().toISOString(), size_bytes: size, versions };
307 writeBaseLock(unit, lock);
308 onLine(`base ${tag}: built in ${seconds(built - started)}${push ? `, pushed in ${seconds(Date.now() - built)}` : ""}, ${(size / 1e9).toFixed(2)} GB unpacked`);
309 return lock;
310}
311
312/** Builds (and unless `dryRun`, deploys) one unit. */
313async function ship(unit, decision, { head, subject, dirty, dryRun, docker, rebuildImage, rebuildBase }) {
314 const started = Date.now();
315 const out = unitLogger(unit.id);
316 const cwd = join(ROOT, unit.path);
317 const result = { unit: unit.id, stage: unit.stage, ok: false, version: null, ms: 0, note: "" };
318 try {
319 if (unit.kind === "react-router" || unit.kind === "astro") {
320 // One command string: Node warns about arguments passed beside shell: true.
321 const built = await exec("npm run build", [], { cwd, onLine: out.line, shell: true, env: wranglerEnv() });
322 if (built.code !== 0) throw new Error(`npm run build failed:\n${lastLines(built.out)}`);
323 }
324 const args = ["deploy"];
325 if (dryRun) {
326 args.push("--dry-run", "--outdir", join(tmpdir(), "g1t-deploy", "dist", unit.id));
327 } else {
328 const { message, tag } = annotation(head, subject);
329 args.push("--message", dirty ? message.replace(/^g1t-deploy/, "g1t-deploy-dirty") : message, "--tag", tag);
330 }
331 if (unit.image) {
332 // Wrangler is given the image by reference, so it builds nothing.
333 const image = await runnerImage(unit, { dryRun, docker, rebuildImage, rebuildBase }, out);
334 args.push("--config", writeDeployConfig(unit, image.ref));
335 // Nothing the image is built from changed: the running sandboxes
336 // keep going, and new ones start from the same image.
337 if (!rebuildImage && !rebuildBase && !decision.image) args.push("--containers-rollout", "none");
338 result.note = image.note;
339 }
Merge main into rm-agents: mirroring removal, open signups, free-tier limits, the g1t CLI and runner fixes, with the agent removal kept. The CLI's table of operations is regenerated from the API without the agent, customize and AI credit operations; no CLI command used them.340 // Tried again after a wait when Cloudflare, not what is deployed, is
341 // why it failed: deploying the same version again is harmless.
342 let deployed = await wrangler(args, { cwd, onLine: out.line });
343 for (const wait of DEPLOY_RETRY_WAITS_MS) {
344 if (deployed.code === 0 || dryRun || !retryableDeploy(deployed.out)) break;
345 out.line(`wrangler deploy failed; trying again in ${wait / 1000}s:\n${wranglerError(deployed.out)}`);
346 await new Promise((resolve) => setTimeout(resolve, wait));
347 deployed = await wrangler(args, { cwd, onLine: out.line });
348 }
349 if (deployed.code !== 0) throw new Error(`wrangler deploy failed:\n${wranglerError(deployed.out)}`);
Checkout tries a failed fetch twice more, after 2 s and 5 s350 result.version = dryRun ? "(dry run)" : versionFrom(deployed.out);
351 result.ok = true;
352 } catch (error) {
353 result.note = String(error.message ?? error);
354 } finally {
355 if (unit.image) removeDeployConfig(unit);
356 }
357 result.ms = Date.now() - started;
358 out.save();
359 if (!result.ok) result.note += `\n full log: ${out.file}`;
360 return result;
361}
362
363async function migrate(stack, units, migrations, opts) {
364 const due = units.filter((u) => migrations[u.id]?.pending?.length);
365 const broken = units.filter((u) => migrations[u.id]?.error);
366 if (broken.length) {
367 throw new Error(`Could not read pending migrations for ${broken.map((u) => u.id).join(", ")}; nothing was deployed.`);
368 }
369 if (!due.length) return [];
370 log(`== migrations: ${due.map((u) => `${u.d1.database} (${migrations[u.id].pending.length})`).join(", ")}`);
371 const results = await pool(due, opts.concurrency, async (unit) => {
372 const started = Date.now();
373 const out = unitLogger(`${unit.id}-migrations`);
374 const applied = await applyMigrations(unit, out.line);
375 out.save();
376 return {
377 unit: `${unit.id} (D1 ${unit.d1.database})`,
378 stage: "migrations",
379 ok: applied.code === 0,
380 version: `${migrations[unit.id].pending.length} applied`,
381 ms: Date.now() - started,
382 note: applied.code === 0 ? "" : `${lastLines(applied.out)}\n full log: ${out.file}`,
383 };
384 });
385 return results;
386}
387
388function summary(results) {
389 const rows = results.map((r) => [r.unit, r.stage, r.ok ? "ok" : r.skipped ? "not started" : "FAILED", r.version ?? "", r.ms ? seconds(r.ms) : "", r.note.split("\n")[0]]);
390 console.log(`\n${table(rows, ["unit", "stage", "result", "version", "time", "note"])}`);
391 for (const r of results.filter((r) => !r.ok && !r.skipped)) console.log(`\n${r.unit}: ${r.note}`);
392}
393
394async function deploy(stack, opts, { dryRun = false } = {}) {
395 const started = Date.now();
396 // A build reads nothing from Cloudflare: it builds what it is given.
397 const p = dryRun ? buildPlan(stack, opts) : await plan(stack, opts);
398 if (!dryRun) printPlan(stack, p);
399 const deploying = p.decisions.filter((d) => d.deploy);
400 const touched = deploying.map((d) => d.unit);
401 const head = p.head;
402
403 // A deploy names the commit it came from, so a dirty tree would be
404 // recorded as something it is not.
405 const dirtyFiles = git.dirty();
406 const dirty = deploying.some((d) => dirtyFiles.some((file) => file.startsWith(`${d.unit.path}/`) || d.unit.dependsOn.some((dir) => file.startsWith(`${dir}/`)) || d.unit.inputs.includes(file)));
407 if (dirty && !dryRun && !opts.allowDirty) {
408 throw new Error("Uncommitted changes touch what would deploy. Commit them, or pass --allow-dirty (the deploy then records no commit, and the next plan deploys it again).");
409 }
410
411 const results = [];
412 if (!dryRun && !opts.noMigrations) {
413 const migrated = await migrate(stack, p.units, p.migrations, opts);
414 results.push(...migrated);
415 if (migrated.some((r) => !r.ok)) {
416 summary(results);
417 return false;
418 }
419 }
420 if (!touched.length) {
421 if (results.length) summary(results);
422 return true;
423 }
424
425 if (touched.some((u) => u.kind === "rust-worker")) ensureWorkerBuild();
426 const docker = touched.some((u) => u.image) ? await dockerAvailable() : false;
427 const context = { head, subject: git.subject(), dirty, dryRun, docker, rebuildImage: opts.rebuildImage, rebuildBase: opts.rebuildBase };
Deploying: a deploy by hand shows on the repository's Deployments page428 // A deploy run by hand shows on the repository's Deployments page; a
429 // dirty tree is not a commit anyone can look at, so it is left out.
430 const settle = dryRun || dirty ? null : await reportDeployment({ head, subject: context.subject, units: touched.map((u) => u.id), log });
Checkout tries a failed fetch twice more, after 2 s and 5 s431
Merge status detection: first-byte speed probe, deploy windows, 4 of 5 with a re-check, check history, reminders432 // status.g1t.sh hears the deploy start and finish (STATUS_DEPLOY_TOKEN;
433 // nothing without it), so the restarts it causes are not drafted as
434 // incidents. Never in a dry run, and it never fails a deploy.
Checkout tries a failed fetch twice more, after 2 s and 5 s435 let failed = false;
Merge status detection: first-byte speed probe, deploy windows, 4 of 5 with a re-check, check history, reminders436 await withDeployWindow(
437 async () => {
438 for (const { stage, units } of byStage(stack, touched)) {
439 if (failed) {
440 for (const unit of units) results.push({ unit: unit.id, stage, ok: false, skipped: true, ms: 0, note: "an earlier stage failed" });
441 continue;
442 }
The artifacts service is services/artifacts, the Worker g1t-artifacts, bound as ARTIFACTS by the API, the site and the agents; its live rooms move to it with a Durable Object transfer from g1t-docs-service, and its database, bucket, indexes and queue keep their names. The git store's binding and settings are GITSTORE, its ops scripts gitstore-*, and workflow run artifacts keep their compatible API under run_artifacts modules. The deploy tool puts a Worker that has never deployed before the Workers in its stage that bind to it, and the deploy guide gives the cutover runbook.443 // A Worker that does not exist yet goes before those bound to it.
444 for (const wave of waves(units, (unit) => Boolean(deploying.find((d) => d.unit === unit)?.missing))) {
445 if (failed) {
446 for (const unit of wave) results.push({ unit: unit.id, stage, ok: false, skipped: true, ms: 0, note: "a Worker it binds to failed to deploy" });
447 continue;
448 }
449 log(`== ${stage}: ${wave.map((u) => u.id).join(", ")}`);
450 const shipped = await pool(wave, opts.concurrency, (unit) => ship(unit, deploying.find((d) => d.unit === unit), context));
451 results.push(...shipped);
452 failed = shipped.some((r) => !r.ok);
453 }
Merge status detection: first-byte speed probe, deploy windows, 4 of 5 with a re-check, check history, reminders454 }
455 },
456 { id: head ?? null, dryRun, log },
457 );
Checkout tries a failed fetch twice more, after 2 s and 5 s458 summary(results);
Deploying: a deploy by hand shows on the repository's Deployments page459 await settle?.(!failed);
Checkout tries a failed fetch twice more, after 2 s and 5 s460 console.log(`\n${failed ? "Failed" : dryRun ? "Built" : "Deployed"} in ${seconds(Date.now() - started)}.`);
461 return !failed;
462}
463
464async function doctor(stack, opts) {
465 const units = selected(stack, opts);
466 const rows = await pool(units, 8, async (unit) => {
467 if (!unit.secrets.length) return [unit.id, "", "", ""];
468 const found = await wrangler(["secret", "list", "--name", unit.worker, "--format", "json"], { cwd: join(ROOT, unit.path) });
469 if (found.code !== 0) return [unit.id, unit.secrets.join(" "), "?", "could not read"];
470 const have = new Set(jsonFrom(found.out).map((s) => s.name));
471 const missing = unit.secrets.filter((name) => !have.has(name));
472 return [unit.id, unit.secrets.join(" "), missing.join(" "), missing.length ? "missing" : "ok"];
473 });
474 console.log(table(rows, ["unit", "secrets", "missing", "result"]));
475 return rows.every((r) => r[3] !== "missing");
476}
477
478async function install(stack, opts) {
479 const units = selected(stack, opts);
480 const args = npmCiArgs(units, npmWorkspace());
481 log(`npm ${args.join(" ")}`);
482 const done = await exec(`npm ${args.join(" ")}`, [], { cwd: ROOT, shell: true, onLine: (line) => log(line) });
483 return done.code === 0;
484}
485
486function manifest(stack, opts) {
487 const found = problems(stack, findWranglerConfigs());
488 if (opts.check) {
489 for (const problem of found) console.log(`- ${problem}`);
490 console.log(found.length ? `\n${found.length} problems in deploy/stack.jsonc.` : "deploy/stack.jsonc is consistent with every wrangler.jsonc.");
491 return !found.length;
492 }
493 const out = stack.units.map(({ config, ...unit }) => ({
494 ...unit,
Merge branch 'worktree-agent-ad8a36dfcd4176015' into spend-guardrails495 queues: {
496 produces: (config?.queues?.producers ?? []).map((q) => q.queue),
497 consumes: (config?.queues?.consumers ?? []).map((q) => q.queue),
498 dead_letter: [...new Set((config?.queues?.consumers ?? []).map((q) => q.dead_letter_queue).filter(Boolean))],
499 },
Checkout tries a failed fetch twice more, after 2 s and 5 s500 kv: (config?.kv_namespaces ?? []).map((kv) => stack.resources.kv?.[kv.id] ?? kv.id),
501 r2: (config?.r2_buckets ?? []).map((b) => b.bucket_name),
502 vectorize: (config?.vectorize ?? []).map((v) => v.index_name),
503 dispatch_namespaces: (config?.dispatch_namespaces ?? []).map((d) => d.namespace),
504 routes: (config?.routes ?? []).map((r) => r.pattern),
505 }));
506 if (opts.json) console.log(JSON.stringify({ stages: stack.stages, units: out }, null, 2));
507 else
508 console.log(
509 table(
510 out.map((u) => [u.id, u.stage, u.kind, u.worker, u.d1?.database ?? "", u.dependsOn.join(" ")]),
511 ["unit", "stage", "kind", "worker", "d1", "built from (besides its folder)"],
512 ),
513 );
514 return true;
515}
516
517/** The unit with a Containers image (the runner), or the one named. */
518function imageUnit(stack, opts) {
519 const units = (opts.only.length ? pick(stack, opts.only) : stack.units).filter((u) => u.image?.base);
520 if (units.length !== 1) throw new Error("Name the unit with a Containers image: --only runner");
521 return units[0];
522}
523
524async function main() {
525 const opts = parseArgs(process.argv.slice(2));
526 const stack = resolvedStack();
527 switch (opts.command) {
528 case "plan": {
529 const p = await plan(stack, opts);
530 const data = planJson(stack, p.decisions, p.migrations, p.head);
531 if (opts.out) writeFileSync(opts.out, `${JSON.stringify(data)}\n`);
532 if (opts.githubOutput) writeGithubOutputs(data, p);
533 if (opts.json) console.log(JSON.stringify(data, null, 2));
534 else if (!opts.githubOutput || process.env.GITHUB_OUTPUT) printPlan(stack, p);
535 return true;
536 }
537 case "deploy":
538 return deploy(stack, opts);
539 case "build":
540 return deploy(stack, { ...opts, force: true }, { dryRun: true });
541 case "migrate": {
542 const units = selected(stack, opts);
543 const { migrations } = await survey(units.filter((u) => u.d1), { ...opts, noMigrations: false, noLive: true });
544 const results = await migrate(stack, units, migrations, opts);
545 if (results.length) summary(results);
546 else console.log("No migrations to apply.");
547 return results.every((r) => r.ok);
548 }
549 case "manifest":
550 return manifest(stack, opts);
551 case "doctor":
552 return doctor(stack, opts);
553 case "install":
554 return install(stack, opts);
555 case "build-base": {
556 const unit = imageUnit(stack, opts);
557 if (!(await dockerAvailable())) throw new Error("build-base needs Docker.");
558 const lock = await newBase(unit, { push: !opts.noPush, noCache: opts.noCache });
559 console.log(JSON.stringify(lock, null, 2));
560 if (lock.pushed) console.log(`\nCommit ${unit.image.base.lock}: the next deploy builds the runner's image on this base.`);
561 return true;
562 }
563 case "image": {
564 const unit = imageUnit(stack, opts);
565 const out = unitLogger(`${unit.id}-image`);
566 const docker = await dockerAvailable();
567 const image = await runnerImage(unit, { dryRun: Boolean(opts.noPush), docker, rebuildImage: opts.rebuildImage, rebuildBase: opts.rebuildBase }, out);
568 out.save();
569 console.log(`${image.ref}\n${image.note}`);
570 return true;
571 }
572 default:
573 console.error(USAGE);
574 return false;
575 }
576}
577
578main().then(
579 (ok) => process.exit(ok ? 0 : 1),
580 (error) => {
581 console.error(`\n${error.message ?? error}`);
582 process.exit(1);
583 },
584);

This file's history is long; its oldest lines are credited to the oldest commit read.