Skip to content
Sign up

flagon-io/g1t

public
0
367 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check1// Cloudflare, as the deploy tool uses it: reading which commit each Worker
2// runs and which D1 migrations are pending (Cloudflare's REST API when a
3// token is set, Wrangler otherwise), applying migrations, and deploying.
4// Every Wrangler call runs in the unit's own folder, so Wrangler reads that
5// unit's config (and not a .env at the repository root, which may hold a
6// token meant for something else).
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow7
8import { spawn } from "node:child_process";
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check9import { readdirSync } from "node:fs";
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow10import { join } from "node:path";
11
12import { ROOT } from "./stack.mjs";
13
14const WRANGLER = join(ROOT, "node_modules/wrangler/bin/wrangler.js");
15export const ACCOUNT_ID = "1e6f2cffa3f445920836e8ebe446bb58";
16
g1t-runner 0.1.0 is released: signed binaries for five platforms at g1t.sh/downloads/runner; the Artifacts checks' results17/**
18 * Headers for Cloudflare's REST and GraphQL APIs, for the ops scripts:
19 * CLOUDFLARE_API_TOKEN as a bearer token, or else a global API key
20 * (CLOUDFLARE_API_KEY with CLOUDFLARE_EMAIL). Null when neither is set.
21 */
22export function cloudflareAuth(env = process.env) {
23 if (env.CLOUDFLARE_API_TOKEN) return { authorization: `Bearer ${env.CLOUDFLARE_API_TOKEN}` };
24 if (env.CLOUDFLARE_API_KEY && env.CLOUDFLARE_EMAIL) {
25 return { "x-auth-key": env.CLOUDFLARE_API_KEY, "x-auth-email": env.CLOUDFLARE_EMAIL };
26 }
27 return null;
28}
29
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow30/** What a deploy's version message starts with, followed by the commit. */
31export const MESSAGE_PREFIX = "g1t-deploy";
32
33/**
34 * The environment Wrangler runs with. In CI (CI=true) it is the job's:
35 * CLOUDFLARE_API_TOKEN from the repository's secret. On a laptop it is
36 * your `wrangler login`, unless CLOUDFLARE_DEPLOY_TOKEN is set, as
37 * scripts/deploy.sh always did: a CLOUDFLARE_API_TOKEN or global API key
38 * in your shell is for other tools.
39 */
40export function wranglerEnv(base = process.env) {
41 const env = { ...base, WRANGLER_SEND_METRICS: "false", NO_COLOR: "1", FORCE_COLOR: "0" };
42 env.CLOUDFLARE_ACCOUNT_ID ||= ACCOUNT_ID;
43 if (base.CLOUDFLARE_DEPLOY_TOKEN) {
44 env.CLOUDFLARE_API_TOKEN = base.CLOUDFLARE_DEPLOY_TOKEN;
45 } else if (base.CI !== "true") {
46 env.CLOUDFLARE_API_TOKEN = "";
47 delete env.CLOUDFLARE_API_KEY;
48 delete env.CLOUDFLARE_EMAIL;
49 }
50 return env;
51}
52
53/**
54 * Runs a command; resolves with { code, out } (stdout and stderr together,
Fast pages, required checks on the branch, self-hosted runners, honest incidents55 * in order). `onLine` sees each line as it comes; `input` is written to
56 * its stdin.
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow57 */
Fast pages, required checks on the branch, self-hosted runners, honest incidents58export function exec(command, args, { cwd = ROOT, env = process.env, onLine, shell = false, input } = {}) {
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow59 return new Promise((resolve) => {
60 const child = spawn(command, args, { cwd, env, shell, windowsHide: true });
Fast pages, required checks on the branch, self-hosted runners, honest incidents61 if (input !== undefined) child.stdin.end(input);
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow62 let out = "";
63 let partial = "";
64 const take = (chunk) => {
65 const text = chunk.toString();
66 out += text;
67 if (!onLine) return;
68 const lines = (partial + text).split(/\r?\n/);
69 partial = lines.pop();
70 for (const line of lines) onLine(line);
71 };
72 child.stdout.on("data", take);
73 child.stderr.on("data", take);
74 child.on("error", (error) => resolve({ code: 127, out: `${out}${error.message}\n` }));
75 child.on("close", (code) => {
76 if (onLine && partial) onLine(partial);
77 resolve({ code: code ?? 1, out });
78 });
79 });
80}
81
82/** Runs the repository's own Wrangler in `cwd`. */
83export function wrangler(args, { cwd, env = wranglerEnv(), onLine } = {}) {
84 return exec(process.execPath, [WRANGLER, ...args], { cwd, env, onLine });
85}
86
87/** The first JSON value in Wrangler's output (it may print notices first). */
88export function jsonFrom(out) {
89 const start = out.search(/^[[{]/m);
90 if (start < 0) throw new Error(`no JSON in: ${out.slice(0, 300)}`);
91 return JSON.parse(out.slice(start));
92}
93
94/** The message and tag a deploy of `sha` is annotated with. */
95export function annotation(sha, subject = "") {
96 const message = `${MESSAGE_PREFIX} ${sha} ${subject}`.trim().slice(0, 100);
97 return { message, tag: `g1t-${sha.slice(0, 12)}` };
98}
99
100/** The commit a version message names, or null. Dirty deploys name none. */
101export function commitFrom(message) {
102 const match = new RegExp(`^${MESSAGE_PREFIX} ([0-9a-f]{40})(?:\\s|$)`).exec(message ?? "");
103 return match ? match[1] : null;
104}
105
106/**
107 * Which commit a Worker's live version was deployed from, given Wrangler's
108 * `deployments status --json` and `versions list --json`. A version made by
109 * `wrangler secret put` keeps the code of the one before it, so those are
110 * looked through. Anything else without our message (a deploy by hand, a
111 * dashboard edit) leaves the commit unknown, and the unit is deployed again.
112 */
113export function liveCommit(status, versions) {
114 const live = [...(status.versions ?? [])].sort((a, b) => b.percentage - a.percentage);
115 if (!live.length) return { sha: null, why: "no live version" };
116 const split = live.length > 1 && live[1].percentage > 0;
117 const byNumber = [...versions].sort((a, b) => b.number - a.number);
118 let index = byNumber.findIndex((v) => v.id === live[0].version_id);
119 if (index < 0) return { sha: null, why: "its live version is not among the recent ones", version: live[0].version_id };
120 const version = byNumber[index];
121 while (index < byNumber.length) {
122 const candidate = byNumber[index];
123 const sha = commitFrom(candidate.annotations?.["workers/message"]);
124 if (sha) {
125 return {
126 sha,
127 version: version.id,
128 at: candidate.metadata?.created_on ?? null,
129 by: candidate.metadata?.author_email ?? null,
130 split,
131 why: split ? "a gradual deployment is in progress; its main version is used" : null,
132 };
133 }
134 if (candidate.annotations?.["workers/triggered_by"] !== "secret") break;
135 index++;
136 }
137 return { sha: null, version: version.id, why: "its live version was not deployed by scripts/deploy.mjs" };
138}
139
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check140// ── Reading production ───────────────────────────────────────────────────
141//
142// The plan reads Cloudflare's REST API directly when it has a token (always
143// in CI): one request per question, all in parallel, where a Wrangler
144// process would boot Node and check its login before each. Without one (a
145// laptop's `wrangler login`) it asks Wrangler, as it always did. The
146// requests are the ones Wrangler makes: `deployments status` prints the
147// first of GET .../deployments, `versions list` the items of GET
148// .../versions?deployable=true, and `d1 migrations list` compares the names
149// in the database's d1_migrations table with the files in its migrations
150// folder.
151
152const API = "https://api.cloudflare.com/client/v4";
153/** At most this many requests to Cloudflare at once. */
154export const API_CONCURRENCY = 16;
155
156/**
157 * The token and account the plan reads Cloudflare's API with: the token
158 * Wrangler would be given (see wranglerEnv), or null, and then Wrangler is
159 * asked instead, with your `wrangler login`.
160 */
161export function apiAuth(base = process.env) {
162 const env = wranglerEnv(base);
163 if (!env.CLOUDFLARE_API_TOKEN) return null;
164 return { token: env.CLOUDFLARE_API_TOKEN, account: env.CLOUDFLARE_ACCOUNT_ID };
165}
166
167let inFlight = 0;
168const waiting = [];
169async function limited(task) {
170 while (inFlight >= API_CONCURRENCY) await new Promise((resolve) => waiting.push(resolve));
171 inFlight++;
172 try {
173 return await task();
174 } finally {
175 inFlight--;
176 waiting.shift()?.();
177 }
178}
179
180/** Cloudflare's errors in an answer, as one line. */
181export function apiError(status, body) {
182 const errors = (body?.errors ?? []).map((e) => (e.code ? `${e.message} (${e.code})` : e.message)).filter(Boolean);
183 return `Cloudflare API ${status || "request"} failed${errors.length ? `: ${errors.join("; ")}` : ""}`;
184}
185
186/**
187 * One request to Cloudflare's API. Resolves with { status, result } or
188 * { status, error, codes }; never throws. Retried once after a refusal
189 * that may pass (a 403 while a token propagates, 429, 5xx, the network).
190 */
191export async function cloudflareApi(auth, path, { method = "GET", body, fetchImpl = fetch, retries = 1, timeoutMs = 30_000 } = {}) {
192 for (let attempt = 0; ; attempt++) {
193 let status = 0;
194 let data = null;
195 let error = null;
196 try {
197 const response = await limited(async () => {
198 const res = await fetchImpl(`${API}${path}`, {
199 method,
200 headers: { authorization: `Bearer ${auth.token}`, ...(body === undefined ? {} : { "content-type": "application/json" }) },
201 body: body === undefined ? undefined : JSON.stringify(body),
202 signal: AbortSignal.timeout(timeoutMs),
203 });
204 return { status: res.status, text: await res.text() };
205 });
206 status = response.status;
207 try {
208 data = JSON.parse(response.text);
209 } catch {
210 error = `Cloudflare API ${status}: ${response.text.trim().slice(0, 300) || "an empty answer"}`;
211 }
212 } catch (failure) {
213 error = `Cloudflare API request failed: ${failure?.message ?? failure}`;
214 }
215 if (!error && status < 400 && data?.success !== false) return { status, result: data?.result ?? null };
216 error ??= apiError(status, data);
217 const codes = (data?.errors ?? []).map((e) => e.code);
218 const mayPass = status === 0 || status === 403 || status === 429 || status >= 500;
219 if (attempt >= retries || !mayPass) return { status, error, codes };
220 await new Promise((resolve) => setTimeout(resolve, 500 * (attempt + 1)));
221 }
222}
223
224/** Cloudflare's code for a Worker that does not exist. */
225const SCRIPT_NOT_FOUND = 10007;
226
227/**
228 * The live commit from the API's answers: `deployments` is the result of
229 * GET .../deployments ({ deployments: [newest first] }), `versions` that of
230 * GET .../versions?deployable=true ({ items }), or null if it could not be
231 * read. `wrangler deployments status --json` prints the first deployment,
232 * and `versions list --json` those items.
233 */
234export function liveFromApi(worker, deployments, versions) {
235 const latest = deployments?.deployments?.[0];
236 if (!latest) return { sha: null, error: `The Worker ${worker} has no deployments.` };
237 return liveCommit(latest, versions?.items ?? []);
238}
239
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow240/** Reads the commit a unit's Worker runs. Never throws. */
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check241export async function readLive(unit, { auth = apiAuth(), fetchImpl = fetch } = {}) {
242 if (!auth) return readLiveWithWrangler(unit);
243 const script = `/accounts/${auth.account}/workers/scripts/${encodeURIComponent(unit.worker)}`;
244 const [deployments, versions] = await Promise.all([
245 cloudflareApi(auth, `${script}/deployments`, { fetchImpl }),
246 cloudflareApi(auth, `${script}/versions?deployable=true`, { fetchImpl }),
247 ]);
248 if (deployments.error) {
249 if (deployments.status === 404 || deployments.codes.includes(SCRIPT_NOT_FOUND)) return { sha: null, missing: true, why: "never deployed" };
250 return { sha: null, error: deployments.error };
251 }
252 try {
253 return liveFromApi(unit.worker, deployments.result, versions.error ? null : versions.result);
254 } catch (error) {
255 return { sha: null, error: String(error.message ?? error) };
256 }
257}
258
259/** readLive through Wrangler: `deployments status` and `versions list`. */
260async function readLiveWithWrangler(unit) {
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow261 const cwd = join(ROOT, unit.path);
262 const [status, versions] = await Promise.all([
263 wrangler(["deployments", "status", "--name", unit.worker, "--json"], { cwd }),
264 wrangler(["versions", "list", "--name", unit.worker, "--json"], { cwd }),
265 ]);
266 if (status.code !== 0) {
267 if (/not found|does not exist|10007/i.test(status.out)) return { sha: null, missing: true, why: "never deployed" };
268 return { sha: null, error: lastLines(status.out) };
269 }
270 try {
271 return liveCommit(jsonFrom(status.out), versions.code === 0 ? jsonFrom(versions.out) : []);
272 } catch (error) {
273 return { sha: null, error: String(error.message ?? error) };
274 }
275}
276
277/** Migration files Wrangler lists as not yet applied. */
278export function pendingFrom(out) {
279 if (/No migrations to apply/i.test(out)) return [];
280 const names = [...out.matchAll(/([\w.-]+\.sql)\b/g)].map((m) => m[1]);
281 return [...new Set(names)];
282}
283
Merge a faster deploy plan: Cloudflare's API read directly and in parallel (2 s against 4 min), and the plan runs beside the check284/** Wrangler's default name for the table of applied migrations. */
285export const MIGRATIONS_TABLE = "d1_migrations";
286
287/**
288 * A unit's database as its wrangler.jsonc names it: { id, table, dir }
289 * (dir relative to the repository), or null.
290 */
291export function databaseOf(unit) {
292 const db = (unit.config?.d1_databases ?? []).find((d) => d.database_name === unit.d1?.database);
293 if (!db?.database_id) return null;
294 return { id: db.database_id, table: db.migrations_table || MIGRATIONS_TABLE, dir: join(unit.path, unit.d1.migrations) };
295}
296
297/** The migration files in a folder, as Wrangler finds them: its *.sql files. */
298export function migrationFiles(dir) {
299 return readdirSync(dir, { withFileTypes: true })
300 .filter((entry) => entry.isFile() && entry.name.endsWith(".sql"))
301 .map((entry) => entry.name)
302 .sort();
303}
304
305/**
306 * The files not yet applied, in the files' order, given the D1 query API's
307 * result for `SELECT name FROM d1_migrations` ([{ results: [{ name }] }]).
308 */
309export function pendingAgainst(files, result) {
310 const applied = new Set((result?.[0]?.results ?? []).map((row) => row.name));
311 return files.filter((file) => !applied.has(file));
312}
313
314const quoteIdentifier = (name) => `"${name.replaceAll('"', '""')}"`;
315
316/** Pending migrations of a unit's database: { pending } or { error }. Never throws. */
317export async function pendingMigrations(unit, { auth = apiAuth(), fetchImpl = fetch, root = ROOT } = {}) {
318 const db = databaseOf(unit);
319 if (!auth || !db) return pendingMigrationsWithWrangler(unit);
320 let files;
321 try {
322 files = migrationFiles(join(root, db.dir));
323 } catch (error) {
324 return { error: `Could not read ${db.dir}: ${error.message ?? error}` };
325 }
326 const answer = await cloudflareApi(auth, `/accounts/${auth.account}/d1/database/${db.id}/query`, {
327 method: "POST",
328 body: { sql: `SELECT name FROM ${quoteIdentifier(db.table)} ORDER BY id` },
329 fetchImpl,
330 });
331 if (answer.error) {
332 // A database no migration was ever applied to has no table yet.
333 if (/no such table/i.test(answer.error)) return { pending: files };
334 return { error: answer.error };
335 }
336 return { pending: pendingAgainst(files, answer.result) };
337}
338
339/** pendingMigrations through Wrangler: `d1 migrations list --remote`. */
340async function pendingMigrationsWithWrangler(unit) {
Money in billing's messages reads to the cent, and the deploy reads migrations again after a failure341 const list = () => wrangler(["d1", "migrations", "list", unit.d1.database, "--remote"], { cwd: join(ROOT, unit.path) });
342 // Once more after a failure: Cloudflare's API sometimes answers 403
343 // while Wrangler's login refreshes (seen on 2026-10-06).
344 let found = await list();
345 if (found.code !== 0) found = await list();
Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow346 if (found.code !== 0) return { error: lastLines(found.out) };
347 return { pending: pendingFrom(found.out) };
348}
349
350export function applyMigrations(unit, onLine) {
351 return wrangler(["d1", "migrations", "apply", unit.d1.database, "--remote"], { cwd: join(ROOT, unit.path), onLine });
352}
353
354/** The version a deploy made, from Wrangler's output. */
355export function versionFrom(out) {
356 return /Current Version ID:\s*([0-9a-f-]{36})/i.exec(out)?.[1] ?? null;
357}
358
359/** Whether Docker can build here (for a Containers image). */
360export async function dockerAvailable() {
361 const found = await exec("docker", ["info", "--format", "{{.ServerVersion}}"]);
362 return found.code === 0;
363}
364
365export function lastLines(text, count = 12) {
366 return text.trim().split(/\r?\n/).slice(-count).join("\n");
367}

This file's history is long; its oldest lines are credited to the oldest commit read.