Skip to content
Sign up

flagon-io/g1t

public
00

Where people and agents ship software together. The open-source git platform for the whole job: issues, agents, checks and deploys to the edge.

gitagentscloudflarerustforgemcp

Commit

Keep frontend assets available across deployments and gate cutovers

syntaqxcommitted Parentf3f2211Browse files
18 files+577−90/18 viewed
+37−0
6969
7070 ## The tool
7171
72+Web releases retain fingerprinted JavaScript, CSS and fonts in the existing
73+`DOWNLOADS` R2 bucket under `web-assets/assets/`. Before switching the Worker,
74+the deploy tool ensures the currently live site's asset graph and the new
75+build's assets are archived. Release manifests under `web-assets/manifests/`
76+let later deploys skip unchanged files without scanning deployment history or
77+fetching the entire live build again. Provider rate limits retry with a bound;
78+an archive failure stops the web cutover. A confirmed first deployment skips
79+the live-site copy. The deploy credential needs object read and write access
80+to this bucket.
81+
82+Every web build also runs the compiled Worker locally in workerd. The gate
83+checks old CSS through R2, missing assets, and old-client navigation with an
84+existing cached response before any backend loaders run. Upload receipts
85+must match the local asset checksums before cutover is allowed.
86+
87+Current files still use Workers Static Assets. Requests for older filenames
88+fall back to R2, so an open tab can finish loading after a release. Archived
89+files are immutable and retained without automatic deletion; do not apply an
90+expiration lifecycle rule to this prefix. Assets already removed before the
91+first protected deployment cannot be recovered by this archive.
92+
93+The public HTML/data cache is separated by build. On navigation, a tab from
94+an older build gets a document navigation to its destination before route
95+loaders run. Background refreshes do not force a reload, and writes are not
96+replayed. Service APIs must remain compatible with existing clients during
97+rollout. Use the deploy tool or the web package's `npm run deploy`; invoking
98+`wrangler deploy` directly bypasses the asset archive step.
99+
100+After cutover, the tool checks the live build identifier and the bytes of
101+both old and new stylesheets, client entries and route manifests. A failed
102+check marks the deploy failed; it does not automatically roll back service
103+code or database migrations. The production workflow serializes deploys
104+without cancelling an active release. Manual deploys must not race that
105+workflow. Roll back only to web versions that include the archive fallback;
106+older versions would remove that protection. Database changes must be
107+additive during rollout, with removals deferred until old callers are gone.
108+
72109 ```sh
73110 node scripts/deploy.mjs plan # what would deploy, and why (read-only)
74111 node scripts/deploy.mjs deploy # migrations, then every changed unit
+5−2
1−import { useEffect, useState } from "react";
2−import { useNavigation } from "react-router";
1+import { useContext, useEffect, useState } from "react";
2+import { UNSAFE_DataRouterContext, useNavigation } from "react-router";
33
44 import { stuckTarget } from "../lib/stuck";
5+import { watchBuildNavigation } from "../lib/resilient-fetch";
56 import { Button } from "./ui/button";
67
78 /** A document load of the destination, while the router still shows the old page. */
89 export function NavigationRecovery() {
10+ const context = useContext(UNSAFE_DataRouterContext);
11+ useEffect(() => watchBuildNavigation(() => context?.router.state.navigation.location), [context?.router]);
912 const navigation = useNavigation();
1013 const target = stuckTarget(navigation);
1114 const key = navigation.location?.key;
+1−1
77 import { installResilientFetch } from "./lib/resilient-fetch";
88
99 // The router's reads outlast a moment offline (lib/resilient-fetch.ts).
10−installResilientFetch();
10+installResilientFetch(import.meta.env.G1T_BUILD_ID);
1111
1212 startTransition(() => {
1313 hydrateRoot(
+33−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+import { archivedAssetKey, serveArchivedAsset } from "./asset-archive.ts";
4+
5+test("archive keys cannot read releases, source maps or private objects", () => {
6+ assert.equal(archivedAssetKey("/assets/app-old12345.css"), "web-assets/assets/app-old12345.css");
7+ for (const path of ["/downloads/runner", "/assets/../secret.js", "/assets/a.js.map", "/assets/%2e%2e/secret.js", "/assets/nested/a.js"]) assert.equal(archivedAssetKey(path), null);
8+});
9+
10+test("a stylesheet removed from the new build stays available with its MIME type", async () => {
11+ const bucket = { get: async (key: string) => {
12+ assert.equal(key, "web-assets/assets/app-old12345.css");
13+ return { body: "body{color:red}", size: 15, httpEtag: '"old"', writeHttpMetadata: (headers: Headers) => headers.set("content-type", "text/css") };
14+ } } as unknown as R2Bucket;
15+ const url = "https://g1t.sh/assets/app-old12345.css";
16+ const response = await serveArchivedAsset(new Request(url), bucket);
17+ assert.equal(response.status, 200);
18+ assert.equal(response.headers.get("content-type"), "text/css");
19+ assert.match(response.headers.get("cache-control")!, /immutable/);
20+ assert.equal(await response.text(), "body{color:red}");
21+ assert.equal(await (await serveArchivedAsset(new Request(url, { method: "HEAD" }), bucket)).text(), "");
22+ assert.equal((await serveArchivedAsset(new Request(url, { headers: { "if-none-match": '"old"' } }), bucket)).status, 304);
23+});
24+
25+test("missing or unavailable assets never become HTML or cacheable errors", async () => {
26+ const url = "https://g1t.sh/assets/missing-12345678.js";
27+ const missing = await serveArchivedAsset(new Request(url), { get: async () => null } as unknown as R2Bucket);
28+ assert.equal(missing.status, 404);
29+ assert.equal(missing.headers.get("cache-control"), "no-store");
30+ const down = await serveArchivedAsset(new Request(url), undefined);
31+ assert.equal(down.status, 503);
32+ assert.equal(down.headers.get("cache-control"), "no-store");
33+});
+26−0
1+/** Only built public assets can be read from the release bucket. */
2+export function archivedAssetKey(pathname: string): string | null {
3+ if (!/^\/assets\/[a-zA-Z0-9_.~-]+\.(?:js|css|woff2?|ttf|otf|png|jpe?g|gif|svg|webp|ico|wasm)$/.test(pathname)) return null;
4+ return `web-assets${pathname}`;
5+}
6+
7+/** Current assets are served by Cloudflare before the Worker; misses land here. */
8+export async function serveArchivedAsset(request: Request, bucket: Pick<R2Bucket, "get"> | undefined): Promise<Response> {
9+ const key = archivedAssetKey(new URL(request.url).pathname);
10+ if (!key || !["GET", "HEAD"].includes(request.method)) return new Response("Not found", { status: 404, headers: { "cache-control": "no-store" } });
11+ try {
12+ if (!bucket) throw new Error("Asset archive is not configured");
13+ const object = await bucket.get(key);
14+ if (!object) return new Response("Not found", { status: 404, headers: { "cache-control": "no-store" } });
15+ const headers = new Headers();
16+ object.writeHttpMetadata(headers);
17+ headers.set("etag", object.httpEtag);
18+ headers.set("cache-control", "public, max-age=31536000, immutable");
19+ headers.set("x-content-type-options", "nosniff");
20+ if (request.headers.get("if-none-match") === object.httpEtag) return new Response(null, { status: 304, headers });
21+ headers.set("content-length", String(object.size));
22+ return new Response(request.method === "HEAD" ? null : object.body, { headers });
23+ } catch {
24+ return new Response("Asset temporarily unavailable", { status: 503, headers: { "cache-control": "no-store", "retry-after": "1" } });
25+ }
26+}
+12−0
1+import { redirectDocument, type MiddlewareFunction } from "react-router";
2+import { needsBuildRefresh } from "./build-version";
3+import { pageOf } from "./confirm-gate";
4+
5+export const buildVersionMiddleware: MiddlewareFunction<Response> = async ({ request }, next) => {
6+ if (needsBuildRefresh(request, import.meta.env.G1T_BUILD_ID)) {
7+ const url = new URL(request.url);
8+ url.searchParams.delete("_routes");
9+ throw redirectDocument(pageOf(url.pathname) + url.search, { headers: { "cache-control": "no-store" } });
10+ }
11+ return next();
12+};
+12−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+import { BUILD_HEADER, needsBuildRefresh } from "./build-version.ts";
4+
5+test("old navigation reads switch builds before loaders; writes and ordinary documents do not", () => {
6+ const request = (path: string, method = "GET", build = "old") => new Request(`https://g1t.sh${path}`, { method, headers: { [BUILD_HEADER]: build } });
7+ assert.equal(needsBuildRefresh(request("/flagon-io/-/home.data"), "new"), true);
8+ assert.equal(needsBuildRefresh(request("/flagon-io/-/home.data", "GET", "new"), "new"), false);
9+ assert.equal(needsBuildRefresh(request("/flagon-io/-/home.data", "POST"), "new"), false);
10+ assert.equal(needsBuildRefresh(request("/flagon-io/-/home"), "new"), false);
11+ assert.equal(needsBuildRefresh(new Request("https://g1t.sh/a.data"), "new"), false);
12+});
+7−0
1+export const BUILD_HEADER = "x-g1t-build";
2+
3+/** Only safe route reads can switch builds automatically; never replay writes. */
4+export function needsBuildRefresh(request: Request, build: string): boolean {
5+ const requested = request.headers.get(BUILD_HEADER);
6+ return request.method === "GET" && new URL(request.url).pathname.endsWith(".data") && requested !== null && requested !== build;
7+}
+25−1
11 import assert from "node:assert/strict";
22 import test from "node:test";
33
4−import { PATIENCE_MS, REFRESH_PATIENCE_MS, isNetworkFailure, keepTrying, refreshesCurrent, retriable, retryDelay } from "./resilient-fetch.ts";
4+import { PATIENCE_MS, REFRESH_PATIENCE_MS, installResilientFetch, isNetworkFailure, keepTrying, refreshesCurrent, retriable, retryDelay, watchBuildNavigation } from "./resilient-fetch.ts";
55 import { reconnectDelay } from "./reconnect.ts";
66
77 const ORIGIN = "https://g1t.sh";
88 const url = (path: string) => new URL(path, ORIGIN);
99
10+test("only an active navigation carries its build; fetchers, refreshes and writes never force a document switch", async () => {
11+ const calls: Headers[] = [];
12+ const previous = Object.getOwnPropertyDescriptor(globalThis, "window");
13+ const browser = { location: new URL("https://g1t.sh/edit"), fetch: async (_input: unknown, init?: RequestInit) => { calls.push(new Headers(init?.headers)); return new Response(null); } };
14+ Object.defineProperty(globalThis, "window", { value: browser, configurable: true });
15+ let destination: { pathname: string; search: string } | undefined;
16+ const stop = watchBuildNavigation(() => destination);
17+ try {
18+ installResilientFetch("old-build");
19+ await browser.fetch("/notifications.data");
20+ destination = { pathname: "/next", search: "?tab=one" };
21+ await browser.fetch("/next.data?tab=one&_routes=root");
22+ await browser.fetch("/unrelated.data");
23+ await browser.fetch("/next.data?tab=one", { method: "POST" });
24+ await browser.fetch("/edit.data");
25+ await browser.fetch("https://external.example/next.data?tab=one");
26+ assert.deepEqual(calls.map((headers) => headers.get("x-g1t-build")), [null, "old-build", null, null, null, null]);
27+ } finally {
28+ stop();
29+ if (previous) Object.defineProperty(globalThis, "window", previous);
30+ else Reflect.deleteProperty(globalThis, "window");
31+ }
32+});
33+
1034 test("a request that got no answer, in each browser's words, is a network failure", () => {
1135 assert.equal(isNetworkFailure(new TypeError("Failed to fetch")), true);
1236 assert.equal(isNetworkFailure(new TypeError("NetworkError when attempting to fetch resource.")), true);
+19−2
2222 * sending it twice could do it twice.
2323 */
2424 import { useSyncExternalStore } from "react";
25+import { BUILD_HEADER } from "./build-version.ts";
2526
2627 /** How long a read keeps trying; under lib/stuck.ts's `STUCK_NAVIGATION_MS`, so a click still lands on an error page that recovers rather than a whole-page load that cannot. */
2728 export const PATIENCE_MS = 45_000;
156157 }
157158
158159 let installed = false;
160+type NavigationLocation = { pathname: string; search: string };
161+let navigationLocation: () => NavigationLocation | undefined = () => undefined;
159162
163+/** Read router state synchronously: React effects run after a navigation's fetch starts. */
164+export function watchBuildNavigation(read: () => NavigationLocation | undefined) {
165+ navigationLocation = read;
166+ return () => { if (navigationLocation === read) navigationLocation = () => undefined; };
167+}
168+
160169 /** Wraps the browser's `fetch` so the router's reads survive a moment offline. Browser only; once. */
161−export function installResilientFetch() {
170+export function installResilientFetch(build?: string) {
162171 if (installed || typeof window === "undefined") return;
163172 installed = true;
164173 const original = window.fetch.bind(window);
172181 }
173182 const method = init?.method ?? request?.method ?? "GET";
174183 if (!retriable(url, method, window.location.origin)) return original(input, init);
184+ const refresh = refreshesCurrent(url, window.location);
185+ // Switch builds only on a navigation, never from a background refresh
186+ // that could interrupt someone editing the current page.
187+ const destination = navigationLocation();
188+ if (build && !refresh && destination && refreshesCurrent(url, destination)) {
189+ const headers = new Headers(init?.headers ?? request?.headers);
190+ headers.set(BUILD_HEADER, build);
191+ init = { ...init, headers };
192+ }
175193 const signal = init?.signal ?? request?.signal;
176− const refresh = refreshesCurrent(url, window.location);
177194 const started = Date.now();
178195 let onlineMs = 0;
179196 let counted = false;
+2−1
100100 import { NavigationRecovery } from "./components/navigation-recovery";
101101 import { isNetworkFailure } from "./lib/resilient-fetch";
102102 import { useReconnect } from "./lib/reconnect";
103+import { buildVersionMiddleware } from "./lib/build-version.server";
103104
104105 export const links: Route.LinksFunction = () => [
105106 { rel: "icon", href: "/favicon.ico", sizes: "32x32" },
130131 },
131132 ];
132133
133−export const middleware: Route.MiddlewareFunction[] = [viewerMiddleware];
134+export const middleware: Route.MiddlewareFunction[] = [buildVersionMiddleware, viewerMiddleware];
134135
135136 export async function loader({ context, params, request }: Route.LoaderArgs) {
136137 const user = getViewer(context);
+2−2
33 "private": true,
44 "type": "module",
55 "scripts": {
6− "build": "react-router build && node scripts/load-check.mjs",
6+ "build": "react-router build && node scripts/load-check.mjs && node scripts/deploy-check.mjs",
77 "test": "node --test app/**/*.test.ts",
88 "dev": "react-router dev",
99 "typecheck": "wrangler types --include-env=false && react-router typegen && tsc -b --force",
10− "deploy": "npm run build && wrangler deploy",
10+ "deploy": "npm run build && node ../../scripts/deploy/web-assets.mjs --deploy",
1111 "preview": "npm run build && vite preview",
1212 "cf-typegen": "wrangler types --include-env=false",
1313 "postinstall": "wrangler types --include-env=false"
+58−0
1+// Exercise the built Worker in workerd before it can be deployed. Storage is
2+// temporary and local; no service bindings or Cloudflare credentials are used.
3+import assert from "node:assert/strict";
4+import { readFileSync, readdirSync } from "node:fs";
5+import { join, resolve } from "node:path";
6+import { Miniflare, convertV4MiniflareOptions } from "miniflare";
7+
8+const root = resolve("build/server");
9+const config = JSON.parse(readFileSync(join(root, "wrangler.json"), "utf8"));
10+const source = readFileSync(join(root, "index.js"), "utf8");
11+const build = source.match(/searchParams\.set\("_g1t_build", "([^"]+)"\)/)?.[1];
12+assert.ok(build, "The page cache must be separated by build");
13+const css = "body{color:red}";
14+// Seed R2 and the page cache through a test-only entry, keeping all R2 I/O
15+// inside workerd. The application entry itself is the unmodified build.
16+const seed = `import app from './index.js';
17+export default { async fetch(request, env, ctx) {
18+ if (new URL(request.url).pathname === '/__seed') {
19+ await env.DOWNLOADS.put('web-assets/assets/app-old12345.css', ${JSON.stringify(css)}, {httpMetadata:{contentType:'text/css'}});
20+ const cache = new URL('/pricing.data', request.url);
21+ cache.searchParams.set('_g1t_build', ${JSON.stringify(build)});
22+ await caches.default.put(new Request(cache), new Response('wrong-build-cache', {headers:{'cache-control':'public, max-age=300','x-g1t-kept-at':String(Date.now())}}));
23+ return new Response('seeded');
24+ }
25+ return app.fetch(request, env, ctx);
26+}};`;
27+const files = [join(root, "index.js"), ...readdirSync(join(root, "assets")).filter((name) => name.endsWith(".js")).map((name) => join(root, "assets", name))];
28+const runtime = new Miniflare(convertV4MiniflareOptions({
29+ name: "web-deploy-check", cf: false, modulesRoot: root,
30+ modules: [{ type: "ESModule", path: join(root, "deploy-check.mjs"), contents: seed }, ...files.map((path) => ({ type: "ESModule", path }))],
31+ compatibilityDate: config.compatibility_date,
32+ compatibilityFlags: config.compatibility_flags,
33+ r2Buckets: ["DOWNLOADS"],
34+}));
35+try {
36+ const origin = await runtime.ready;
37+ const request = (path, init) => fetch(new URL(path, origin), { ...init, signal: AbortSignal.timeout(15_000) });
38+ assert.equal((await request("/__seed")).status, 200);
39+ const old = await request("/assets/app-old12345.css");
40+ assert.equal(old.status, 200);
41+ assert.equal(old.headers.get("content-type"), "text/css");
42+ assert.equal(await old.text(), css);
43+ const missing = await request("/assets/app-missing0.css");
44+ assert.equal(missing.status, 404);
45+ assert.equal(missing.headers.get("cache-control"), "no-store");
46+ assert.doesNotMatch(missing.headers.get("content-type") ?? "", /html/);
47+ for (const path of ["/pricing.data", "/flagon-io/-/home.data?tab=one&_routes=root"]) {
48+ const answer = await request(path, { headers: { "x-g1t-build": "previous-build" } });
49+ assert.equal(answer.status, 202);
50+ const body = await answer.text();
51+ assert.match(body, /reload/);
52+ assert.ok(body.includes(path.split(".data")[0]));
53+ assert.doesNotMatch(body, /_routes|wrong-build-cache/);
54+ }
55+ console.log("Deployment safety: old CSS survives, missing assets stay isolated, old navigations bypass cache and loaders.");
56+} finally {
57+ await runtime.dispose();
58+}
+2−0
22 import { cloudflare } from "@cloudflare/vite-plugin";
33 import tailwindcss from "@tailwindcss/vite";
44 import { defineConfig, type Plugin } from "vite";
5+import { randomUUID } from "node:crypto";
56
67 /**
78 * The doc editor's chunk and everything it imports, so a doc's page can
4445 }
4546
4647 export default defineConfig({
48+ define: { "import.meta.env.G1T_BUILD_ID": JSON.stringify(randomUUID()) },
4749 plugins: [
4850 cloudflare({ viteEnvironment: { name: "ssr" } }),
4951 tailwindcss(),
+8−0
1616 import { usercontentPath } from "../app/lib/usercontent";
1717 import { TOKEN_CHALLENGE } from "../app/lib/website-token";
1818 import { serveUsercontent } from "./usercontent";
19+import { serveArchivedAsset } from "../app/lib/asset-archive";
20+import { needsBuildRefresh } from "../app/lib/build-version";
1921
2022 const loadBuild = () => import("virtual:react-router/server-build");
2123
7779
7880 async function site(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
7981 const { pathname } = new URL(request.url);
82+ // An old tab's chunk or stylesheet must not turn into an HTML route/404.
83+ if (pathname.startsWith("/assets/")) return serveArchivedAsset(request, env.DOWNLOADS);
8084 // Git over HTTPS shares this hostname but belongs to the repos service.
8185 // Its answer goes back to the git client as it is: a repository under a
8286 // renamed workspace's old name answers with a 301, which git follows and
142146
143147 function anonymousPage(request: Request, pathname: string): boolean {
144148 if (request.method !== "GET") return false;
149+ // A cached data response must not bypass the build-switch middleware.
150+ if (needsBuildRefresh(request, import.meta.env.G1T_BUILD_ID)) return false;
145151 if (/(?:^|;\s*)g1t_session=/.test(request.headers.get("cookie") ?? "")) return false;
146152 // A token signs the request in (app/lib/website-token.ts): never kept, never served a kept page.
147153 if (request.headers.has("authorization")) return false;
152158 const cache = (caches as unknown as { default: Cache }).default;
153159 // Visitors asked about analytics first get a page that says so, kept apart.
154160 const keyUrl = new URL(request.url);
161+ // Cached HTML and route data belong to the build that rendered them.
162+ keyUrl.searchParams.set("_g1t_build", import.meta.env.G1T_BUILD_ID);
155163 if (visitorAsksFirst(request)) keyUrl.searchParams.set("_g1t_consent", "1");
156164 const key = new Request(keyUrl, { method: "GET" });
157165 const repository = PUBLIC_PROJECT.test(new URL(request.url).pathname) ? repositoryOfPage(new URL(request.url).pathname) : null;
+6−0
6868 import { reportDeployment } from "./deploy/report.mjs";
6969 import { ROOT, byStage, codeStages, containersRollout, findWranglerConfigs, npmCiArgs, npmWorkspace, pick, problems, resolvedStack, waves } from "./deploy/stack.mjs";
7070 import { withDeployWindow } from "./deploy/status-window.mjs";
71+import { publishWebAssets, verifyWebAssets } from "./deploy/web-assets.mjs";
7172
7273 const USAGE = "usage: node scripts/deploy.mjs plan|deploy|build|migrate|manifest|doctor|install|build-base|image [--all] [--only a,b] [--skip a,b] [--force] [--rollback] [--concurrency N] [--stage S] [--json]";
7374
350351 const built = await exec("npm run build", [], { cwd, onLine: out.line, shell: true, env: wranglerEnv() });
351352 if (built.code !== 0) throw new Error(`npm run build failed:\n${lastLines(built.out)}`);
352353 }
354+ const webRelease = unit.id === "web" && !dryRun ? await publishWebAssets(cwd, { onLine: out.line, seedLive: !decision.missing }) : null;
353355 const args = ["deploy"];
354356 if (dryRun) {
355357 args.push("--dry-run", "--outdir", join(tmpdir(), "g1t-deploy", "dist", unit.id));
374376 deployed = await wrangler(args, { cwd, onLine: out.line });
375377 }
376378 if (deployed.code !== 0) throw new Error(`wrangler deploy failed:\n${wranglerError(deployed.out)}`);
379+ if (webRelease) {
380+ await verifyWebAssets(webRelease);
381+ out.line("Web cutover verified: current and previous build assets respond");
382+ }
377383 result.version = dryRun ? "(dry run)" : versionFrom(deployed.out);
378384 result.ok = true;
379385 if (unit.image && !dryRun) await reportContainers(unit, cwd, out);
+200−0
1+// Public, fingerprinted files outlive a Worker release. Upload before cutover;
2+// never delete or overwrite a filename with different bytes.
3+import { createHash, randomUUID } from "node:crypto";
4+import { readFile, readdir } from "node:fs/promises";
5+import { join, resolve } from "node:path";
6+import { pathToFileURL } from "node:url";
7+import { archivedAssetKey } from "../../apps/web/app/lib/asset-archive.ts";
8+import { cloudflareAuth, jsonFrom, wrangler, wranglerEnv } from "./cloudflare.mjs";
9+
10+const TYPES = { js: "text/javascript", css: "text/css", woff: "font/woff", woff2: "font/woff2", ttf: "font/ttf", otf: "font/otf", svg: "image/svg+xml", png: "image/png", jpg: "image/jpeg", jpeg: "image/jpeg", gif: "image/gif", webp: "image/webp", ico: "image/x-icon", wasm: "application/wasm" };
11+
12+export function assetReferences(text) {
13+ return [...new Set([...text.matchAll(/(?:\/?assets\/|["'(]\.\/)([a-zA-Z0-9_.~-]+\.(?:js|css|woff2?|ttf|otf|png|jpe?g|gif|svg|webp|ico|wasm))\b/g)].map((match) => match[1]))]
14+ // Vite's emitted filenames have an eight-character content hash.
15+ // Libraries can also contain unused relative defaults like elk-worker.min.js.
16+ .filter((name) => /-[a-zA-Z0-9_-]{8}\.[a-z0-9]+$/.test(name));
17+}
18+
19+async function together(items, run) {
20+ let index = 0;
21+ const results = await Promise.allSettled(Array.from({ length: Math.min(6, items.length) }, async () => {
22+ for (let i = index++; i < items.length; i = index++) await run(items[i]);
23+ }));
24+ const failed = results.find((result) => result.status === "rejected");
25+ if (failed) throw failed.reason;
26+}
27+
28+/** Bounded provider retries: archive trouble delays cutover, never skips it. */
29+export function retryingFetch(fetchImpl = fetch, wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms))) {
30+ return async (url, options = {}) => {
31+ for (let attempt = 0; ; attempt++) {
32+ const response = await fetchImpl(url, { ...options, signal: AbortSignal.timeout(60_000) });
33+ if (![429, 500, 502, 503, 504].includes(response.status) || attempt === 6) return response;
34+ const seconds = Number(response.headers.get("retry-after"));
35+ await response.body?.cancel();
36+ await wait(response.status === 429 ? Math.min(60_000, Math.max(1000, seconds * 1000 || 60_000)) : Math.min(1000 * 2 ** attempt, 30_000));
37+ }
38+ };
39+}
40+
41+const checksum = (bytes) => `"${createHash("md5").update(bytes).digest("hex")}"`;
42+const objectUrl = (account, bucket, key) => `https://api.cloudflare.com/client/v4/accounts/${account}/r2/buckets/${bucket}/objects/${key}`;
43+const manifestName = (names) => names.find((name) => /^manifest-[a-zA-Z0-9_-]{8}\.js$/.test(name));
44+
45+async function confirmUpload(response, bytes, label) {
46+ if (!response.ok) { await response.body?.cancel(); throw new Error(`Asset archive upload failed (${response.status}): ${label}`); }
47+ let receipt;
48+ try { receipt = await response.json(); } catch { throw new Error(`Invalid asset upload receipt: ${label}`); }
49+ if (receipt.success !== true || `"${receipt.result?.etag?.replaceAll('"', "")}"` !== checksum(bytes)) throw new Error(`Asset upload checksum mismatch: ${label}`);
50+}
51+
52+/** One manifest per release: work never grows with the number of old releases. */
53+export async function readArchiveManifest(url, headers, fetchImpl = fetch) {
54+ const response = await fetchImpl(url, { headers });
55+ if (response.status === 404) { await response.body?.cancel(); return null; }
56+ if (!response.ok) throw new Error(`Cannot read archived build (${response.status})`);
57+ const data = await response.json();
58+ if (data.version !== 1 || !Array.isArray(data.assets) || !data.assets.length || data.assets.some((row) => !Array.isArray(row) || row.length !== 2 || typeof row[0] !== "string" || archivedAssetKey(row[0].replace(/^web-assets/, "")) !== row[0] || !/^"[a-f0-9]{32}"$/.test(row[1]))) throw new Error("Invalid archived build manifest");
59+ return new Map(data.assets);
60+}
61+
62+export async function archiveAsset({ name, bytes, account, bucket, headers, index, fetchImpl = fetch }) {
63+ const key = archivedAssetKey(`/assets/${name}`);
64+ if (!key) throw new Error(`Not a public build asset: ${name}`);
65+ const url = objectUrl(account, bucket, key);
66+ const expected = checksum(bytes);
67+ if (index?.has(key)) {
68+ if (index.get(key) !== expected) throw new Error(`Asset filename collision: ${name}`);
69+ return;
70+ }
71+ // Cloudflare's R2 REST gateway does not implement HEAD. Range avoids
72+ // downloading unchanged files; cancel the body even if Range is ignored.
73+ const existing = await fetchImpl(url, { headers: { ...headers, range: "bytes=0-0" }, signal: AbortSignal.timeout(30_000) });
74+ if (existing.ok) {
75+ await existing.body?.cancel();
76+ if (existing.headers.get("etag") !== expected) throw new Error(`Asset filename collision: ${name}`);
77+ index?.set(key, expected);
78+ return;
79+ }
80+ await existing.body?.cancel();
81+ if (existing.status !== 404) throw new Error(`Asset archive lookup failed (${existing.status}): ${name}`);
82+ const uploaded = await fetchImpl(url, {
83+ method: "PUT", headers: { ...headers, "content-type": TYPES[name.split(".").at(-1)], "cache-control": "public, max-age=31536000, immutable" },
84+ body: bytes, signal: AbortSignal.timeout(60_000),
85+ });
86+ await confirmUpload(uploaded, bytes, name);
87+ index?.set(key, expected);
88+}
89+
90+/** Seed the currently live build too, so the first protected release saves existing tabs. */
91+async function liveReferences(origin, fetchImpl) {
92+ const pageUrl = new URL(origin);
93+ // The application's public HTML cache may still contain a previous release.
94+ pageUrl.searchParams.set("_g1t_archive", randomUUID());
95+ const page = await fetchImpl(pageUrl, { signal: AbortSignal.timeout(30_000) });
96+ if (!page.ok) throw new Error(`Cannot read the current site before cutover (${page.status})`);
97+ const pending = assetReferences(await page.text());
98+ if (!pending.some((name) => name.endsWith(".js")) || !pending.some((name) => name.endsWith(".css"))) throw new Error("Current site did not expose its JS and CSS assets; refusing an unprotected cutover");
99+ return pending;
100+}
101+
102+export async function archiveLiveAssets(origin, save, fetchImpl = fetch, references) {
103+ let pending = references ?? await liveReferences(origin, fetchImpl);
104+ const seen = new Set();
105+ while (pending.length) {
106+ const batch = [...new Set(pending)].filter((name) => !seen.has(name));
107+ pending = [];
108+ for (const name of batch) seen.add(name);
109+ if (seen.size > 4000) throw new Error("Current site's asset graph exceeds the archive limit");
110+ await together(batch, async (name) => {
111+ const response = await fetchImpl(new URL(`/assets/${name}`, origin), { signal: AbortSignal.timeout(30_000) });
112+ if (!response.ok || /text\/html/i.test(response.headers.get("content-type") ?? "")) throw new Error(`Current asset is unavailable: ${name}`);
113+ const bytes = Buffer.from(await response.arrayBuffer());
114+ await save(name, bytes);
115+ if (/\.(js|css)$/.test(name)) pending.push(...assetReferences(bytes.toString("utf8")));
116+ });
117+ }
118+ return seen.size;
119+}
120+
121+export async function publishWebAssets(cwd, { onLine = console.log, fetchImpl = fetch, seedLive = true, auth = cloudflareAuth(wranglerEnv()) } = {}) {
122+ const config = JSON.parse(await readFile(join(cwd, "build/server/wrangler.json"), "utf8"));
123+ const bucket = config.r2_buckets.find((binding) => binding.binding === "DOWNLOADS")?.bucket_name;
124+ if (!bucket || !config.account_id) throw new Error("Web asset archive needs the DOWNLOADS bucket and Cloudflare account");
125+ let headers = auth;
126+ if (!headers) {
127+ // Capture credentials only in memory; never pipe this output to deploy logs.
128+ const auth = await wrangler(["auth", "token", "--json"], { cwd });
129+ if (auth.code !== 0) throw new Error("Cannot authenticate the asset archive; run wrangler login");
130+ let credentials;
131+ try { credentials = jsonFrom(auth.out); } catch { throw new Error("Cannot read Wrangler credentials for asset archive"); }
132+ headers = credentials.type === "api_key" ? { "x-auth-key": credentials.key, "x-auth-email": credentials.email } : { authorization: `Bearer ${credentials.token}` };
133+ }
134+ const providerFetch = retryingFetch(fetchImpl);
135+ const manifestUrl = (name) => objectUrl(config.account_id, bucket, `web-assets/manifests/${name}.json`);
136+ const saveManifest = async (name, assets) => {
137+ const body = JSON.stringify({ version: 1, assets: [...assets] });
138+ const response = await providerFetch(manifestUrl(name), { method: "PUT", headers: { ...headers, "content-type": "application/json" }, body });
139+ await confirmUpload(response, body, name);
140+ };
141+ let index = new Map();
142+ const domain = config.routes?.find((route) => route.custom_domain)?.pattern;
143+ if (domain && seedLive) {
144+ const origin = `https://${domain}`;
145+ const references = await liveReferences(origin, fetchImpl);
146+ const manifest = manifestName(references);
147+ if (!manifest) throw new Error("Current site has no versioned route manifest");
148+ const archived = await readArchiveManifest(manifestUrl(manifest), headers, providerFetch);
149+ if (archived) {
150+ index = archived;
151+ onLine(`Current build is already archived (${index.size} assets)`);
152+ } else {
153+ const save = (name, bytes) => archiveAsset({ name, bytes, account: config.account_id, bucket, headers, index, fetchImpl: providerFetch });
154+ onLine(`Archived ${await archiveLiveAssets(origin, save, fetchImpl, references)} current assets before cutover`);
155+ await saveManifest(manifest, index);
156+ }
157+ }
158+ const directory = join(cwd, "build/client/assets");
159+ const previous = new Map(index);
160+ const names = (await readdir(directory)).filter((name) => archivedAssetKey(`/assets/${name}`));
161+ const manifest = manifestName(names);
162+ if (!manifest || !names.some((name) => name.endsWith(".css"))) throw new Error("Web build has no manifest/CSS assets");
163+ const current = new Map();
164+ await together(names, async (name) => {
165+ const bytes = await readFile(join(directory, name));
166+ await archiveAsset({ name, bytes, account: config.account_id, bucket, headers, index, fetchImpl: providerFetch });
167+ current.set(archivedAssetKey(`/assets/${name}`), checksum(bytes));
168+ });
169+ await saveManifest(manifest, current);
170+ onLine(`Verified ${names.length} assets for the new build; old filenames remain available`);
171+ const samples = new Map();
172+ for (const assets of [previous, current]) {
173+ for (const pattern of [/\.css$/, /\/entry\.client-.*\.js$/, /\/manifest-.*\.js$/]) {
174+ const entry = [...assets].find(([key]) => pattern.test(key));
175+ if (entry) samples.set(...entry);
176+ }
177+ }
178+ return { origin: domain ? `https://${domain}` : null, manifest, samples: [...samples] };
179+}
180+
181+/** Check both sides of cutover, including filenames no longer in the new bundle. */
182+export async function verifyWebAssets(release, fetchImpl = fetch) {
183+ if (!release.origin) return;
184+ const references = await liveReferences(release.origin, fetchImpl);
185+ if (!references.includes(release.manifest)) throw new Error("Web cutover is not serving the expected build");
186+ await together(release.samples, async ([key, etag]) => {
187+ const response = await fetchImpl(new URL(key.replace(/^web-assets/, ""), release.origin), { signal: AbortSignal.timeout(30_000) });
188+ if (!response.ok || /text\/html/i.test(response.headers.get("content-type") ?? "") || checksum(Buffer.from(await response.arrayBuffer())) !== etag) throw new Error(`Web cutover asset check failed: ${key}`);
189+ });
190+}
191+
192+if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) {
193+ const release = await publishWebAssets(process.cwd());
194+ if (process.argv.includes("--deploy")) {
195+ const deployed = await wrangler(["deploy"], { cwd: process.cwd(), onLine: console.log });
196+ if (deployed.code !== 0) throw new Error("Web deploy failed");
197+ await verifyWebAssets(release);
198+ console.log("Web cutover verified: current and previous build assets respond");
199+ }
200+}
+122−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+import { createHash } from "node:crypto";
4+import { mkdtemp, mkdir, writeFile, rm } from "node:fs/promises";
5+import { tmpdir } from "node:os";
6+import { dirname, join, resolve } from "node:path";
7+import { archiveAsset, archiveLiveAssets, assetReferences, publishWebAssets, readArchiveManifest, retryingFetch, verifyWebAssets } from "./web-assets.mjs";
8+
9+test("discover actual hashed assets without archiving icons, source maps or library defaults", () => {
10+ assert.deepEqual(assetReferences('<link href="/assets/app-12345678.css"><img src="/apple-touch-icon.png"> import "./chunk-abcdefgh.js"; import "./elk-worker.min.js"; url(/assets/font-12345678.woff2)'), ["app-12345678.css", "chunk-abcdefgh.js", "font-12345678.woff2"]);
11+});
12+
13+test("archive the old release's transitive chunks before replacing it", async () => {
14+ const responses = {
15+ "/": '<link href="/assets/app-12345678.css"><script src="/assets/entry-12345678.js"></script>',
16+ "/assets/app-12345678.css": "body{color:red}",
17+ "/assets/entry-12345678.js": 'import "./lazy-12345678.js"; import "./lazy-12345678.js"',
18+ "/assets/lazy-12345678.js": 'import "./entry-12345678.js"',
19+ };
20+ const saved = [];
21+ const total = await archiveLiveAssets("https://example.com", async (name) => saved.push(name), async (url) => new Response(responses[new URL(url).pathname]));
22+ assert.equal(total, 3);
23+ assert.deepEqual(saved.sort(), ["app-12345678.css", "entry-12345678.js", "lazy-12345678.js"]);
24+});
25+
26+test("archive failures block cutover and immutable filename collisions are refused", async () => {
27+ const args = { name: "app-12345678.css", bytes: Buffer.from("body{}"), account: "account", bucket: "bucket", headers: {} };
28+ const calls = [];
29+ await assert.rejects(archiveAsset({ ...args, fetchImpl: async (_url, init) => {
30+ calls.push(init.method ?? "GET");
31+ return new Response(null, { status: init.method === "PUT" ? 503 : 404 });
32+ } }), /upload failed/);
33+ assert.deepEqual(calls, ["GET", "PUT"]);
34+ await assert.rejects(archiveAsset({ ...args, fetchImpl: async (_url, init) => init.method === "PUT" ? Response.json({ success: true, result: { etag: "incorrect" } }) : new Response(null, { status: 404 }) }), /checksum mismatch/);
35+ await assert.rejects(archiveAsset({ ...args, fetchImpl: async () => new Response(null, { headers: { etag: '"different"' } }) }), /collision/);
36+ const etag = `"${createHash("md5").update(args.bytes).digest("hex")}"`;
37+ await archiveAsset({ ...args, fetchImpl: async (_url, init) => {
38+ assert.notEqual(init.method, "PUT");
39+ return new Response(null, { headers: { etag } });
40+ } });
41+});
42+
43+test("a missing live asset aborts the archive instead of saving an HTML error as CSS", async () => {
44+ await assert.rejects(archiveLiveAssets("https://example.com", async () => assert.fail(), async (url) => new URL(url).pathname === "/"
45+ ? new Response('/assets/app-12345678.css /assets/entry-12345678.js')
46+ : new Response("not found", { headers: { "content-type": "text/html" } })), /unavailable/);
47+});
48+
49+test("a release manifest skips unchanged uploads without scanning deployment history", async () => {
50+ const bytes = Buffer.from("body{}");
51+ const etag = createHash("md5").update(bytes).digest("hex");
52+ const index = await readArchiveManifest("https://provider/manifest", {}, async () => Response.json({ version: 1, assets: [["web-assets/assets/app-12345678.css", `"${etag}"`]] }));
53+ await archiveAsset({ name: "app-12345678.css", bytes, account: "a", bucket: "b", headers: {}, index, fetchImpl: async () => assert.fail("unchanged asset should not be uploaded") });
54+ await assert.rejects(archiveAsset({ name: "app-12345678.css", bytes: Buffer.from("different"), account: "a", bucket: "b", headers: {}, index }), /collision/);
55+});
56+
57+test("provider rate limits wait and retry; persistent failure remains a failure", async () => {
58+ const waits = [];
59+ let calls = 0;
60+ const response = await retryingFetch(async () => new Response(null, { status: ++calls < 3 ? 429 : 200, headers: { "retry-after": "2" } }), async (ms) => waits.push(ms))("https://provider");
61+ assert.equal(response.status, 200);
62+ assert.deepEqual(waits, [2000, 2000]);
63+ calls = 0;
64+ assert.equal((await retryingFetch(async () => { calls++; return new Response(null, { status: 503 }); }, async () => {})("https://provider")).status, 503);
65+ assert.equal(calls, 7);
66+});
67+
68+test("cutover verifies old and new assets and rejects HTML fallback or wrong build", async () => {
69+ const css = "body{}";
70+ const release = { origin: "https://example.com", manifest: "manifest-12345678.js", samples: [["web-assets/assets/app-old12345.css", `"${createHash("md5").update(css).digest("hex")}"`]] };
71+ const page = '<link href="/assets/app-12345678.css"><script src="/assets/manifest-12345678.js"></script>';
72+ await verifyWebAssets(release, async (url) => new Response(new URL(url).pathname === "/" ? page : css));
73+ await assert.rejects(verifyWebAssets(release, async (url) => new Response(new URL(url).pathname === "/" ? page : "error", { headers: { "content-type": "text/html" } })), /asset check failed/);
74+ await assert.rejects(verifyWebAssets(release, async () => new Response(page.replaceAll("12345678", "87654321"))), /expected build/);
75+});
76+
77+test("1000 successive releases keep old tabs' assets and perform bounded incremental archive work", async () => {
78+ const cwd = await mkdtemp(join(tmpdir(), "g1t-web-releases-"));
79+ const directory = join(cwd, "build/client/assets");
80+ await mkdir(directory, { recursive: true });
81+ await mkdir(join(cwd, "build/server"), { recursive: true });
82+ await writeFile(join(cwd, "build/server/wrangler.json"), JSON.stringify({ account_id: "a", r2_buckets: [{ binding: "DOWNLOADS", bucket_name: "b" }], routes: [{ custom_domain: true, pattern: "example.com" }] }));
83+ const objects = new Map();
84+ const shared = 'export const shared = true;';
85+ let live = null;
86+ let apiCalls = 0;
87+ const fetchImpl = async (input, init = {}) => {
88+ const url = new URL(input);
89+ if (url.hostname === "example.com") {
90+ assert.equal(url.pathname, "/", "already protected releases must not crawl all live chunks again");
91+ return new Response(`/assets/${live} /assets/app-12345678.css`);
92+ }
93+ apiCalls++;
94+ assert.ok(url.pathname.includes("/objects/"), "must not list all previous releases");
95+ const key = url.pathname.split("/objects/")[1];
96+ if (init.method === "PUT") {
97+ objects.set(key, Buffer.from(init.body));
98+ return Response.json({ success: true, result: { etag: createHash("md5").update(init.body).digest("hex") } });
99+ }
100+ const bytes = objects.get(key);
101+ return bytes ? new Response(bytes, { headers: { etag: `"${createHash("md5").update(bytes).digest("hex")}"` } }) : new Response(null, { status: 404 });
102+ };
103+ try {
104+ await writeFile(join(directory, "app-12345678.css"), "body{}");
105+ await writeFile(join(directory, "shared-12345678.js"), shared);
106+ for (let i = 0; i < 1000; i++) {
107+ const manifest = `manifest-${i.toString(16).padStart(8, "0")}.js`;
108+ if (live) await rm(join(directory, live));
109+ await writeFile(join(directory, manifest), `window.manifest=${i}`);
110+ apiCalls = 0;
111+ await publishWebAssets(cwd, { auth: { authorization: "test" }, fetchImpl, seedLive: i > 0, onLine: () => {} });
112+ assert.equal(apiCalls, i === 0 ? 7 : 4, `release ${i} should upload only its changed file and manifest`);
113+ live = manifest;
114+ }
115+ assert.equal(objects.get("web-assets/assets/shared-12345678.js").toString(), shared);
116+ assert.equal(objects.get("web-assets/assets/manifest-00000000.js").toString(), "window.manifest=0");
117+ assert.equal(objects.get("web-assets/assets/manifest-000003e7.js").toString(), "window.manifest=999");
118+ } finally {
119+ assert.equal(dirname(resolve(cwd)), resolve(tmpdir()));
120+ await rm(cwd, { recursive: true, force: true });
121+ }
122+});

Comments

Sign in to comment.