flagon-io/g1t

public

Compare changes

Choose two branches to see what one has that the other does not, then open a pull request for it.

2 commits

Merge branch 'worktree-agent-af58ac8933b0dd125'Chase Pierce1d51c601d ago
Costs: margin is measured on what was sold; comped workspaces, free periods, the trial and the pools are given away, a budget shown beside itChase Pierceb7ac05c1d ago

22 files+366−1840/22 viewed

180	180		Then open http://localhost:8787 and sign up. The confirmation mail is in
181	181		Mailpit at http://localhost:8025. Repositories, push and clone, issues,
182	182		pull requests and code browsing work, and the API and MCP server answer at
183		−	http://localhost:8789; agents, deployments and context search are off in
184		−	this version.
	183	+	http://localhost:8789; packages and container images are kept in the
	184	+	bundled S3-compatible store, RustFS. Agents, deployments and context search
	185	+	are off in this version.
185	186		[docs/SELF_HOSTING.md](docs/SELF_HOSTING.md) says what works and what is next.
186	187
187	188		### On Cloudflare

+37−9View BlameViewed

17	17		\| --- \| --- \|
18	18		\| Sign up, sign in, email confirmation \| Works. Mail goes to the bundled Mailpit inbox. \|
19	19		\| Workspaces, members, access tokens \| Works \|
20		−	\| Repositories: create, push and clone over HTTP, browse code, commits \| Works. A fresh clone's pack is kept in the bundled MinIO, so the next clone of the same commit is served from there. \|
	20	+	\| Repositories: create, push and clone over HTTP, browse code, commits \| Works. A fresh clone's pack is kept in the bundled object store, so the next clone of the same commit is served from there. \|
21	21		\| Issues, comments, labels \| Works \|
22	22		\| Pull requests from a branch or from a fork, merged onto the default branch \| Works, when the pull request is up to date with the default branch. Bringing one up to date first needs g1t's agent, which is off. \|
23	23		\| The merge queue \| Takes pull requests and shows them waiting. Testing and landing them needs g1t's agent, which is off: take a pull request out of the queue, or turn the queue off, to merge it. \|
24	24		\| [The REST API](/reference/api/), OAuth and [MCP](/reference/mcp/) \| Work, on a port of their own: `http://localhost:8789`, with the MCP server at `http://localhost:8789/mcp` \|
25		−	\| [Container images](/guides/containers/): `docker login`, push and pull at your `PUBLIC_URL` \| Works, kept in the bundled MinIO, with no limit on a layer's size or on pulls \|
	25	+	\| [Container images](/guides/containers/): `docker login`, push and pull at your `PUBLIC_URL` \| Works, kept in the bundled object store, with no limit on a layer's size or on pulls \|
26	26		\| Site search \| Works \|
27	27		\| A status page of your own \| Works, at `http://localhost:8788` ([below](#the-status-page)) \|
28	28		\| Webhooks, integrations \| Work, retries included \|
⋯
116	116		the site.
117	117		2. It makes an access token and calls the API, the OAuth metadata and the
118	118		MCP server with it.
119		−	3. It opens a pull request from a branch and one from a fork, through the
	119	+	3. It publishes an npm package to your installation's registry and
	120	+	installs it back.
	121	+	4. It opens a pull request from a branch and one from a fork, through the
120	122		API, and merges both onto `main`.
121		−	4. It turns the merge queue on, merges a pull request into it, takes it
	123	+	5. It turns the merge queue on, merges a pull request into it, takes it
122	124		out again, and merges it with the queue off.
123	125
124	126		```sh
⋯
135	137		```
136	138
137	139		It prints `All checks passed` when every step worked. It needs `curl`,
138		−	`git` and `node`.
	140	+	`git`, `node` and `npm`; `PACKAGES=off` skips the npm package.
139	141
140	142		## Settings
141	143
⋯
155	157		\| `REGISTRATION_MODE` \| `open` \| `open`: anyone can make an account. `invite`: every new account needs an [invite](/guides/authentication/#invites), as on g1t.sh. \|
156	158		\| `INVITES_PER_USER` \| `5` \| How many invites each person can have out, while `REGISTRATION_MODE` is `invite` \|
157	159		\| `WAITLIST_NOTIFY_EMAIL` \| (none) \| Where a summary of new access requests goes, at most every 15 minutes. Empty sends none; requests still wait for you in the database. \|
158		−	\| `S3_ENDPOINT`, `S3_BUCKET`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` \| the bundled MinIO, bucket `g1t-packages` \| Where packages' files are kept: any S3-compatible store. Change the two keys before first start; MinIO is made with them. \|
	160	+	\| `S3_ENDPOINT`, `S3_BUCKET`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` \| the bundled RustFS, bucket `g1t-packages` \| Where packages' files are kept: any S3-compatible store. Change the two keys before first start; RustFS is made with them. \|
159	161		\| `S3_PUBLIC_ENDPOINT` \| (none) \| The store's address as clients reach it. When set, large layers are downloaded from it directly with a signed URL. \|
160		−	\| `PACK_S3_BUCKET` \| `g1t-git-packs` \| The bucket on the same store that packs for fresh clones are kept in, so the next clone of the same commit is not built again. The bundled MinIO deletes packs after 7 days; on another store, give the bucket a rule that expires objects under `packs/` and unfinished multipart uploads. \|
161		−	\| `MINIO_IMAGE` \| `pgsty/minio:latest` \| The MinIO server image the bundled store runs. MinIO no longer publishes its own images; this is a community build of the same server. \|
	162	+	\| `PACK_S3_BUCKET` \| `g1t-git-packs` \| The bucket on the same store that packs for fresh clones are kept in, so the next clone of the same commit is not built again. The bundled store deletes packs after 7 days, and uploads left unfinished after a day; on another store, give the bucket a lifecycle rule that does the same. \|
	163	+	\| `RUSTFS_IMAGE` \| `rustfs/rustfs:1.0.1` \| The image the bundled object store runs: [RustFS](https://rustfs.com), an S3-compatible server. \|
	164	+	\| `AWS_CLI_IMAGE` \| `amazon/aws-cli:2.37.10` \| The image `storage-setup` makes the buckets and the packs' lifecycle rule with. \|
162	165		\| `BACKUP_S3_BUCKET` \| `g1t-backups` \| The bucket on the same store that nightly repository backups (a `git bundle` of each repository whose branches or tags changed) are kept in. The bundles are cut by g1t's runner, which this installation does not run yet, so the bucket stays empty for now: copy the volumes, as below. \|
163	166		\| `STATUS_PORT` \| `8788` \| The port the status page is published on \|
164	167		\| `STATUS_PROBE_REPO` \| (none) \| A public repository, `workspace/repo`, whose branches the status page lists every minute as a clone would. Empty: git is not checked. \|
⋯
238	241		\| --- \| --- \|
239	242		\| `g1t_g1t-data` \| Accounts, workspaces, issues and every other record, as SQLite files; the keys that seal stored secrets (`keys.env`) \|
240	243		\| `g1t_g1t-git` \| Your repositories, one bare git repository each \|
241		−	\| `g1t_g1t-packages` \| Container images' layers and other package files, the `g1t-backups` bucket and the clone packs in `g1t-git-packs` (MinIO) \|
	244	+	\| `g1t_g1t-objects` \| The bundled object store (RustFS): container images' layers and other package files in `g1t-packages`, the `g1t-backups` bucket and the clone packs in `g1t-git-packs` \|
242	245		\| `g1t_g1t-secrets` \| The key the site and the git store share \|
243	246
244	247		To back up, stop g1t and copy the volumes:
⋯
263	266		docker compose -f deploy/self-host/docker-compose.yml up --build -d
264	267		```
265	268
	269	+	### Installations started before 7 October 2026
	270	+
	271	+	These kept packages' files and backups in MinIO, in the `g1t_g1t-packages`
	272	+	volume. The bundled store is now RustFS, in `g1t_g1t-objects`, and starts
	273	+	empty. After the upgrade above, copy the old objects across (use your own
	274	+	`S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY` if you changed them):
	275	+
	276	+	```sh
	277	+	docker run -d --name g1t-old-store --network g1t_default \
	278	+	-v g1t_g1t-packages:/data -e MINIO_ROOT_USER=g1t \
	279	+	-e MINIO_ROOT_PASSWORD=g1t-packages-secret pgsty/minio server /data
	280	+	docker run --rm --network g1t_default -e AWS_ACCESS_KEY_ID=g1t \
	281	+	-e AWS_SECRET_ACCESS_KEY=g1t-packages-secret -e AWS_DEFAULT_REGION=us-east-1 \
	282	+	--entrypoint sh amazon/aws-cli:2.37.10 -c '
	283	+	for b in g1t-packages g1t-backups; do
	284	+	aws --endpoint-url http://g1t-old-store:9000 s3 sync "s3://$b" "/tmp/$b" &&
	285	+	aws --endpoint-url http://rustfs:9000 s3 sync "/tmp/$b" "s3://$b"
	286	+	done'
	287	+	docker rm -f g1t-old-store
	288	+	```
	289	+
	290	+	The clone packs are not copied: they are a cache, and are made again on
	291	+	the next clone. Once your images and packages pull, remove the old volume
	292	+	with `docker volume rm g1t_g1t-packages`.
	293	+
266	294		## Stop and remove
267	295
268	296		```sh

+17−5View BlameViewed

143	143		// Margin under the floor at the top; drift and leaks are in their own table, workspaces on Reach out.
144	144		const banner = report.alerts.filter((a) => a.kind === "overall" \|\| a.kind === "margin");
145	145		const elsewhere = report.alerts.length - banner.length;
	146	+	// What g1t gave away on purpose (comped workspaces, free periods, the
	147	+	// trial, the pools) is a budget, watched under g1t's own spend; the
	148	+	// margin is measured on what was sold. Older reports lack the fields.
	149	+	const given = report.overall.givenMicros ?? 0;
	150	+	const soldMicros = report.overall.soldMarginMicros ?? report.overall.marginMicros;
	151	+	const soldPercent = report.overall.soldMarginPercent !== undefined ? report.overall.soldMarginPercent : report.overall.marginPercent;
146	152		return (
147	153		<main className="mx-auto max-w-6xl px-4 py-8 sm:py-10">
148	154		<PageHeader
⋯
199	205		value={usd(report.overall.usageMicros + report.overall.plansMicros)}
200	206		hint={`${usd(report.overall.usageMicros)} usage, ${usd(report.overall.plansMicros)} plans`}
201	207		/>
202		−	<Stat label="Cloudflare cost" value={usd(report.overall.costMicros)} hint={`${report.since} to ${report.until}`} />
203	208		<Stat
204		−	label="Margin"
205		−	value={percentLabel(report.overall.marginPercent)}
206		−	hint={usd(report.overall.marginMicros)}
207		−	tone={marginTone(report.overall.marginPercent, floor)}
	209	+	label="Cost"
	210	+	value={usd(report.overall.costMicros)}
	211	+	hint={given > 0 ? `${usd(given)} of it given away` : `${report.since} to ${report.until}`}
	212	+	/>
	213	+	<Stat
	214	+	label="Margin on what was sold"
	215	+	value={percentLabel(soldPercent)}
	216	+	hint={given > 0 ? `${usd(soldMicros)}; ${percentLabel(report.overall.marginPercent)} with what was given` : usd(soldMicros)}
	217	+	tone={marginTone(soldPercent, floor)}
208	218		/>
209	219		<Stat
210	220		label="Proposals waiting"
⋯
401	411		<tr className="border-b border-line text-left text-xs text-muted">
402	412		<th className="px-4 py-2 font-medium sm:px-5">Workspace</th>
403	413		<th className="px-4 py-2 text-right font-medium">Cost to g1t</th>
	414	+	<th className="px-4 py-2 text-right font-medium">Given away</th>
404	415		<th className="px-4 py-2 text-right font-medium">Paid</th>
405	416		<th className="px-4 py-2 text-right font-medium sm:pr-5">Net</th>
406	417		</tr>
⋯
417	428		{w.internal && <span className="ml-2 text-xs text-faint">g1t's own</span>}
418	429		</td>
419	430		<td className="tabular px-4 py-2.5 text-right">{usd(w.costMicros)}</td>
	431	+	<td className="tabular px-4 py-2.5 text-right text-fg-soft">{usd(w.givenMicros ?? 0)}</td>
420	432		<td className="tabular px-4 py-2.5 text-right">{usd(w.revenueMicros)}</td>
421	433		<td className={`tabular px-4 py-2.5 text-right sm:pr-5 ${net < 0 && !w.internal ? "text-danger" : "text-fg-soft"}`}>{usd(net)}</td>
422	434		</tr>

+1−1View BlameViewed

3	3		version = "0.1.0"
4	4		edition.workspace = true
5	5		license.workspace = true
6		−	description = "Object storage behind one port: R2 on Cloudflare, any S3-compatible store (MinIO) when self-hosted."
	6	+	description = "Object storage behind one port: R2 on Cloudflare, any S3-compatible store (RustFS) when self-hosted."
7	7
8	8		[dependencies]
9	9		g1t-contracts.workspace = true

+1−1View BlameViewed

1	1		//! Object storage behind one port, `BlobStore`: R2 on Cloudflare, and any
2		−	//! S3-compatible store (MinIO in the self-host compose file) elsewhere.
	2	+	//! S3-compatible store (RustFS in the self-host compose file) elsewhere.
3	3		//!
4	4		//! Every service that keeps objects names its own [`Config`]: the variable
5	5		//! that chooses the store (`r2`, the default, or `s3`), the R2 bucket

+3−3View BlameViewed

1	1		//! The S3 adapter, for self-hosted installations: any S3-compatible store
2		−	//! (MinIO, Ceph, Garage, AWS) over fetch, signed with SigV4, path-style.
	2	+	//! (RustFS, Ceph, Garage, AWS) over fetch, signed with SigV4, path-style.
3	3		//! S3_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY and S3_REGION say
4	4		//! where, and the service's own variable (`Config::s3_bucket`) which
5	5		//! bucket. Its public endpoint variable, when it names one and that is
⋯
13	13		use crate::{BlobStore, Config, Got, Part, Wanted, var};
14	14
15	15		pub struct S3Store {
16		−	/// `http://minio:9000`, without a trailing slash.
	16	+	/// `http://rustfs:9000`, without a trailing slash.
17	17		endpoint: String,
18	18		/// Where clients reach the same store, for signed URLs.
19	19		public_endpoint: Option<String>,
⋯
249	249		let xml = "<InitiateMultipartUploadResult><Bucket>b</Bucket><UploadId>abc-123</UploadId></InitiateMultipartUploadResult>";
250	250		assert_eq!(xml_value(xml, "UploadId"), Some("abc-123"));
251	251		assert_eq!(xml_value(xml, "Key"), None);
252		−	assert_eq!(host_of("http://minio:9000"), "minio:9000");
	252	+	assert_eq!(host_of("http://rustfs:9000"), "rustfs:9000");
253	253		assert_eq!(host_of("https://s3.example.com/base"), "s3.example.com");
254	254		}
255	255		}

+14−1View BlameViewed

2386	2386		pub overhead: bool,
2387	2387		}
2388	2388
2389		−	/// All of g1t over the range: money in against every cost.
	2389	+	/// All of g1t over the range: money in against every cost, and against
	2390	+	/// the cost of what was sold (every cost less what g1t gave away).
2390	2391		#[derive(Clone, Debug, Default, Serialize, Deserialize)]
2391	2392		#[serde(rename_all = "camelCase")]
2392	2393		pub struct OverallMargin {
⋯
2396	2397		pub cost_micros: i64,
2397	2398		pub margin_micros: i64,
2398	2399		pub margin_percent: Option<f64>,
	2400	+	/// Of `cost_micros`, what went on usage g1t gave away on purpose:
	2401	+	/// comped workspaces, free periods, the trial and the open-source pool.
	2402	+	#[serde(default)]
	2403	+	pub given_micros: i64,
	2404	+	/// Money in against `cost_micros - given_micros`.
	2405	+	#[serde(default)]
	2406	+	pub sold_margin_micros: i64,
	2407	+	#[serde(default)]
	2408	+	pub sold_margin_percent: Option<f64>,
2399	2409		}
2400	2410
2401	2411		/// A count, cost or leak that does not add up.
⋯
2481	2491		pub workspace: String,
2482	2492		pub cost_micros: i64,
2483	2493		pub revenue_micros: i64,
	2494	+	/// Of `cost_micros`, what g1t gave away.
	2495	+	#[serde(default)]
	2496	+	pub given_micros: i64,
2484	2497		/// One of g1t's own (comped) workspaces.
2485	2498		pub internal: bool,
2486	2499		}

+4−4View BlameViewed

9	9		// - ARTIFACTS (git storage) becomes a service binding to workers/artifacts,
10	10		// which keeps repositories in the git store (gitstore/server.mjs);
11	11		// - EMAIL (Email Sending) becomes a service binding to workers/mail;
12		−	// - the packages service keeps files in S3-compatible storage (MinIO)
	12	+	// - the packages service keeps files in S3-compatible storage (RustFS)
13	13		// instead of R2, and the repos service its nightly backups (a bucket of
14	14		// their own, BACKUP_S3_BUCKET);
15	15		// - services that are off in this phase (agents, the context hub, the
⋯
216	216		// Access requests are summarised to your own address, not g1t.sh's.
217	217		config.vars.WAITLIST_NOTIFY_EMAIL = process.env.WAITLIST_NOTIFY_EMAIL ?? "";
218	218		}
219		−	// Packages' files go to the compose file's MinIO (or any S3-compatible
	219	+	// Packages' files go to the compose file's RustFS (or any S3-compatible
220	220		// store) instead of R2, with no request size limit, and package
221	221		// addresses start with this installation's host.
222	222		if (hosted.name === "g1t-packages") {
223	223		Object.assign(config.vars, {
224	224		BLOB_STORE: "s3",
225		−	S3_ENDPOINT: process.env.S3_ENDPOINT ?? "http://minio:9000",
	225	+	S3_ENDPOINT: process.env.S3_ENDPOINT ?? "http://rustfs:9000",
226	226		S3_BUCKET: process.env.S3_BUCKET ?? "g1t-packages",
227	227		S3_REGION: process.env.S3_REGION ?? "us-east-1",
228	228		S3_ACCESS_KEY_ID: process.env.S3_ACCESS_KEY_ID ?? "",
⋯
252	252		BACKUP_S3_BUCKET: process.env.BACKUP_S3_BUCKET ?? "g1t-backups",
253	253		PACK_STORE: "s3",
254	254		PACK_S3_BUCKET: process.env.PACK_S3_BUCKET ?? "g1t-git-packs",
255		−	S3_ENDPOINT: process.env.S3_ENDPOINT ?? "http://minio:9000",
	255	+	S3_ENDPOINT: process.env.S3_ENDPOINT ?? "http://rustfs:9000",
256	256		S3_REGION: process.env.S3_REGION ?? "us-east-1",
257	257		S3_ACCESS_KEY_ID: process.env.S3_ACCESS_KEY_ID ?? "",
258	258		S3_SECRET_ACCESS_KEY: process.env.S3_SECRET_ACCESS_KEY ?? "",

+32−34View BlameViewed

9	9		# What runs: the site and every core service in one workerd (g1t), git
10	10		# repositories as bare repos on a volume (gitstore), the API in a second
11	11		# workerd beside it, packages' files, backups and the clone pack cache in
12		−	# MinIO, and Mailpit for mail.
	12	+	# RustFS (S3-compatible storage), and Mailpit for mail.
13	13		# Agents, deployments, context search and billing are off. See
14	14		# docs/SELF_HOSTING.md.
15	15		name: g1t
⋯
51	51		INVITES_PER_USER: ${INVITES_PER_USER:-}
52	52		# Where summaries of new access requests go; empty sends none.
53	53		WAITLIST_NOTIFY_EMAIL: ${WAITLIST_NOTIFY_EMAIL:-}
54		−	# Packages' files (container images and the rest), in MinIO below or
	54	+	# Packages' files (container images and the rest), in RustFS below or
55	55		# any S3-compatible store. S3_PUBLIC_ENDPOINT, when clients can reach
56	56		# the store, sends large downloads there directly.
57		−	S3_ENDPOINT: ${S3_ENDPOINT:-http://minio:9000}
	57	+	S3_ENDPOINT: ${S3_ENDPOINT:-http://rustfs:9000}
58	58		S3_BUCKET: ${S3_BUCKET:-g1t-packages}
59	59		S3_REGION: ${S3_REGION:-us-east-1}
60	60		S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-g1t}
⋯
64	64		# the same store (docs/SELF_HOSTING.md, "Backups").
65	65		BACKUP_S3_BUCKET: ${BACKUP_S3_BUCKET:-g1t-backups}
66	66		# Packs kept for fresh clones, so the next clone of the same commit
67		−	# does not rebuild one; minio-setup expires them after 7 days.
	67	+	# does not rebuild one; storage-setup expires them after 7 days.
68	68		PACK_S3_BUCKET: ${PACK_S3_BUCKET:-g1t-git-packs}
69	69		volumes:
70	70		- g1t-data:/data
⋯
74	74		condition: service_healthy
75	75		mailpit:
76	76		condition: service_started
77		−	minio-setup:
	77	+	storage-setup:
78	78		condition: service_completed_successfully
79	79		restart: unless-stopped
80	80
⋯
110	110		# Not published: only the g1t container reaches it.
111	111		restart: unless-stopped
112	112
113		−	# Packages' files, backups and clone packs. Not published: only the g1t
114		−	# container reaches it, unless you publish 9000 and set
115		−	# S3_PUBLIC_ENDPOINT. MinIO no longer publishes images of its own;
116		−	# MINIO_IMAGE is a community build of the same server, with `mc` in it.
117		−	# Any S3-compatible store works in its place (S3_ENDPOINT).
118		−	minio:
119		−	image: ${MINIO_IMAGE:-pgsty/minio:latest}
120		−	command: ["server", "/data"]
	113	+	# Packages' files, backups and clone packs, in RustFS (S3-compatible).
	114	+	# Not published: only the g1t container reaches it, unless you publish
	115	+	# 9000 and set S3_PUBLIC_ENDPOINT. Any S3-compatible store works in its
	116	+	# place (S3_ENDPOINT).
	117	+	rustfs:
	118	+	image: ${RUSTFS_IMAGE:-rustfs/rustfs:1.0.1}
121	119		environment:
122		−	MINIO_ROOT_USER: ${S3_ACCESS_KEY_ID:-g1t}
123		−	MINIO_ROOT_PASSWORD: ${S3_SECRET_ACCESS_KEY:-g1t-packages-secret}
124		−	# What the health check's `mc ready local` asks.
125		−	MC_HOST_local: http://localhost:9000
	120	+	RUSTFS_ACCESS_KEY: ${S3_ACCESS_KEY_ID:-g1t}
	121	+	RUSTFS_SECRET_KEY: ${S3_SECRET_ACCESS_KEY:-g1t-packages-secret}
	122	+	RUSTFS_CONSOLE_ENABLE: "false"
126	123		volumes:
127		−	- g1t-packages:/data
	124	+	- g1t-objects:/data
128	125		healthcheck:
129		−	test: ["CMD", "mc", "ready", "local"]
	126	+	test: ["CMD", "curl", "-fsS", "-o", "/dev/null", "http://localhost:9000/health/ready"]
130	127		interval: 5s
131	128		retries: 20
132	129		restart: unless-stopped
133	130
134	131		# Makes the buckets once, then exits: packages' files, backups, and
135		−	# clone packs with a rule that deletes packs 7 days old. (MinIO itself
136		−	# removes uploads left unfinished after 24 hours.)
137		−	minio-setup:
138		−	image: ${MINIO_IMAGE:-pgsty/minio:latest}
	132	+	# clone packs, with a lifecycle rule on the packs' bucket that deletes
	133	+	# packs 7 days old and uploads left unfinished after a day.
	134	+	storage-setup:
	135	+	image: ${AWS_CLI_IMAGE:-amazon/aws-cli:2.37.10}
139	136		depends_on:
140		−	minio:
	137	+	rustfs:
141	138		condition: service_healthy
142	139		entrypoint:
143	140		- sh
144	141		- -c
145	142		- >-
146	143		set -e;
147		−	mc alias set local http://minio:9000 "$$MINIO_ROOT_USER" "$$MINIO_ROOT_PASSWORD";
148		−	mc mb --ignore-existing "local/$$S3_BUCKET";
149		−	mc mb --ignore-existing "local/$$BACKUP_S3_BUCKET";
150		−	mc mb --ignore-existing "local/$$PACK_S3_BUCKET";
151		−	if ! mc ilm rule ls "local/$$PACK_S3_BUCKET" >/dev/null 2>&1; then
152		−	mc ilm rule add --prefix packs/ --expire-days 7 "local/$$PACK_S3_BUCKET";
153		−	fi
	144	+	for bucket in "$$S3_BUCKET" "$$BACKUP_S3_BUCKET" "$$PACK_S3_BUCKET"; do
	145	+	aws s3api head-bucket --bucket "$$bucket" >/dev/null 2>&1 \|\| aws s3api create-bucket --bucket "$$bucket" >/dev/null;
	146	+	done;
	147	+	aws s3api put-bucket-lifecycle-configuration --bucket "$$PACK_S3_BUCKET" --lifecycle-configuration
	148	+	'{"Rules":[{"ID":"expire-packs","Status":"Enabled","Filter":{"Prefix":"packs/"},"Expiration":{"Days":7}},{"ID":"abort-unfinished-uploads","Status":"Enabled","Filter":{"Prefix":""},"AbortIncompleteMultipartUpload":{"DaysAfterInitiation":1}}]}';
	149	+	echo "buckets ready"
154	150		environment:
155		−	MINIO_ROOT_USER: ${S3_ACCESS_KEY_ID:-g1t}
156		−	MINIO_ROOT_PASSWORD: ${S3_SECRET_ACCESS_KEY:-g1t-packages-secret}
	151	+	AWS_ENDPOINT_URL: http://rustfs:9000
	152	+	AWS_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-g1t}
	153	+	AWS_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-g1t-packages-secret}
	154	+	AWS_DEFAULT_REGION: ${S3_REGION:-us-east-1}
157	155		S3_BUCKET: ${S3_BUCKET:-g1t-packages}
158	156		BACKUP_S3_BUCKET: ${BACKUP_S3_BUCKET:-g1t-backups}
159	157		PACK_S3_BUCKET: ${PACK_S3_BUCKET:-g1t-git-packs}
⋯
173	171
174	172		volumes:
175	173		g1t-data:
176		−	g1t-packages:
	174	+	g1t-objects:
177	175		g1t-status:
178	176		g1t-git:
179	177		g1t-secrets:

+25−3View BlameViewed

2	2		# End-to-end check of a self-hosted g1t: sign up, confirm the email, make a
3	3		# workspace and a repository, push and clone over HTTP, open an issue, and
4	4		# read the code back through the site. Then the API on its own port (REST,
5		−	# OAuth metadata and MCP, with an access token), pull requests from a branch
	5	+	# OAuth metadata and MCP, with an access token), an npm package published to
	6	+	# the installation's registry and installed back, pull requests from a branch
6	7		# and from a fork merged onto main, the merge queue taking a pull request
7	8		# and giving it back, and every cron handler the scheduler runs.
8	9		#
⋯
21	22		# (In Git Bash on Windows, start the command with `env MSYS_NO_PATHCONV=1`
22	23		# so /data is not rewritten into a Windows path.)
23	24		# PACK_CACHE=off skips the check that a second clone is served from the
24		−	# clone pack cache.
	25	+	# clone pack cache. PACKAGES=off skips publishing an npm package to the
	26	+	# installation's registry and installing it back.
25	27		#
26		−	# Needs curl, git and node (to read JSON).
	28	+	# Needs curl, git and node (to read JSON), and npm for the package.
27	29		set -euo pipefail
28	30
29	31		G1T_URL="${G1T_URL:-http://localhost:8787}"
⋯
35	37		# off: this installation keeps no clone packs (no PACK_STORE), so a second
36	38		# clone is not checked for a kept one.
37	39		PACK_CACHE="${PACK_CACHE:-on}"
	40	+	# off: skip publishing and installing an npm package (which needs npm).
	41	+	PACKAGES="${PACKAGES:-on}"
38	42		RUN="$(date +%s)"
39	43		USER_NAME="smoke${RUN}"
40	44		EMAIL="${USER_NAME}@example.com"
⋯
209	213		[ "$code" = 200 ] && grep -q '"tools"' "$WORK/api" \|\| fail "MCP tools/list: $code $(head -c 300 "$WORK/api")"
210	214		echo "MCP lists its tools at $MCP_URL"
211	215
	216	+	if [ "$PACKAGES" != off ]; then
	217	+	step "publish an npm package and install it"
	218	+	# The tarball is kept in the packages store (S3), and read back from it.
	219	+	npmrc="@$WORKSPACE:registry=$G1T_URL/-/npm/
	220	+	//${G1T_URL#*://}/-/npm/:_authToken=$TOKEN"
	221	+	mkdir -p "$WORK/pkg" "$WORK/app"
	222	+	printf '%s\n' "$npmrc" > "$WORK/pkg/.npmrc"
	223	+	printf '%s\n' "$npmrc" > "$WORK/app/.npmrc"
	224	+	printf '{"name":"@%s/%s","version":"1.0.0","repository":"%s/%s/%s","main":"index.js"}\n' \
	225	+	"$WORKSPACE" "$REPO" "$G1T_URL" "$WORKSPACE" "$REPO" > "$WORK/pkg/package.json"
	226	+	printf 'module.exports = "published by smoke.sh %s";\n' "$RUN" > "$WORK/pkg/index.js"
	227	+	(cd "$WORK/pkg" && npm publish --silent) \|\| fail "npm publish"
	228	+	(cd "$WORK/app" && npm init -y >/dev/null && npm install --silent --no-audit --no-fund "@$WORKSPACE/$REPO@1.0.0") \|\| fail "npm install"
	229	+	got="$(cd "$WORK/app" && node -p "require('@$WORKSPACE/$REPO')")"
	230	+	[ "$got" = "published by smoke.sh $RUN" ] \|\| fail "the installed package says: $got"
	231	+	echo "installed @$WORKSPACE/$REPO@1.0.0"
	232	+	fi
	233	+
212	234		step "a pull request from a branch, merged"
213	235		git -C "$WORK/clone" config credential.helper ""
214	236		commit_file "$WORK/clone" "$remote" from-branch docs/branch.md

+3−3View BlameViewed

373	373		\| R10 \| Built in repos; work unchanged \| `divergence` works out the target's side once per target head per isolate (`coalesce.rs`: the head under the refs version, then the history by hash, kept 60 s), and what the target changed between two trees once per pair (10 minutes). `readCommit` and logs by hash come from the cache. Work's fan-out (`after_push`, up to 100 pull requests) is unchanged: its 100 `divergence` calls now cost one walk of the target instead of 100. \|
374	374		\| R7 \| Built; the namespaces are yours to make \| `shards.rs`: bindings named in `ARTIFACTS_NAMESPACES` (JSON, binding → namespace; `ARTIFACTS` → `g1t` always there), a repository's namespace kept in its `store` column as `<namespace>/<key>` (no prefix means the `ARTIFACTS` namespace, so every existing key reads the same), forks always in their repository's namespace. Placing (`Placement::choose`, loads from `namespaces.rs`): among `ARTIFACTS_NEW_REPOS`, the healthy namespaces (bound, taking writes, not failing, under `ARTIFACTS_NAMESPACE_LIMITS`' `max_repos`, busiest minute under 70% of the 12,000-a-minute limit) within 100 repositories or 5% of the emptiest, spread by an FNV hash of the id; loads are read (one D1 query each for the registry and `store_health`, kept a minute) only when there is more than one to choose from. Moving (`moves.rs`): `move_repository` or `scripts/ops/artifacts-namespaces.mjs move` queues one; the hourly sweep pauses writes, copies every ref of the repository and its working copies over git (one streamed upload-pack into one receive-pack each), switches every `store` key in one batch, and deletes the old copies after 7 days. EU residency: identity's `workspaces.data_residency`, set by an owner in the workspace's settings (shown only once `storage_options` says an EU namespace takes repositories), read by the repos service at creation only while `ARTIFACTS_EU_NAMESPACE` is set; an EU workspace's repository goes to that namespace or is not made. Health and limits: `namespaces` RPC and `scripts/ops/artifacts-namespaces.mjs`. Nothing changes until bindings and variables name new namespaces. See "R7: sharding, moves and EU residency" below. \|
375	375		\| R8 \| Built \| `crates/runner/src/clone.rs`: every sandbox clones at `--depth=1` (a full g1t clone took 5.4 s, depth 1 took 3.8 s). Work that merges (catch-up, the merge queue, merge checks, a review's diff) deepens 50, 500, then 5000 commits until the two sides share one, and fetches everything only as the last resort (`share_history`). `G1T_CLONE_DEPTH` (0 or `full` for everything) and `G1T_CLONE_FILTER=blob:none` change it per runner. \|
376		−	\| R6 \| Built; the bucket must exist before it deploys \| `pack_cache.rs`: an upload-pack POST with wants and no `have` or `shallow` lines (a fresh clone, the sandboxes' `deepen 1` ones included), uncompressed and at most 1 MiB, is keyed `packs/<repo id>/<refs_version>/<sha256>` over the request normalized: protocol v2 capabilities without `agent=`/`session-id=` and its arguments, each sorted and deduplicated; v0/v1 wants sorted, the first want's capabilities split off, sorted and without `agent=`, then `deepen`/`filter` lines, a flush and `done`. Only while `refs_cache::usable` (the version known, and no push credential out of g1t's hands), so never across a refs change. Looked up after authorization, alongside the free-workspace limits and the kept refs answer; a hit streams from the bucket (`Server-Timing` `pack;desc=hit`). A miss streams the store's 200 to git through a tee that copies it to a fill in `ctx.wait_until` (at most 5 MiB queued between them, 2 fills per isolate, one per key): under 5 MiB it is one `put` once it all arrived; larger, 5 MiB multipart parts completed only after the last part and a check that it is one whole side-band pack (well-formed pkt-lines, `PACK` on channel 1, no `ERR` or channel 3, a closing flush). Over 200 MB, a queue that falls behind, git going away or the store's stream failing lets the fill go and aborts the upload; nothing partial can be read. Meters `pack_cache.hit` (with the bytes served) and `pack_cache.miss` (counted with `record`, bytes added at the end), neither an operation by default; a hit records no `git.fetch`. Storage is behind the `PackStore` port, whose adapter puts the shared `BlobStore` (`crates/blobstore`) behind it: R2 hosted (`GIT_PACKS`, bucket `g1t-git-packs`, lifecycle: packs deleted after 7 days, unfinished uploads after 1), or with `PACK_STORE=s3` the S3 bucket `PACK_S3_BUCKET` names (self-hosted: MinIO's `g1t-git-packs`, packs deleted after 7 days, unfinished uploads by MinIO after 24 hours; `node services/repos/dev/clone-check.mjs --s3` checks it). With neither, nothing is kept. \|
	376	+	\| R6 \| Built; the bucket must exist before it deploys \| `pack_cache.rs`: an upload-pack POST with wants and no `have` or `shallow` lines (a fresh clone, the sandboxes' `deepen 1` ones included), uncompressed and at most 1 MiB, is keyed `packs/<repo id>/<refs_version>/<sha256>` over the request normalized: protocol v2 capabilities without `agent=`/`session-id=` and its arguments, each sorted and deduplicated; v0/v1 wants sorted, the first want's capabilities split off, sorted and without `agent=`, then `deepen`/`filter` lines, a flush and `done`. Only while `refs_cache::usable` (the version known, and no push credential out of g1t's hands), so never across a refs change. Looked up after authorization, alongside the free-workspace limits and the kept refs answer; a hit streams from the bucket (`Server-Timing` `pack;desc=hit`). A miss streams the store's 200 to git through a tee that copies it to a fill in `ctx.wait_until` (at most 5 MiB queued between them, 2 fills per isolate, one per key): under 5 MiB it is one `put` once it all arrived; larger, 5 MiB multipart parts completed only after the last part and a check that it is one whole side-band pack (well-formed pkt-lines, `PACK` on channel 1, no `ERR` or channel 3, a closing flush). Over 200 MB, a queue that falls behind, git going away or the store's stream failing lets the fill go and aborts the upload; nothing partial can be read. Meters `pack_cache.hit` (with the bytes served) and `pack_cache.miss` (counted with `record`, bytes added at the end), neither an operation by default; a hit records no `git.fetch`. Storage is behind the `PackStore` port, whose adapter puts the shared `BlobStore` (`crates/blobstore`) behind it: R2 hosted (`GIT_PACKS`, bucket `g1t-git-packs`, lifecycle: packs deleted after 7 days, unfinished uploads after 1), or with `PACK_STORE=s3` the S3 bucket `PACK_S3_BUCKET` names (self-hosted: RustFS's `g1t-git-packs`, with the same lifecycle rule, put by the compose file's `storage-setup`; `node services/repos/dev/clone-check.mjs --s3` checks it). With neither, nothing is kept. \|
377	377		\| R11 \| Built; not yet deployed \| Nightly `git bundle` backups to the `g1t-backups` R2 bucket, and a restore drill. Migration `0013_backups.sql` (`repo_backups`, and an `operation_mapping` row). See "R11: backups and the restore drill" below. \|
378	378		\| R12 \| Built; the host is yours to make \| `scripts/ops/restore-to-gitstore.mjs` rebuilds every repository from its bundle chain into the git store (`deploy/self-host/gitstore`, now with namespaced keys, `<root>/<namespace>/<name>.git`, and `GITSTORE_READ_ONLY=1`), on the host or over its API, and later lists and reconciles what the fallback took. `fallback.rs`: with `GIT_FALLBACK_URL`, `GIT_FALLBACK_SECRET` and `GIT_FALLBACK_NAMESPACES` set, a namespace's `GitStore` calls go to the git store's API instead of the Artifacts binding (same metering, retries and breaker, its own health as `<namespace>@fallback`), read-only unless `GIT_FALLBACK_WRITES=allow`: writes are refused before they are asked, and say so in words; kept ref listings and packs are not used, nor backups cut. status.g1t.sh shows Git storage degraded meanwhile. See "R12: the fallback store and the outage runbook" below. \|
379	379
⋯
654	654
655	655		Storage, through the `BlobStore` port in `crates/blobstore` (the adapters packages already used):
656	656		the `BACKUPS` binding (bucket `g1t-backups`) with `BACKUP_STORE=r2`; any S3-compatible store with
657		−	`BACKUP_STORE=s3` and `BACKUP_S3_BUCKET` (self-hosted: MinIO). Without either, backups are off and
	657	+	`BACKUP_STORE=s3` and `BACKUP_S3_BUCKET` (self-hosted: RustFS). Without either, backups are off and
658	658		the nightly cron does nothing.
659	659
660	660		```text
⋯
696	696		at random, the repository is one whose refs have not moved since its last backup, so any
697	697		difference is the backup's. Run it after the first night, then monthly, and after any change to
698	698		`backups.rs` or `backup.rs`. `--bundles <dir>` reads a local copy of the bucket instead
699		−	(self-hosted: `mc mirror local/g1t-backups <dir>`), with `--repo-id` and `--live <url or path>`.
	699	+	(self-hosted: `aws --endpoint-url <S3_ENDPOINT> s3 sync s3://g1t-backups <dir>`), with `--repo-id` and `--live <url or path>`.
700	700		`npm run test:ops` runs it against bundles cut with git.
701	701
702	702		A real restore into the store, as it can be done today:

+14−3View BlameViewed

110	110		what the plan's included usage, a trial, the open-source pool or g1t paid.
111	111		g1t's own (comped) workspaces are valued at cost plus the margin.
112	112		- Cash = what workspaces paid: `-amount_micros`, and the plan's price.
	113	+	- Given away = the part of the cost that went on usage g1t paid for
	114	+	itself on purpose: all of a comped workspace's, all of a free period's,
	115	+	and what the trial and the open-source pool paid. Each workspace's day
	116	+	is split by the share of its value at price that was given (value less
	117	+	cash less the plan's included usage, which the plan's price paid for),
	118	+	and that share of each of its buckets' cost is given, its part of
	119	+	running g1t included. Stored as `given_micros` on `margin_days` and
	120	+	`workspace_costs`.
113	121		- Month-end meters: a day's figure is that day's `pending_days`
114	122		snapshot less the day before's, within a month. Their month-end ledger
115	123		entries are left out, so nothing is counted twice.
116		−	- Product margin = (value − cost) / value. Overall margin =
117		−	(Σ cash − Σ cost) / Σ cash.
	124	+	- Product margin = (value − cost) / value. **Margin on what was
	125	+	sold**, sudo's headline and the overall alert = (Σ cash − (Σ cost − Σ
	126	+	given)) / Σ cash. The margin with what was given, (Σ cash − Σ cost) / Σ
	127	+	cash, shows under it. What was given is a budget, watched under g1t's own
	128	+	spend, not a price below cost.
118	129		- Quantities: where a mapping names an `own_meter`, Cloudflare's
119	130		billed quantity of those lines (or, without one, Artifacts' operation
120	131		events) against g1t's own count.
⋯
204	215		\| Alert \| Raised when \| First steps \|
205	216		\| --- \| --- \| --- \|
206	217		\| Margin under the floor \| A product's value against cost under `margin_floor_percent` (10%) for `alert_days` (3) days running, each with at least `min_daily_cost` \| Open the product on Costs & margin. Cost up? Check proposals (approve a rise; it waits out the notice). Value down? A mapping or `revenue_map` may have moved. \|
207		−	\| All of g1t under the floor \| The same for money in against every cost, comped workspaces' share left out (their spend is a budget, watched in budget.rs). While less than $1 a day comes in, it says the dollars, not a percentage \| Look at which products moved; check `platform` (it has no revenue of its own and grows with traffic). Before launch, with little paid usage, expect it. \|
	218	+	\| All of g1t under the floor \| The same for money in against the cost of what was sold: every cost less what was given away (comped workspaces, free periods, the trial, the pools), which is a budget watched in budget.rs. While less than $1 a day comes in, it says the dollars, not a percentage \| Look at which products moved; check `platform` (it has no revenue of its own and grows with traffic). Before launch, with little paid usage, expect it. \|
208	219		\| Leak \| Drift of kind leak \| Map the meter, or decide it is overhead. \|
209	220		\| Drift \| Count drift \| See Drift above. Cloudflare's definitions change in beta: ask them in writing ([ARTIFACTS.md](ARTIFACTS.md), §7). \|
210	221		\| Costs more than it pays \| A workspace's shared cost over 30 days above what its usage was priced at (`value_micros`, whoever paid: card, trial, gift or included usage) × `anomaly_factor`, at least `anomaly_floor`; not comped workspaces \| Shown on Reach out as "Costs more than it pays": its usage is priced below what it costs. Abuse (Abuse & fraud page) or a gap in pricing. Not emailed. A trial or gift paying for usage does not raise it. \|

+1−1View BlameViewed

36	36		`packages/contracts`. Other services talk to it over RPC and hear from it through events.
37	37		- Cloudflare in production, anything in a self-hosted install. Files go through a
38	38		`BlobStore` port. In production its adapter is R2; self-hosted it is S3-compatible storage
39		−	(MinIO in the compose file) or a local directory. Upload URLs (`presign`) are part of the port:
	39	+	(RustFS in the compose file) or a local directory. Upload URLs (`presign`) are part of the port:
40	40		R2 and S3 sign them, the disk adapter answers with a path back through the service.
41	41		Metadata is D1, which self-hosting already runs (workerd's D1 over SQLite).
42	42		- Content-addressed. Every file is stored once by its SHA-256 (`blobs/sha256/<hex>`). A

+40−25View BlameViewed

40	40		answers "agents are off" instead of failing.
41	41		- Proven on this machine with Docker: sign up, confirm the email
42	42		through Mailpit, create a workspace and a repository, push and clone over
43		−	HTTP (the second clone from the clone pack cache in MinIO), open an
	43	+	HTTP (the second clone from the clone pack cache in RustFS), open an
44	44		issue, and browse code, commits and files in the site; then the REST API,
45		−	OAuth metadata and MCP on their own port, pull requests from a branch and
46		−	from a fork merged onto `main`, the merge queue taking a pull request and
	45	+	OAuth metadata and MCP on their own port, an npm package published and
	46	+	installed and a container image pushed and pulled through RustFS, pull
	47	+	requests from a branch and from a fork merged onto `main`, the merge queue taking a pull request and
47	48		giving it back, and every cron handler the scheduler runs. All of it runs
48	49		against local storage. See [Phase 1: what works today](#3-phase-1-what-works-today).
49	50		- Long term: keep workerd as the runtime, because it is what hosted
⋯
68	69		\| Workers runtime, service bindings \| Every service. Rust through `worker` 0.8 (`#[event(fetch\\|queue\\|scheduled)]`, `Env`, `Fetcher`); TS as `export default { fetch, queue, scheduled }` \| woven (as a runtime), thin (as an API) \| workerd: the same runtime, open source. Service bindings work as they do hosted. Calls are HTTP (`POST /rpc/<method>`), so a native port could use plain HTTP clients. \|
69	70		\| Workers RPC (JS methods across a binding) \| Only `RUNNER`: `RunnerService extends WorkerEntrypoint` (`services/runner/src/index.ts:626`). `apps/web` calls `env.RUNNER.enabled/run/plan/...` directly in 11 routes. \| thin \| workerd supports it. A native port needs these on `/rpc/*` as well; the runner already has a `fetch` shim for Rust callers. \|
70	71		\| D1 \| System of record for 13 services. Rust: `env.d1("DB")`; TS: `D1Database`; `db.batch()` in `crates/kit` `rename` \| woven (SQL), thin (API) \| SQLite files. workerd/Miniflare implements D1 on SQLite, and the same `migrations/` apply with `wrangler d1 migrations apply --local`. A native port would need a `Database` port over `rusqlite`/`better-sqlite3`; the SQL is already SQLite, including FTS5. \|
71		−	\| KV \| `BLOBS`: Actions artifacts and cache (`apps/api/src/blobs.rs`, `apps/web/app/lib/artifacts.server.ts`). `AVATARS`: `services/identity/src/avatars.rs`, `apps/web/workers/app.ts`, `services/og`. `DOMAINS`: `services/deployments/src/domains.ts`, `services/pages` \| thin \| Miniflare KV on disk (SQLite plus blob files). Natively: a `BlobStore` port on the filesystem or S3/MinIO. \|
	72	+	\| KV \| `BLOBS`: Actions artifacts and cache (`apps/api/src/blobs.rs`, `apps/web/app/lib/artifacts.server.ts`). `AVATARS`: `services/identity/src/avatars.rs`, `apps/web/workers/app.ts`, `services/og`. `DOMAINS`: `services/deployments/src/domains.ts`, `services/pages` \| thin \| Miniflare KV on disk (SQLite plus blob files). Natively: a `BlobStore` port on the filesystem or S3. \|
72	73		\| Queues: the event bus \| Producer: `services/events` `BUS.sendBatch` (`lib.rs:67`). The consumer writes the log, then fans out to every binding named `SUBSCRIBER_*` (`lib.rs:161`). Twelve consumers, one queue each. Private job queues in search (`g1t-search-jobs`) and context (`g1t-context-jobs`); consumers branch on the queue name. \| woven \| Miniflare Queues: in-process and persisted, which works today. Natively: a `Bus` port with a SQLite outbox and a poller per subscriber, or NATS/Redis Streams. At-least-once delivery and idempotent consumers are already the contract. \|
73	74		\| Durable Objects \| Only `AttemptSandbox` (runner), as the containers library's base class. Uses `ctx.storage.get/put/delete`, `schedule()` (alarm), `idFromName`/`idFromString`, DO RPC (`run`, `destroy`, `noteBlocked`). Not used: WebSocket hibernation, raw `alarm()`, `ctx.storage.sql`, `ctx.exports`. \| woven, in the runner only \| workerd supports Durable Objects (on-disk SQLite). Runner state can move to the sandbox supervisor (phase 2). \|
74	75		\| Containers (`@cloudflare/containers`) \| `services/runner`: one sandbox per agent run, Actions job and deploy build. `sleepAfter`, `start({ envVars, enableInternet })`, `onStop`. Image: `services/runner/Dockerfile` (node 24, git, toolchains, Claude Code, `g1t-runner`). \| woven \| Docker or Podman through the socket, with the same image. Wrangler can already run Containers locally through Docker; whether that covers outbound interception has to be tested. \|
⋯
88	89		\| Static Assets \| `apps/web` (Vite plugin build), `apps/docs`, `apps/sudo` (`run_worker_first`) \| thin \| workerd serves them. \|
89	90		\| `placement`, `observability`, routes, custom domains \| every `wrangler.jsonc` \| config only \| Dropped by `deploy/self-host/configs.mjs`. \|
90	91		\| `cf-ray` \| Used as an audit request id, with a fallback: `services/repos/src/run_access.rs:131`, `apps/api/src/audit.rs:37` \| thin \| Falls back already. \|
91		−	\| R2 \| `services/packages` (`BLOBS`: container layers and other package files), `services/repos` (`BACKUPS`: nightly backup bundles; `GIT_PACKS`: the clone pack cache), the API's Actions cache (`ACTIONS_CACHE`), the runner's downloads \| thin \| S3-compatible storage: the `BlobStore` port in `crates/blobstore` has an R2 adapter and an S3 one (`s3.rs`, SigV4 over fetch); each service names its own bucket (`BLOB_STORE`/`S3_BUCKET` for packages, `BACKUP_STORE`/`BACKUP_S3_BUCKET` for backups, `PACK_STORE`/`PACK_S3_BUCKET` for clone packs), run against MinIO in the compose file. \|
	92	+	\| R2 \| `services/packages` (`BLOBS`: container layers and other package files), `services/repos` (`BACKUPS`: nightly backup bundles; `GIT_PACKS`: the clone pack cache), the API's Actions cache (`ACTIONS_CACHE`), the runner's downloads \| thin \| S3-compatible storage: the `BlobStore` port in `crates/blobstore` has an R2 adapter and an S3 one (`s3.rs`, SigV4 over fetch); each service names its own bucket (`BLOB_STORE`/`S3_BUCKET` for packages, `BACKUP_STORE`/`BACKUP_S3_BUCKET` for backups, `PACK_STORE`/`PACK_S3_BUCKET` for clone packs), run against RustFS in the compose file. \|
92	93		\| Not used \| Hyperdrive, Workflows, Analytics Engine, Browser Rendering, Images, Turnstile, Secrets Store, `connect()`, HTMLRewriter, `request.cf` \| — \| — \|
93	94
94	95		### By service
⋯
106	107		\| `apps/docs` \| Static \| — \| Not run (docs.g1t.sh serves them) \|
107	108		\| `apps/status` \| TS Worker \| Email Sending, cron; bound only to billing \| Runs in a process of its own (`status.sh`), so it stays up when the site does not \|
108	109		\| `services/identity` \| Rust \| Email Sending, KV `AVATARS` \| Runs unchanged; `EMAIL` goes to the mail shim \|
109		−	\| `services/repos` \| Rust \| Artifacts, R2 (`BACKUPS`), Cache API, optional KV `GIT_CACHE` with `REPOS_KEY`, optional R2 `GIT_PACKS` \| Runs unchanged; `ARTIFACTS` goes to the git store, backups to MinIO's `g1t-backups` bucket (`BACKUP_STORE=s3`), and the clone pack cache to `g1t-git-packs` (`PACK_STORE=s3`: the `PackStore` port in `src/pack_cache.rs` over the shared `BlobStore`, multipart, an object only once whole; `minio-setup` gives the bucket a rule that deletes packs after 7 days, and MinIO removes unfinished uploads after 24 hours). Without `GIT_CACHE` and `REPOS_KEY`, credentials and ref listings are kept per isolate only. Its nightly cron queues backups, but bundles are cut by the runner, which is off in phase 1: none are made yet \|
	110	+	\| `services/repos` \| Rust \| Artifacts, R2 (`BACKUPS`), Cache API, optional KV `GIT_CACHE` with `REPOS_KEY`, optional R2 `GIT_PACKS` \| Runs unchanged; `ARTIFACTS` goes to the git store, backups to the `g1t-backups` bucket in RustFS (`BACKUP_STORE=s3`), and the clone pack cache to `g1t-git-packs` (`PACK_STORE=s3`: the `PackStore` port in `src/pack_cache.rs` over the shared `BlobStore`, multipart, an object only once whole; `storage-setup` gives the bucket a lifecycle rule that deletes packs after 7 days and aborts uploads unfinished after a day). Without `GIT_CACHE` and `REPOS_KEY`, credentials and ref listings are kept per isolate only. Its nightly cron queues backups, but bundles are cut by the runner, which is off in phase 1: none are made yet \|
110	111		\| `services/work` \| Rust \| Queue consumer \| Runs unchanged \|
111	112		\| `services/events` \| Rust \| Queues (producer and fan-out) \| Runs unchanged; the off services' queues are not produced to \|
112	113		\| `services/projects` \| TS \| Queue consumer \| Runs unchanged \|
⋯
116	117		\| `services/actions` \| Rust \| Queue, cron, `ACTIONS_KEY` \| Runs; jobs need the runner, which is off \|
117	118		\| `services/webhooks` \| Rust \| Queue, cron, `WEBHOOKS_KEY` \| Runs; retries through `scheduler.mjs` \|
118	119		\| `services/integrations` \| Rust \| Queue, `INTEGRATIONS_KEY` \| Runs unchanged \|
119		−	\| `services/packages` \| Rust \| R2 (`BLOBS`), cron, queue, `PACKAGES_TOKEN_SECRET` \| Runs with `BLOB_STORE=s3` against the compose file's MinIO (`deploy/self-host/configs.mjs`); no request size limit (`MAX_REQUEST_BYTES` 0 means none) \|
	120	+	\| `services/packages` \| Rust \| R2 (`BLOBS`), cron, queue, `PACKAGES_TOKEN_SECRET` \| Runs with `BLOB_STORE=s3` against the compose file's RustFS (`deploy/self-host/configs.mjs`); no request size limit (`MAX_REQUEST_BYTES` 0 means none) \|
120	121		\| `services/deployments` \| TS \| Workers for Platforms, REST API, KV `DOMAINS`, cron \| Runs with no API token: nothing deploys \|
121	122		\| `services/runner` \| TS \| Containers, Durable Objects, outbound interception, AI Gateway, cron \| Off: bound to the off Worker \|
122	123		\| `services/context` \| TS \| Vectorize, Workers AI, Queues \| Off: bound to the off Worker \|
⋯
345	346		\| Issues, pull requests, review \| On \| On \| — \|
346	347		\| Merge queue \| On \| Takes pull requests; testing and landing them needs sandboxes \| Phase 2 \|
347	348		\| Bringing a pull request up to date before it lands (catch-up) \| On \| Off: needs a sandbox \| Phase 2 \|
348		−	\| Clone pack cache \| R2 \| MinIO (`g1t-git-packs`) \| — \|
	349	+	\| Clone pack cache \| R2 \| RustFS (`g1t-git-packs`) \| — \|
349	350		\| Site search (FTS5) \| On \| On \| — \|
350	351		\| Email \| Email Sending \| Mailpit, logged \| SMTP relay \|
351	352		\| Webhooks, integrations \| On \| On (retries through `scheduler.mjs`) \| — \|
⋯
417	418		replication. The repositories get hosted g1t's nightly bundles
418	419		(docs/ARTIFACTS.md, R11) once the runner runs: the storage is already
419	420		configured (`BACKUP_STORE=s3`, the `g1t-backups` bucket that
420		−	`minio-setup` makes, `BACKUP_S3_BUCKET` to choose another), and the
	421	+	`storage-setup` makes, `BACKUP_S3_BUCKET` to choose another), and the
421	422		restore drill reads a copy of that bucket
422		−	(`mc mirror local/g1t-backups ./copy`, then
	423	+	(`aws --endpoint-url <S3_ENDPOINT> s3 sync s3://g1t-backups ./copy`, then
423	424		`node scripts/ops/backup-restore-drill.mjs --bundles ./copy --repo-id <id> --live <bare repository>`).
424	425
425	426		## 3. Phase 1: what works today
⋯
428	429
429	430		\| File \| What it is \|
430	431		\| --- \| --- \|
431		−	\| `docker-compose.yml` \| `g1t` (every core Worker in one workerd on 8787, and the API in a second on 8789), `status`, `gitstore` (bare repositories), `minio` and `minio-setup` (packages, backups and clone packs, with the packs' expiry rule), and `mailpit` (mail). Volumes: `g1t-data`, `g1t-git`, `g1t-packages`, `g1t-status`, `g1t-secrets`. MinIO no longer publishes `minio/minio` or `minio/mc` images; `MINIO_IMAGE` (default `pgsty/minio`, a community build with `mc` in it) is the server. \|
	432	+	\| `docker-compose.yml` \| `g1t` (every core Worker in one workerd on 8787, and the API in a second on 8789), `status`, `gitstore` (bare repositories), `rustfs` (S3-compatible storage for packages, backups and clone packs; `RUSTFS_IMAGE`, default `rustfs/rustfs:1.0.1`), `storage-setup` (the AWS CLI, `AWS_CLI_IMAGE`, default `amazon/aws-cli:2.37.10`: makes the three buckets and puts the packs' bucket's lifecycle rule, which expires `packs/` after 7 days and aborts multipart uploads unfinished after a day; RustFS's scanner applies it), and `mailpit` (mail). Volumes: `g1t-data`, `g1t-git`, `g1t-objects`, `g1t-status`, `g1t-secrets`. \|
432	433		\| `Dockerfile` \| Compiles the ten Rust services to WebAssembly with `worker-build`, as hosted does. Builds the site with React Router. The runtime image has Node, Wrangler, workerd and the built Workers. \|
433	434		\| `Dockerfile.dockerignore` \| Build-context rules for this image only (the root `.dockerignore` leaves out the site). \|
434	435		\| `start.sh` \| Makes the sealing keys once, writes the configs, applies migrations, runs `wrangler dev` with every config on `0.0.0.0:8787`, persisting to `/data/state`, and the API's `wrangler dev` on `0.0.0.0:8789` once the first answers. Starts `scheduler.mjs`. \|
⋯
438	439		\| `workers/artifacts/index.js` \| The `ARTIFACTS` binding, implemented against the git store. \|
439	440		\| `workers/mail/index.js` \| The `EMAIL` binding: logs, then sends to Mailpit. \|
440	441		\| `workers/off/index.js` \| The runner and the context hub when they are off. \|
441		−	\| `smoke.sh` \| The end-to-end check, including the API, pull requests, the merge queue and (with `SCHEDULER_ONCE`) every cron handler. \|
	442	+	\| `smoke.sh` \| The end-to-end check, including the clone pack cache, the API, an npm package published and installed, pull requests, the merge queue and (with `SCHEDULER_ONCE`) every cron handler. \|
442	443
443	444		Workers running: the site; identity, repos, work, events, projects, search,
444	445		billing, security, actions, webhooks, integrations, packages and
⋯
460	461		The workspace context page answered 403 from the off stand-in, as
461	462		intended.
462	463		2. With Docker Compose (2026-10-07, `docker compose up --build`, with
463		−	`API_PORT=18789` because 8789 was taken on this machine). `smoke.sh`
	464	+	`API_PORT=18789` because 8789 was taken on this machine; the store is
	465	+	RustFS 1.0.1, its buckets made by `storage-setup`). `smoke.sh`
464	466		passed every step: the ones above; a second clone answered from the pack
465		−	cache (`Server-Timing: pack;desc=hit`), with the packs in MinIO's
466		−	`g1t-git-packs` and its 7-day rule in place; an access token made in the
467		−	site; `GET /user`, the API index (`mcp_url`, `git_url`), the OAuth
	467	+	cache (`Server-Timing: pack;desc=hit`), with the packs in RustFS's
	468	+	`g1t-git-packs` and its lifecycle rule in place (packs expire after 7
	469	+	days, unfinished uploads are aborted after 1); an access token made in the
	470	+	site; an npm package published to the installation's registry and
	471	+	installed back, its tarball in `g1t-packages`; `GET /user`, the API index (`mcp_url`, `git_url`), the OAuth
468	472		metadata (`issuer` the API's address, `authorization_endpoint` on the
469	473		site), MCP's 401 challenge and `tools/list`; a pull request from a branch
470	474		and one from a fork (`create_pull_request` without a branch: a fork in
⋯
473	477		shown `waiting`, taken out (`unqueue`), the queue turned off and the
474	478		pull request merged; and `scheduler.mjs --once`, every handler `ok`.
475	479		The repository page's clone box and MCP line named the installation's
476		−	own addresses, with no social card tags.
477		−	3. The clone pack cache against MinIO without the stack.
478		−	`node services/repos/dev/clone-check.mjs --s3` (MinIO in Docker):
479		−	misses then hits for full and shallow clones over protocol v2 and v0, a
480		−	miss after the refs version moves, five whole packs in the bucket (12 MB
481		−	each, so uploaded in parts), no unfinished upload, and the expiry rule.
	480	+	own addresses, with no social card tags. By hand against the same
	481	+	stack: `docker push` and `docker pull` of an image with a 20 MB layer
	482	+	(uploaded in 10 MiB parts), the layer read back from `/v2/.../blobs/`
	483	+	with a matching digest, the `g1t-backups` bucket present (empty: the
	484	+	runner cuts bundles), no unfinished upload in any bucket, and the
	485	+	guide's upgrade copy from an old MinIO volume into RustFS.
	486	+	3. The clone pack cache against RustFS without the stack.
	487	+	`node services/repos/dev/clone-check.mjs --s3` (RustFS in Docker, the
	488	+	bucket and lifecycle rule made with the AWS CLI): misses then hits for
	489	+	full and shallow clones over protocol v2 and v0, a miss after the refs
	490	+	version moves, five whole packs in the bucket (12 MB each, multipart
	491	+	objects of 3 parts), each reading back at its listed size, no
	492	+	unfinished upload, and both lifecycle rules.
482	493
483	494		### Not verified, or not working yet
484	495
⋯
521	532		a registry directory, a development feature like the rest. If the API
522	533		starts and a binding says `[not connected]`, restart the container. The
523	534		phase 2 launcher serves both from one workerd.
524		−	- The MinIO image. MinIO stopped publishing `minio/minio` and
525		−	`minio/mc`. The compose file uses a community build (`MINIO_IMAGE`,
526		−	`pgsty/minio`); any S3-compatible store can take its place.
	535	+	- The object store. The compose file runs RustFS (Apache-2.0),
	536	+	pinned to a release tag, and makes its buckets with the AWS CLI
	537	+	(Apache-2.0). Any S3-compatible store can take its place
	538	+	(`S3_ENDPOINT`), given the same buckets and the packs' lifecycle rule.
	539	+	Installations started before 2026-10-07 kept these files in MinIO, in
	540	+	the `g1t-packages` volume; the guide's "Upgrade" section copies them
	541	+	across.
527	542		- Cron goes through Wrangler's local API. `scheduler.mjs` asks
528	543		`/cdn-cgi/local/explorer/api/local/scheduled`, a development endpoint
529	544		that may change between Wrangler releases (pinned by the lockfile).

+13−2View BlameViewed

1056	1056		};
1057	1057
1058	1058		/** All of g1t: money in (usage and the plan) against every cost. */
1059		−	export type OverallMargin = { usageMicros: number; plansMicros: number; costMicros: number; marginMicros: number; marginPercent: number \| null };
	1059	+	export type OverallMargin = {
	1060	+	usageMicros: number;
	1061	+	plansMicros: number;
	1062	+	costMicros: number;
	1063	+	marginMicros: number;
	1064	+	marginPercent: number \| null;
	1065	+	/** Of costMicros, what went on usage g1t gave away on purpose: comped workspaces, free periods, the trial and the open-source pool. */
	1066	+	givenMicros?: number;
	1067	+	/** Money in against costMicros - givenMicros. */
	1068	+	soldMarginMicros?: number;
	1069	+	soldMarginPercent?: number \| null;
	1070	+	};
1060	1071
1061	1072		/** A count, cost or leak that does not add up. */
1062	1073		export type CostDrift = {
⋯
1117	1128		appliedAt: string \| null;
1118	1129		};
1119	1130
1120		−	export type WorkspaceCost = { workspace: string; costMicros: number; revenueMicros: number; internal: boolean };
	1131	+	export type WorkspaceCost = { workspace: string; costMicros: number; revenueMicros: number; givenMicros?: number; internal: boolean };
1121	1132
1122	1133		export type CostLineSummary = {
1123	1134		product: string;

+2−2View BlameViewed

21	21		// backup's.
22	22		// --repo-id <id> the repository by id (no database needed with --bundles).
23	23		// --bundles <dir> read the bucket from a local copy (`backups/<id>/...`
24		−	// under it, as `mc mirror` or `rclone copy` leave it)
25		−	// instead of R2, e.g. a self-hosted MinIO's.
	24	+	// under it, as `aws s3 sync` or `rclone copy` leave it)
	25	+	// instead of R2, e.g. a self-hosted store's.
26	26		// --live <url or path> the live repository to compare with; default
27	27		// https://g1t.sh/<workspace>/<name>.git.
28	28		// --keep keep the temporary directory, and say where it is.

+6−0View BlameViewed

1	+	-- What g1t gave away on purpose: the part of each day's cost that went on
2	+	-- usage nobody paid for (g1t's own comped workspaces, a free period, the
3	+	-- trial and the open-source pool). Sudo measures margin on what was sold,
4	+	-- with this shown beside it, so comping does not read as lost money.
5	+	ALTER TABLE margin_days ADD COLUMN given_micros INTEGER NOT NULL DEFAULT 0;
6	+	ALTER TABLE workspace_costs ADD COLUMN given_micros INTEGER NOT NULL DEFAULT 0;

+91−49View BlameViewed

76	76		/// says they are the same units.
77	77		pub cf_quantity: f64,
78	78		pub own_quantity: f64,
	79	+	/// Of `cost()`, what went on usage g1t gave away (the workspaces'
	80	+	/// `WorkspaceDay::given`, added up).
	81	+	pub given_micros: i64,
79	82		}
80	83
81	84		impl ProductDay {
⋯
116	119		pub value: i64,
117	120		pub cash: i64,
118	121		pub cost: i64,
	122	+	/// Of `value`, what g1t gave away: all of it for g1t's own (comped)
	123	+	/// workspaces and in a free period, else what the trial and the pools
	124	+	/// paid. The Team plan's credit was paid for, so it is not given.
	125	+	pub given: i64,
119	126		}
120	127
121	128		/// One workspace's share of a product's cost on one day.
⋯
129	136		pub revenue: i64,
130	137		/// What its usage was priced at, whoever paid for it.
131	138		pub value: i64,
	139	+	/// Of `cost`, the part g1t gave away: the cost times the share of the
	140	+	/// workspace's usage that day that g1t paid for (see `UsageRow::given`).
	141	+	pub given: i64,
132	142		}
133	143
134	144		fn micros(dollars: f64) -> i64 {
⋯
206	216		let mut revenue: BTreeMap<(String, String, String), i64> = BTreeMap::new();
207	217		let mut valued: BTreeMap<(String, String, String), i64> = BTreeMap::new();
208	218		let mut active: BTreeMap<String, Vec<(String, f64)>> = BTreeMap::new();
	219	+	let mut gave: BTreeMap<(String, String), (i64, i64)> = BTreeMap::new();
209	220		for u in usage {
	221	+	let g = gave.entry((u.day.clone(), u.workspace.clone())).or_default();
	222	+	g.0 += u.given;
	223	+	g.1 += u.value;
210	224		let bucket = bucket_of(&u.key);
211	225		let key = (u.day.clone(), bucket.clone());
212	226		let row = days.entry(key.clone()).or_insert_with(\|\| entry(&key.0, &key.1));
⋯
238	252		}
239	253		}
240	254		let keys: BTreeSet<(String, String, String)> = shares.keys().chain(revenue.keys()).cloned().collect();
241		−	let workspaces = keys
	255	+	let workspaces: Vec<WorkspaceDay> = keys
242	256		.into_iter()
243		−	.map(\|(day, workspace, bucket)\| WorkspaceDay {
244		−	cost: shares.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0),
245		−	revenue: revenue.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0),
246		−	value: valued.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0),
247		−	day,
248		−	workspace,
249		−	bucket,
	257	+	.map(\|(day, workspace, bucket)\| {
	258	+	let cost = shares.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0);
	259	+	// The day's share given away applies to every bucket, so a
	260	+	// comped workspace's part of running g1t is given too.
	261	+	let given = match gave.get(&(day.clone(), workspace.clone())) {
	262	+	Some(&(given, value)) if value > 0 => (cost as i128 * given.clamp(0, value) as i128 / value as i128) as i64,
	263	+	_ => 0,
	264	+	};
	265	+	WorkspaceDay {
	266	+	cost,
	267	+	revenue: revenue.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0),
	268	+	value: valued.get(&(day.clone(), workspace.clone(), bucket.clone())).copied().unwrap_or(0),
	269	+	given,
	270	+	day,
	271	+	workspace,
	272	+	bucket,
	273	+	}
250	274		})
251	275		.collect();
	276	+	for w in &workspaces {
	277	+	if let Some(row) = days.get_mut(&(w.day.clone(), w.bucket.clone())) {
	278	+	row.given_micros += w.given;
	279	+	}
	280	+	}
252	281		(days.into_values().collect(), workspaces)
253	282		}
254	283
⋯
269	298		};
270	299		let (cost, charge) = ((cost - before_cost).max(0), (charge - before_charge).max(0));
271	300		if cost > 0 \|\| charge > 0 {
272		−	out.push(UsageRow { day: day.clone(), workspace: workspace.clone(), key: source.clone(), value: charge, cash: charge, cost });
	301	+	out.push(UsageRow { day: day.clone(), workspace: workspace.clone(), key: source.clone(), value: charge, cash: charge, cost, given: 0 });
273	302		}
274	303		previous = Some(snap);
275	304		}
⋯
566	595		cash_micros: i64,
567	596		cf_quantity: f64,
568	597		own_quantity: f64,
	598	+	#[serde(default)]
	599	+	given_micros: Option<i64>,
569	600		}
570	601
571	602		impl From<MarginRow> for ProductDay {
⋯
579	610		cash_micros: r.cash_micros,
580	611		cf_quantity: r.cf_quantity,
581	612		own_quantity: r.own_quantity,
	613	+	given_micros: r.given_micros.unwrap_or(0),
582	614		}
583	615		}
584	616		}
⋯
655	687		own_provider: i64,
656	688		cash: Option<i64>,
657	689		drawn: Option<i64>,
	690	+	credit: Option<i64>,
658	691		cost: Option<i64>,
659	692		}
660	693		let charged_here = crate::storage::CHARGED_HERE.iter().map(\|s\| format!("'{s}'")).collect::<Vec<_>>().join(", ");
⋯
668	701		CASE WHEN billed_to = 'workspace' THEN 1 ELSE 0 END AS own_provider,
669	702		-SUM(amount_micros) AS cash,
670	703		SUM(COALESCE(credit_micros, 0) + COALESCE(trial_micros, 0) + COALESCE(oss_micros, 0) + COALESCE(given_micros, 0)) AS drawn,
	704	+	SUM(COALESCE(credit_micros, 0)) AS credit,
671	705		SUM(COALESCE(cost_micros, 0)) AS cost
672	706		FROM ledger
673	707		WHERE kind = 'usage' AND created_at >= ?1 AND created_at <= ?2 AND COALESCE(task, '') NOT IN ({charged_here})
⋯
678	712		.all()
679	713		.await?
680	714		.results::<Row>()?;
	715	+	let mut internal = BTreeSet::new();
681	716		let mut out: Vec<UsageRow> = rows
682	717		.into_iter()
683	718		.map(\|r\| {
⋯
687	722		let cash = r.cash.unwrap_or(0);
688	723		let paid = cash + r.drawn.unwrap_or(0);
689	724		let value = usage_value(r.internal == 1, cost, paid, self.margin_percent);
690		−	UsageRow { day: r.day, workspace: r.workspace, key: r.key, value, cash, cost }
	725	+	let given = if r.internal == 1 { value } else { (value - cash - r.credit.unwrap_or(0)).max(0) };
	726	+	if r.internal == 1 {
	727	+	internal.insert(r.workspace.clone());
	728	+	}
	729	+	UsageRow { day: r.day, workspace: r.workspace, key: r.key, value, cash, cost, given }
691	730		})
692	731		.collect();
693	732		// Month-end sources, from their daily snapshots.
⋯
710	749		.filter(\|s\| crate::storage::CHARGED_HERE.contains(&s.source.as_str()) \|\| s.source == "domains")
711	750		.map(\|s\| (s.day, s.workspace, s.source, s.cost_micros, s.charge_micros))
712	751		.collect::<Vec<_>>();
713		−	out.extend(pending_deltas(&snaps).into_iter().filter(\|u\| u.day.as_str() >= since));
	752	+	out.extend(pending_deltas(&snaps).into_iter().filter(\|u\| u.day.as_str() >= since).map(\|mut u\| {
	753	+	if internal.contains(&u.workspace) {
	754	+	u.given = u.value;
	755	+	}
	756	+	u
	757	+	}));
714	758		// The plan's price, spread over the 30 days it pays for, so a month's
715	759		// payment does not read as one very good day and 29 bad ones.
716	760		#[derive(Deserialize)]
⋯
732	776		for p in plans {
733	777		for (day, micros) in spread(&p.day, p.micros.unwrap_or(0), PLAN_DAYS) {
734	778		if day.as_str() >= since && day.as_str() <= until {
735		−	out.push(UsageRow { day, workspace: p.workspace.clone(), key: "plan".into(), value: micros, cash: micros, cost: 0 });
	779	+	out.push(UsageRow { day, workspace: p.workspace.clone(), key: "plan".into(), value: micros, cash: micros, cost: 0, given: 0 });
736	780		}
737	781		}
738	782		}
⋯
785	829		statements.push(
786	830		self.db
787	831		.prepare(
788		−	"INSERT OR REPLACE INTO margin_days (day, bucket, cf_cost_micros, own_cost_micros, value_micros, cash_micros, cf_quantity, own_quantity, computed_at)
789		−	VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
	832	+	"INSERT OR REPLACE INTO margin_days (day, bucket, cf_cost_micros, own_cost_micros, value_micros, cash_micros, cf_quantity, own_quantity, given_micros, computed_at)
	833	+	VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
790	834		)
791	835		.bind(&[
792	836		d.day.as_str().into(),
⋯
797	841		(d.cash_micros as f64).into(),
798	842		d.cf_quantity.into(),
799	843		d.own_quantity.into(),
	844	+	(d.given_micros as f64).into(),
800	845		now.as_str().into(),
801	846		])?,
802	847		);
⋯
808	853		for w in chunk {
809	854		statements.push(
810	855		self.db
811		−	.prepare("INSERT OR REPLACE INTO workspace_costs (day, workspace, bucket, cost_micros, revenue_micros, value_micros) VALUES (?, ?, ?, ?, ?, ?)")
	856	+	.prepare("INSERT OR REPLACE INTO workspace_costs (day, workspace, bucket, cost_micros, revenue_micros, value_micros, given_micros) VALUES (?, ?, ?, ?, ?, ?, ?)")
812	857		.bind(&[
813	858		w.day.as_str().into(),
814	859		w.workspace.as_str().into(),
⋯
816	861		(w.cost as f64).into(),
817	862		(w.revenue as f64).into(),
818	863		(w.value as f64).into(),
	864	+	(w.given as f64).into(),
819	865		])?,
820	866		);
821	867		}
⋯
988	1034		let mut all: BTreeMap<String, (i64, i64)> = BTreeMap::new();
989	1035		for d in &days {
990	1036		let overall = all.entry(d.day.clone()).or_default();
	1037	+	// What g1t gave away (comped workspaces, free periods, the
	1038	+	// trial and the pools) is a budget it chose to spend, watched on
	1039	+	// its own (budget.rs): not part of whether what is sold pays.
991	1040		overall.0 += d.cash_micros;
992		−	overall.1 += d.cost();
	1041	+	overall.1 += (d.cost() - d.given_micros).max(0);
993	1042		if !OVERHEAD.contains(&d.bucket.as_str()) && d.bucket != UNMAPPED {
994	1043		by.entry(d.bucket.clone()).or_default().push((d.day.clone(), d.value_micros, d.cost()));
995	1044		}
⋯
1004	1053		format!("{}: margin under {floor:.0}% for {n} days running, as low as {worst:.1}%.", costs::bucket_title(bucket)),
1005	1054		from,
1006	1055		));
1007		−	}
1008		−	}
1009		−	// Comped workspaces' share is a budget g1t chose to spend, watched
1010		−	// on its own (budget.rs): not part of whether what is sold pays.
1011		−	for (day, cost) in self.comped_costs(&since, until).await? {
1012		−	if let Some(overall) = all.get_mut(&day) {
1013		−	overall.1 = (overall.1 - cost).max(0);
1014	1056		}
1015	1057		}
1016	1058		let series: Vec<(String, i64, i64)> = all.into_iter().map(\|(day, (revenue, cost))\| (day, revenue, cost)).collect();
⋯
1112	1154
1113	1155		/// Workspaces costing g1t more than they pay over 30 days, not g1t's own.
1114	1156		/// Each day's cost shared out to comped workspaces.
1115		−	async fn comped_costs(&self, since: &str, until: &str) -> Result<Vec<(String, i64)>> {
1116		−	#[derive(Deserialize)]
1117		−	struct Row {
1118		−	day: String,
1119		−	cost: Option<i64>,
1120		−	}
1121		−	Ok(self
1122		−	.db
1123		−	.prepare(format!(
1124		−	"SELECT day, SUM(cost_micros) AS cost FROM workspace_costs
1125		−	WHERE day >= ?1 AND day <= ?2 AND workspace IN ({}) GROUP BY day",
1126		−	crate::sales::INTERNAL_SQL
1127		−	))
1128		−	.bind(&[since.into(), until.into()])?
1129		−	.all()
1130		−	.await?
1131		−	.results::<Row>()?
1132		−	.into_iter()
1133		−	.map(\|r\| (r.day, r.cost.unwrap_or(0)))
1134		−	.collect())
1135		−	}
1136		−
1137	1157		async fn workspace_anomalies(&self, until: &str, settings: &CostSettings) -> Result<Vec<(String, i64, i64)>> {
1138	1158		#[derive(Deserialize)]
1139	1159		struct Row {
⋯
1314	1334		p.value_micros += d.value_micros;
1315	1335		p.cost_micros += d.cost();
1316	1336		overall.cost_micros += d.cost();
	1337	+	overall.given_micros += d.given_micros;
1317	1338		if d.bucket == "platform" {
1318	1339		overall.plans_micros += d.cash_micros;
1319	1340		} else {
⋯
1327	1348		let revenue = overall.usage_micros + overall.plans_micros;
1328	1349		overall.margin_micros = revenue - overall.cost_micros;
1329	1350		overall.margin_percent = margin_percent(revenue, overall.cost_micros);
	1351	+	let sold = (overall.cost_micros - overall.given_micros).max(0);
	1352	+	overall.sold_margin_micros = revenue - sold;
	1353	+	overall.sold_margin_percent = margin_percent(revenue, sold);
1330	1354		let mut products: Vec<ProductMargin> = products.into_values().collect();
1331	1355		products.sort_by_key(\|p\| std::cmp::Reverse(p.cost_micros.max(p.value_micros)));
1332	1356
⋯
1364	1388		workspace: String,
1365	1389		cost: Option<i64>,
1366	1390		revenue: Option<i64>,
	1391	+	given: Option<i64>,
1367	1392		internal: i64,
1368	1393		}
1369	1394		let top_workspaces = self
1370	1395		.db
1371	1396		.prepare(format!(
1372		−	"SELECT workspace, SUM(cost_micros) AS cost, SUM(revenue_micros) AS revenue,
	1397	+	"SELECT workspace, SUM(cost_micros) AS cost, SUM(revenue_micros) AS revenue, SUM(given_micros) AS given,
1373	1398		CASE WHEN workspace IN ({}) THEN 1 ELSE 0 END AS internal
1374	1399		FROM workspace_costs WHERE day >= ?1 AND day <= ?2 GROUP BY workspace ORDER BY cost DESC LIMIT 15",
1375	1400		crate::sales::INTERNAL_SQL
⋯
1379	1404		.await?
1380	1405		.results::<Top>()?
1381	1406		.into_iter()
1382		−	.map(\|t\| WorkspaceCost { workspace: t.workspace, cost_micros: t.cost.unwrap_or(0), revenue_micros: t.revenue.unwrap_or(0), internal: t.internal == 1 })
	1407	+	.map(\|t\| WorkspaceCost { workspace: t.workspace, cost_micros: t.cost.unwrap_or(0), revenue_micros: t.revenue.unwrap_or(0), given_micros: t.given.unwrap_or(0), internal: t.internal == 1 })
1383	1408		.collect();
1384	1409
1385	1410		#[derive(Deserialize)]
⋯
1534	1559		}
1535	1560
1536	1561		fn usage(day: &str, workspace: &str, key: &str, value: i64, cash: i64, cost: i64) -> UsageRow {
1537		−	UsageRow { day: day.into(), workspace: workspace.into(), key: key.into(), value, cash, cost }
	1562	+	UsageRow { day: day.into(), workspace: workspace.into(), key: key.into(), value, cash, cost, given: 0 }
1538	1563		}
1539	1564
1540	1565		#[test]
⋯
1626	1651		}
1627	1652
1628	1653		fn day(bucket: &str, cf: i64, own: i64, value: i64, cfq: f64, ownq: f64) -> ProductDay {
1629		−	ProductDay { day: "2026-10-15".into(), bucket: bucket.into(), cf_cost_micros: cf, own_cost_micros: own, value_micros: value, cash_micros: value, cf_quantity: cfq, own_quantity: ownq }
	1654	+	ProductDay { day: "2026-10-15".into(), bucket: bucket.into(), cf_cost_micros: cf, own_cost_micros: own, value_micros: value, cash_micros: value, cf_quantity: cfq, own_quantity: ownq, given_micros: 0 }
1630	1655		}
1631	1656
1632	1657		#[test]
⋯
1681	1706		}
1682	1707
1683	1708		#[test]
	1709	+	fn what_g1t_gives_away_is_kept_apart_from_what_it_sells() {
	1710	+	let map = BTreeMap::new();
	1711	+	// A comped workspace (all of it given), one in its trial (half paid
	1712	+	// by the trial) and one paying in cash, all on models.
	1713	+	let mut comped = usage("2026-10-15", "flagon", "agent", 1_200_000, 0, 1_000_000);
	1714	+	comped.given = comped.value;
	1715	+	let mut trial = usage("2026-10-15", "acme", "agent", 1_200_000, 600_000, 1_000_000);
	1716	+	trial.given = 600_000;
	1717	+	let paying = usage("2026-10-15", "beta", "agent", 1_200_000, 1_200_000, 1_000_000);
	1718	+	let (days, workspaces) = fold(&[], &map, &[], &[], &[comped, trial, paying]);
	1719	+	let models = days.iter().find(\|d\| d.bucket == "models").unwrap();
	1720	+	assert_eq!((models.cost(), models.given_micros), (3_000_000, 1_500_000));
	1721	+	let given = \|w: &str\| workspaces.iter().find(\|x\| x.workspace == w).unwrap().given;
	1722	+	assert_eq!((given("flagon"), given("acme"), given("beta")), (1_000_000, 500_000, 0));
	1723	+	}
	1724	+
	1725	+	#[test]
1684	1726		fn a_workspace_that_costs_more_than_it_pays_is_flagged() {
1685	1727		let rows = vec![("acme".to_string(), 5_000_000, 1_000_000), ("beta".to_string(), 900_000, 0), ("gamma".to_string(), 2_000_000, 3_000_000)];
1686	1728		let found = anomalies(&rows, 1.0, 1_000_000);

+1−1View BlameViewed

1	1		//! Where packages' files are kept: the `BlobStore` port (crates/blobstore),
2		−	//! with R2 behind it on Cloudflare and any S3-compatible storage (MinIO in
	2	+	//! with R2 behind it on Cloudflare and any S3-compatible storage (RustFS in
3	3		//! the compose file) when self-hosted. BLOB_STORE chooses: `r2` (the
4	4		//! default) or `s3`.
5	5		//!

+56−33View BlameViewed

11	11		// cd services/repos && node ../../scripts/build-rust-worker.mjs
12	12		// node dev/clone-check.mjs
13	13		//
14		−	// With `--s3`, packs are kept in MinIO instead of local R2, as a
15		−	// self-hosted installation keeps them (PACK_STORE=s3): it starts MinIO in
16		−	// Docker, makes the `g1t-git-packs` bucket with the same expiry rule the
17		−	// compose file gives it, and checks that what was kept is there, whole,
18		−	// with no upload left unfinished.
	14	+	// With `--s3`, packs are kept in RustFS instead of local R2, as a
	15	+	// self-hosted installation keeps them (PACK_STORE=s3): it starts RustFS in
	16	+	// Docker, makes the `g1t-git-packs` bucket with the same lifecycle rule the
	17	+	// compose file gives it (with the AWS CLI, as the compose file does), and
	18	+	// checks that what was kept is there, whole, with no upload left
	19	+	// unfinished.
19	20		//
20	21		// node dev/clone-check.mjs --s3
21	22		//
22		−	// GITSTORE_PORT, REPOS_PORT and MINIO_PORT move it off 8799, 8791 and 9010.
	23	+	// GITSTORE_PORT, REPOS_PORT and S3_PORT move it off 8799, 8791 and 9010.
	24	+	// RUSTFS_IMAGE and AWS_CLI_IMAGE choose other images.
23	25		//
24	26		// Needs node, git (with git-http-backend) and the repository's npm
25	27		// packages; with `--s3`, Docker too.
⋯
56	58		);
57	59		const KEY = "acme--rocket";
58	60		const children = [];
59		−	// --s3: packs in MinIO (see the top).
	61	+	// --s3: packs in RustFS (see the top). The images are the compose file's.
60	62		const S3 = process.argv.includes("--s3");
61		−	const MINIO = "g1t-clone-check-minio";
62		−	const MINIO_PORT = process.env.MINIO_PORT ?? "9010";
63		−	const MINIO_USER = "g1t";
64		−	const MINIO_PASSWORD = "g1t-clone-check-secret";
	63	+	const STORAGE = "g1t-clone-check-rustfs";
	64	+	const S3_PORT = process.env.S3_PORT ?? "9010";
	65	+	const S3_USER = "g1t";
	66	+	const S3_PASSWORD = "g1t-clone-check-secret";
	67	+	const RUSTFS_IMAGE = process.env.RUSTFS_IMAGE ?? "rustfs/rustfs:1.0.1";
	68	+	const AWS_CLI_IMAGE = process.env.AWS_CLI_IMAGE ?? "amazon/aws-cli:2.37.10";
65	69		const PACKS_BUCKET = "g1t-git-packs";
66		−	/** `mc` inside the MinIO container, against itself. */
67		−	const mc = (args, options = {}) => run("docker", ["exec", MINIO, "mc", ...args], options);
	70	+	/** The AWS CLI's `s3api`, in a container on the store's network, against it. */
	71	+	const s3api = (args, options = {}) =>
	72	+	run("docker", [
	73	+	"run", "--rm", "--network", `container:${STORAGE}`,
	74	+	"-e", `AWS_ACCESS_KEY_ID=${S3_USER}`, "-e", `AWS_SECRET_ACCESS_KEY=${S3_PASSWORD}`, "-e", "AWS_DEFAULT_REGION=us-east-1",
	75	+	AWS_CLI_IMAGE, "--endpoint-url", "http://localhost:9000", "--output", "json", "s3api", ...args,
	76	+	], options);
	77	+	/** The same, its answer parsed. */
	78	+	const s3json = (args) => JSON.parse(s3api(args).stdout.trim() \|\| "{}");
68	79		// Wrangler from the repository's packages, run with node: no shell to quote for.
69	80		const WRANGLER = join(dirname(createRequire(join(service, "package.json")).resolve("wrangler/package.json")), "bin/wrangler.js");
70	81
⋯
163	174		});
164	175		sql("INSERT INTO repos (id, namespace, name, is_private, owner_id, default_branch, refs_version) VALUES ('rep_rocket', 'acme', 'rocket', 0, 'usr_dev', 'main', 1)");
165	176
166		−	// MinIO, with the bucket and expiry rule deploy/self-host/docker-compose.yml makes.
	177	+	// RustFS, with the bucket and lifecycle rule deploy/self-host/docker-compose.yml makes.
167	178		const s3Vars = [];
168	179		if (S3) {
169		−	run("docker", ["rm", "-f", MINIO], { allowFail: true });
	180	+	run("docker", ["rm", "-f", STORAGE], { allowFail: true });
170	181		run("docker", [
171		−	"run", "-d", "--rm", "--name", MINIO, "-p", `${MINIO_PORT}:9000`,
172		−	"-e", `MINIO_ROOT_USER=${MINIO_USER}`, "-e", `MINIO_ROOT_PASSWORD=${MINIO_PASSWORD}`,
173		−	process.env.MINIO_IMAGE ?? "pgsty/minio:latest", "server", "/data",
	182	+	"run", "-d", "--rm", "--name", STORAGE, "-p", `${S3_PORT}:9000`,
	183	+	"-e", `RUSTFS_ACCESS_KEY=${S3_USER}`, "-e", `RUSTFS_SECRET_KEY=${S3_PASSWORD}`, "-e", "RUSTFS_CONSOLE_ENABLE=false",
	184	+	RUSTFS_IMAGE,
	185	+	]);
	186	+	await waitFor(`http://localhost:${S3_PORT}/health/ready`, "RustFS");
	187	+	s3api(["create-bucket", "--bucket", PACKS_BUCKET]);
	188	+	s3api([
	189	+	"put-bucket-lifecycle-configuration", "--bucket", PACKS_BUCKET, "--lifecycle-configuration",
	190	+	JSON.stringify({
	191	+	Rules: [
	192	+	{ ID: "expire-packs", Status: "Enabled", Filter: { Prefix: "packs/" }, Expiration: { Days: 7 } },
	193	+	{ ID: "abort-unfinished-uploads", Status: "Enabled", Filter: { Prefix: "" }, AbortIncompleteMultipartUpload: { DaysAfterInitiation: 1 } },
	194	+	],
	195	+	}),
174	196		]);
175		−	await waitFor(`http://localhost:${MINIO_PORT}/minio/health/ready`, "MinIO");
176		−	mc(["alias", "set", "local", "http://localhost:9000", MINIO_USER, MINIO_PASSWORD]);
177		−	mc(["mb", "--ignore-existing", `local/${PACKS_BUCKET}`]);
178		−	mc(["ilm", "rule", "add", "--prefix", "packs/", "--expire-days", "7", `local/${PACKS_BUCKET}`]);
179	197		for (const [name, value] of Object.entries({
180	198		PACK_STORE: "s3",
181	199		PACK_S3_BUCKET: PACKS_BUCKET,
182		−	S3_ENDPOINT: `http://localhost:${MINIO_PORT}`,
	200	+	S3_ENDPOINT: `http://localhost:${S3_PORT}`,
183	201		S3_REGION: "us-east-1",
184		−	S3_ACCESS_KEY_ID: MINIO_USER,
185		−	S3_SECRET_ACCESS_KEY: MINIO_PASSWORD,
	202	+	S3_ACCESS_KEY_ID: S3_USER,
	203	+	S3_SECRET_ACCESS_KEY: S3_PASSWORD,
186	204		})) {
187	205		s3Vars.push("--var", `${name}:${value}`);
188	206		}
⋯
209	227		if (S3) {
210	228		// Fills finish after git has its answer: give the last one a moment.
211	229		await new Promise((resolve) => setTimeout(resolve, 3000));
212		−	const listed = mc(["ls", "--recursive", "--json", `local/${PACKS_BUCKET}/packs/`]).stdout.trim().split("\n").filter(Boolean).map((line) => JSON.parse(line));
213		−	const sizes = listed.map((entry) => entry.size);
	230	+	const listed = s3json(["list-objects-v2", "--bucket", PACKS_BUCKET, "--prefix", "packs/"]).Contents ?? [];
	231	+	const sizes = listed.map((entry) => entry.Size);
214	232		// Nine clones: four kinds twice, each kept once, and one more after the refs moved.
215		−	check("the packs are in MinIO, one per distinct clone", listed.length === 5, `${listed.length}: ${sizes.join(", ")}`);
	233	+	check("the packs are in RustFS, one per distinct clone", listed.length === 5, `${listed.length}: ${sizes.join(", ")}`);
216	234		check("the full clone's pack went up in parts", sizes.some((size) => size > 5 * 1024 * 1024), `largest ${Math.max(...sizes)}`);
217		−	const incomplete = mc(["ls", "--recursive", "--incomplete", `local/${PACKS_BUCKET}`]).stdout.trim();
218		−	check("no upload is left unfinished", incomplete === "", incomplete);
219		−	const rules = mc(["ilm", "rule", "ls", "--json", `local/${PACKS_BUCKET}`]).stdout;
220		−	check("the bucket expires packs after 7 days", /"Days":\s*7/.test(rules) && rules.includes("packs/"));
	235	+	// A multipart object's ETag ends in -<number of parts>.
	236	+	check("the large pack is one multipart object", listed.some((entry) => /-\d+"?$/.test(entry.ETag ?? "")), listed.map((entry) => entry.ETag).join(", "));
	237	+	const short = listed.filter((entry) => s3json(["head-object", "--bucket", PACKS_BUCKET, "--key", entry.Key]).ContentLength !== entry.Size);
	238	+	check("every pack reads back whole", short.length === 0, short.map((entry) => entry.Key).join(", "));
	239	+	const incomplete = s3json(["list-multipart-uploads", "--bucket", PACKS_BUCKET]).Uploads ?? [];
	240	+	check("no upload is left unfinished", incomplete.length === 0, incomplete.map((upload) => upload.Key).join(", "));
	241	+	const rules = s3json(["get-bucket-lifecycle-configuration", "--bucket", PACKS_BUCKET]).Rules ?? [];
	242	+	check("the bucket expires packs after 7 days", rules.some((rule) => rule.Expiration?.Days === 7 && rule.Filter?.Prefix === "packs/"), JSON.stringify(rules));
	243	+	check("the bucket aborts uploads unfinished after a day", rules.some((rule) => rule.AbortIncompleteMultipartUpload?.DaysAfterInitiation === 1));
221	244		}
222	245		} catch (error) {
223	246		console.error(error);
⋯
227	250		if (process.platform === "win32") spawnSync("taskkill", ["/pid", String(child.pid), "/t", "/f"], { stdio: "ignore" });
228	251		else child.kill();
229	252		}
230		−	if (S3) run("docker", ["rm", "-f", MINIO], { allowFail: true });
	253	+	if (S3) run("docker", ["rm", "-f", STORAGE], { allowFail: true });
231	254		try {
232	255		rmSync(work, { recursive: true, force: true });
233	256		} catch {}

+1−1View BlameViewed

7	7		//! again after [`Settings::full_every`] incremental ones, so a restore
8	8		//! never reads a long chain. Bundles and a manifest that lists the chain
9	9		//! are kept in object storage through the `BlobStore` port: the BACKUPS R2
10		−	//! bucket hosted, any S3-compatible store (MinIO in the compose file)
	10	+	//! bucket hosted, any S3-compatible store (RustFS in the compose file)
11	11		//! self-hosted, as BACKUP_STORE says.
12	12		//!
13	13		//! ```text

+1−1View BlameViewed

89	89		/// Where packs are kept, by the names of the service's bindings and
90	90		/// variables: the `GIT_PACKS` R2 bucket on Cloudflare or, when PACK_STORE
91	91		/// is `s3`, the bucket PACK_S3_BUCKET names on the installation's
92		−	/// S3-compatible store (`g1t-git-packs` on MinIO in the self-host compose
	92	+	/// S3-compatible store (`g1t-git-packs` on RustFS in the self-host compose
93	93		/// file). On either an object exists only once it is whole: a `put` makes
94	94		/// it in one write, and a multipart upload is nothing anyone can read
95	95		/// until it is completed.