flagon-io/g1t

public

The git guide says g1t asks for each push whole, what a thin one costs and how thin;desc= shows it, and its stray repos row is back in the steps table; the Artifacts and performance notes mark step 1 of the push speed-up built, with what read should fall to and why the 437-object push answered 503: every base asked for at once, failures retried and counted against the store's breaker until every read after them was turned away as busy.

syntaqxcommitted 1d agoParente9b14b1Browse files

293	293		\| `upload` \| Only for a push: handing it to the git store and its answer \|
294	294		\| `refs` \| Only for a push: recording that the repository's refs changed \|
295	295		\| `total` \| Everything g1t did \|
	296	+	\| `repos` \| The same, measured where your request arrived \|
296	297
297	298		A push's checks run side by side, so each also has its own entry, after
298		−	the steps and not counted in the total: `read` (reading the push's objects
299		−	and fetching what they build on from the repository), `rules`, `scan`
300		−	(secrets and email addresses) and, for an access token, `gate` (workflow
301		−	files).
302		−	\| `repos` \| The same, measured where your request arrived \|
	299	+	the steps and not counted in the total: `read` (reading the push's
	300	+	objects), `rules`, `scan` (secrets and email addresses) and, for an access
	301	+	token, `gate` (workflow files).
	302	+
	303	+	g1t asks git to send each push whole: every object a delta in it builds on
	304	+	is in the push too (the `no-thin` capability), so checking a push reads
	305	+	nothing from the repository. A push that changes a large file a little
	306	+	uploads more than it would otherwise, and stays within the
	307	+	[size limits](#size-limits). If your git sends a push that builds on
	308	+	objects outside it anyway, g1t reads up to 200 of those from the
	309	+	repository to check it, and says `thin;desc=yes`.
303	310
304		−	Two entries say how a step went rather than how long it took:
	311	+	Some entries say how a step went rather than how long it took:
305	312
306	313		\| Entry \| Values \|
307	314		\| --- \| --- \|
308	315		\| `refs;desc=` \| `hit-colo` or `hit-shared` when the ref listing came from g1t's cache, `miss` when the git store was asked \|
309	316		\| `pack;desc=` \| Only for a fresh clone: `hit` when its pack came from g1t's cache, `miss` when the git store built it \|
310	317		\| `cred;desc=` \| `isolate` or `shared` for a store credential made a moment ago, `mint` for a new one \|
	318	+	\| `thin;desc=` \| Only for a push: `no` when it came whole, `yes` when it built on objects outside it \|
311	319
312	320		The ref listing git asks for first on every clone and fetch is kept for up
313	321		to a minute, and only the same question about the same refs gets the same

169	169		\| Commit one file \| `commit_file.rs` \| mint, `log` \| `info/refs` 1, `receive-pack` 1 \|
170	170		\| Mirror sync or import \| `mirror.rs`, `import.rs` \| mint \| `info/refs` 1–2, `upload-pack` and `receive-pack` 1 each; packs capped at 40 MB \|
171	171		\| Search indexing \| `listing.rs` \| trees by level, blobs in groups \| 0 \|
172		−	\| Push protection \| `secret_scan.rs` \| trees and blobs for bases, up to 24 MB scanned \| 0 \|
	172	+	\| Push protection \| `secret_scan.rs` \| none for bases (`no-thin`); for a client that sends a thin pack anyway, up to 200 bases, 16 at a time; up to 24 MB scanned \| 0 \|
173	173		\| Delete (purge) \| `lifecycle.rs` \| `delete` per key, including forks \| 0 \|
174	174
175	175		Every sandbox job clones in full (`crates/runner/src/{main,checks,review,plan,queue,reply,update,deploy,mergecheck}.rs`:
⋯
238	238
239	239		Next step, in this order:
240	240
241		−	1. Ask for packs without outside bases. Add `no-thin` to the receive-pack advertisement g1t
242		−	forwards, so git sends every delta's base in the pack (git's `send-pack` turns thin packs off
243		−	when it sees it). `read` goes to 0 for every client that honours it, and big pushes no longer
244		−	make hundreds of reads. The cost is a larger upload when a push changes a big file a little;
245		−	pushes stay capped by `MAX_SCANNED_PUSH`. `supply_bases` stays for clients that send thin packs
246		−	anyway.
	241	+	1. Ask for packs without outside bases. Built, not yet deployed. `git_http.rs` `with_no_thin`
	242	+	adds `no-thin` to the receive-pack advertisement g1t forwards (the first ref line's capabilities,
	243	+	or the `capabilities^{}` line of an empty repository; v0 and v1; never upload-pack; an answer it
	244	+	cannot read goes through as it came), so git sends every delta's base in the pack (git's
	245	+	`send-pack` turns thin packs off when it sees it). `read` goes to 0 for every client that
	246	+	honours it, and big pushes no longer make hundreds of reads. The cost is a larger upload when a
	247	+	push changes a big file a little; pushes stay capped by `MAX_SCANNED_PUSH`.
	248	+	`services/repos/dev/push-check.mjs` shows it with git 2.45 against a local store: through g1t,
	249	+	`pack-objects` runs without `--thin`; straight to the store, with it.
	250	+	`supply_bases` stays for clients that send thin packs anyway, now bounded: 200 bases in all
	251	+	(was 500 a round, three rounds), 16 at a time (was all at once), and it stops when the store
	252	+	says it is busy. A push that arrives thin says `thin;desc=yes` in `Server-Timing` and logs the
	253	+	client's user agent with how many bases it lacked, so clients ignoring `no-thin` show up in the
	254	+	tail.
	255	+
	256	+	Why the 437-object push answered 503, as the code explains it (that push's logs were not
	257	+	read, so which reads failed, and how, is not known). Its bases were all asked for at once: up to 500 a
	258	+	round, two binding reads each when the pack's trees do not name the base (old versions of
	259	+	changed files never are), each read with a Cache API look before it, for up to three rounds.
	260	+	Hundreds of reads in flight on one handle, and the reads that failed (dropped or rate limited)
	261	+	were each tried three times with backoff and then counted against the namespace's breaker
	262	+	(`store.rs` `invoke`, `resilience.rs`: five failures in a row open it for 10 s, for the whole
	263	+	isolate). `supply_bases` swallowed its own failures, but once the breaker was open every store
	264	+	read after them, the scan's parent trees and blobs and the rules' reads, was turned away as
	265	+	busy, and `lib.rs` answers busy with 503 and `Retry-After`. The retries and backoff are where
	266	+	the 23 s most likely went. Subrequests (10,000 per invocation) and memory (24 MB pack, small
	267	+	bases) would not have been the limit at that size; concurrency against the store would.
247	268		2. Upload while checking. Stream the pack to Artifacts while the checks run, holding back the
248	269		last 20 bytes (the checksum) until they pass, so a declined push is never stored. A push then
249	270		takes about the larger of `checks` and `upload`, not their sum. Whether Artifacts waits for a

147	147		Anything holding a write token can push past every policy.
148	148		- What we built. `services/repos/src/git_http.rs` `forward` reads the whole push body
149	149		(`request.bytes()`), checks protected branches (`refusal`), then `secret_scan.rs` `scan_push` parses
150		−	the pack in WebAssembly, resolves delta bases by reading objects back through the binding
151		−	(`supply_bases`, up to 500 bases), walks trees and scans up to 24 MB (`MAX_SCANNED_PUSH`) before the
	150	+	the pack in WebAssembly, walks trees and scans up to 24 MB (`MAX_SCANNED_PUSH`) before the
152	151		body is copied again and forwarded. Pushes larger than that are let through unscanned, and we say so in
153		−	the logs. Write tokens handed to sandboxes (`run_access.rs`) bypass all of it, so we keep their life
	152	+	the logs. To check a pack without reading its delta bases back through the binding, g1t adds
	153	+	`no-thin` to the receive-pack advertisement it forwards, so git sends every base in the pack; a
	154	+	client that sends a thin pack anyway has up to 200 bases read for it (`supply_bases`). Write tokens
	155	+	handed to sandboxes (`run_access.rs`) bypass all of it, so we keep their life
154	156		minimal.
155	157		- What it costs. A second implementation of git's pack format in our code, memory pressure on every
156	158		push, and a policy that only holds for pushes that come through our proxy.

489	489		2026-10-09 answered 503 twice and took 23 s the third time. The next step
490	490		is to stop needing bases: see "Pushes" in docs/ARTIFACTS.md.
491	491
	492	+	Step 1 of that, built and not yet deployed: the receive-pack advertisement
	493	+	g1t forwards says `no-thin` (`git_http.rs` `with_no_thin`), so git sends
	494	+	every delta's base in the pack and `read` has nothing to fetch: it should
	495	+	fall from 267–318 ms to the few milliseconds it takes to parse the pack,
	496	+	for every push from a client that honours it (git does; see
	497	+	`services/repos/dev/push-check.mjs`), and stop growing with the push. A
	498	+	push that changes a large file a little uploads more, so `upload` may grow
	499	+	a little for those. A push that arrives thin anyway says `thin;desc=yes` in
	500	+	`Server-Timing`, and is logged with its user agent; its bases are read 16
	501	+	at a time, at most 200, and not at all once the store says it is busy
	502	+	(the 503 above came from all of them at once tripping the store's
	503	+	breaker; see docs/ARTIFACTS.md).
	504	+
492	505		## Client navigation
493	506
494	507		- `<Link prefetch="intent">` on the sidebar, project tabs, breadcrumbs,