Skip to content
Sign up

flagon-io/g1t

public
0
1,461 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

g1t's agents only for listed workspaces, whatever the state of billing1//! Git over HTTPS: the smart HTTP remote at `/<namespace>/<repo>.git`,
2//! proxied to the git store with a short-lived token.
3
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily4use std::cell::RefCell;
5use std::rc::Rc;
6
7use futures_util::StreamExt;
g1t's agents only for listed workspaces, whatever the state of billing8use g1t_contracts::identity::GitCredentialsArgs;
9use g1t_contracts::repos::{GitAccess, GitService, RepoPath};
10use g1t_contracts::{FailureCode, Outcome, Viewer};
11use worker::js_sys::Uint8Array;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily12use worker::wasm_bindgen::JsValue;
g1t's agents only for listed workspaces, whatever the state of billing13use worker::{Fetch, Fetcher, Headers, Method, Request, RequestInit, Response, Result, Url};
14
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily15use crate::meters;
16use crate::pack_limits::{PackSizer, Violation};
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer17use crate::push_checks::{Checks, Verdict};
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily18use crate::resilience::{self, Busy, Failure};
19
g1t's agents only for listed workspaces, whatever the state of billing20const ENDPOINTS: [&str; 3] = ["info/refs", "git-upload-pack", "git-receive-pack"];
21const FORWARDED_HEADERS: [&str; 5] = [
22 "accept",
23 "content-encoding",
24 "content-type",
25 "git-protocol",
26 "user-agent",
27];
28
29/// A git request, parsed from its URL.
30pub struct GitRequest {
31 pub path: RepoPath,
32 pub endpoint: &'static str,
33 pub service: GitService,
34}
35
Merge branch 'worktree-agent-a8385d293d42c913a'36impl GitRequest {
37 /// The same request for the repository of the same name under
38 /// `namespace`: where a workspace alias leads.
39 pub fn under(&self, namespace: &str) -> GitRequest {
40 GitRequest {
41 path: RepoPath {
42 namespace: namespace.to_owned(),
43 name: self.path.name.clone(),
44 },
45 endpoint: self.endpoint,
46 service: self.service,
47 }
48 }
49}
50
g1t's agents only for listed workspaces, whatever the state of billing51/// Parses `/<namespace>/<name>[.git]/<endpoint>`, or returns `None` if the
52/// request is not git's.
53pub fn parse(url: &Url) -> Option<GitRequest> {
54 let path = url.path().strip_prefix('/')?;
55 let endpoint = ENDPOINTS
56 .into_iter()
57 .find(|endpoint| path.ends_with(&format!("/{endpoint}")))?;
58 let repo = &path[..path.len() - endpoint.len() - 1];
59 let (namespace, name) = repo.split_once('/')?;
60 let name = name.strip_suffix(".git").unwrap_or(name);
61 if namespace.is_empty() || name.is_empty() || name.contains('/') {
62 return None;
63 }
64 let service = if endpoint == "info/refs" {
65 url.query_pairs()
66 .find(|(key, _)| key == "service")
67 .map(|(_, value)| value.into_owned())?
68 } else {
69 endpoint.to_owned()
70 };
71 let service = match service.as_str() {
72 "git-upload-pack" => GitService::UploadPack,
73 "git-receive-pack" => GitService::ReceivePack,
74 _ => return None,
75 };
76 Some(GitRequest {
77 path: RepoPath {
78 namespace: namespace.to_owned(),
79 name: name.to_owned(),
80 },
81 endpoint,
82 service,
83 })
84}
85
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms86/// How long each step of a git request took, sent back to git as a
87/// `Server-Timing` header so that a slow clone shows where its time went.
88/// Step names and whole milliseconds only. The Workers clock moves only
89/// while a request waits on something, so each step is the time spent
90/// waiting on the database, another service or the git store. A note
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer91/// says how a step went without a duration: `refs;desc=hit-colo`. A part
92/// is one of several things a step did at once, with its own duration: a
93/// push's `checks` step is its `read`, `rules` and `scan` parts side by side.
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms94pub struct Timing {
95 started: u64,
96 last: u64,
97 steps: Vec<(&'static str, u64)>,
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer98 parts: Vec<(&'static str, u64)>,
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms99 notes: Vec<(&'static str, &'static str)>,
100}
101
102impl Timing {
103 pub fn start() -> Self {
104 let now = g1t_kit::now_ms();
105 Self {
106 started: now,
107 last: now,
108 steps: Vec::new(),
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer109 parts: Vec::new(),
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms110 notes: Vec::new(),
111 }
112 }
113
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer114 /// Says how long `part` of the step under way took. Parts overlap, so
115 /// they are listed after the steps and are not part of the total.
116 pub fn part(&mut self, part: &'static str, ms: u64) {
117 self.parts.push((part, ms));
118 }
119
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms120 /// Says how `name` went: where an answer or a credential came from.
121 pub fn note(&mut self, name: &'static str, description: &'static str) {
122 self.notes.push((name, description));
123 }
124
125 /// Ends a step, named `step`, that began when the last one ended.
126 pub fn mark(&mut self, step: &'static str) {
127 let now = g1t_kit::now_ms();
128 self.steps.push((step, now.saturating_sub(self.last)));
129 self.last = now;
130 }
131
132 /// `response`, with how long each step took.
133 pub fn apply(&self, response: Response) -> Result<Response> {
134 let total = g1t_kit::now_ms().saturating_sub(self.started);
135 let headers = response.headers().clone();
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer136 headers.set("server-timing", &server_timing(&self.steps, &self.parts, &self.notes, total))?;
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms137 Ok(response.with_headers(headers))
138 }
139}
140
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer141/// A `Server-Timing` value: each step with its duration, then each part,
142/// the notes, and the total.
143fn server_timing(steps: &[(&str, u64)], parts: &[(&str, u64)], notes: &[(&str, &str)], total: u64) -> String {
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms144 steps
145 .iter()
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer146 .chain(parts)
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms147 .map(|(step, ms)| format!("{step};dur={ms}"))
148 .chain(notes.iter().map(|(name, description)| format!("{name};desc={description}")))
149 .chain(std::iter::once(format!("total;dur={total}")))
150 .collect::<Vec<_>>()
151 .join(", ")
152}
153
g1t's agents only for listed workspaces, whatever the state of billing154/// The user named by an HTTP Basic `Authorization` header, as git sends it.
155pub async fn viewer(request: &Request, identity: &Fetcher) -> Result<Viewer> {
156 let Some(header) = request.headers().get("authorization")? else {
157 return Ok(None);
158 };
159 let Some((scheme, encoded)) = header.split_once(' ') else {
160 return Ok(None);
161 };
162 if !scheme.eq_ignore_ascii_case("basic") {
163 return Ok(None);
164 }
165 let Some(decoded) = decode_base64(encoded.trim()) else {
166 return Ok(None);
167 };
168 let Some((username, secret)) = decoded.split_once(':') else {
169 return Ok(None);
170 };
171 g1t_kit::call(
172 identity,
173 "user_for_git_credentials",
174 &GitCredentialsArgs {
175 username: username.to_owned(),
176 secret: secret.to_owned(),
177 },
178 )
179 .await
180}
181
182/// Standard base64 to a UTF-8 string, or `None` if either step fails.
183fn decode_base64(input: &str) -> Option<String> {
184 let mut bytes = Vec::with_capacity(input.len() * 3 / 4);
185 let mut buffer = 0u32;
186 let mut bits = 0;
187 for byte in input.bytes().filter(|byte| *byte != b'=') {
188 let value = match byte {
189 b'A'..=b'Z' => byte - b'A',
190 b'a'..=b'z' => byte - b'a' + 26,
191 b'0'..=b'9' => byte - b'0' + 52,
192 b'+' => 62,
193 b'/' => 63,
194 _ => return None,
195 };
196 buffer = (buffer << 6) | u32::from(value);
197 bits += 6;
198 if bits >= 8 {
199 bits -= 8;
200 bytes.push((buffer >> bits) as u8);
201 }
202 }
203 String::from_utf8(bytes).ok()
204}
205
206/// The response for a refused git request. Anonymous callers are asked to
207/// authenticate, which is what makes git prompt for credentials.
208pub fn refuse<T>(outcome: Outcome<T>) -> Result<Response> {
209 let Outcome::Fail(failure) = outcome else {
210 return Response::error("Not found", 404);
211 };
212 let mut response = Response::error(failure.message, failure.code.http_status())?;
213 if failure.code == FailureCode::Unauthenticated {
214 response
215 .headers_mut()
216 .set("www-authenticate", "Basic realm=\"g1t\"")?;
217 }
218 Ok(response)
219}
220
Agents and memory, checks and conflicts, profiles, slug renames, custom domains221/// `url` with its first path segment, the workspace, replaced by `slug`.
222pub fn with_namespace(url: &Url, slug: &str) -> Option<String> {
223 let rest = url.path().strip_prefix('/')?.split_once('/')?.1;
224 let mut moved = url.clone();
225 moved.set_path(&format!("/{slug}/{rest}"));
226 Some(moved.to_string())
227}
228
229/// Where a git request for a renamed workspace's old address should go
230/// now, if its first segment is an old slug that still redirects.
231pub async fn renamed(url: &Url, identity: &Fetcher) -> Result<Option<String>> {
232 let Some(old) = url.path().strip_prefix('/').and_then(|path| path.split('/').next()) else {
233 return Ok(None);
234 };
235 let current: Option<String> = g1t_kit::call(
236 identity,
237 "resolve_slug",
238 &g1t_contracts::identity::SlugArgs {
239 slug: old.to_owned(),
240 },
241 )
242 .await?;
243 Ok(current.and_then(|slug| with_namespace(url, &slug)))
244}
245
Merge branch 'worktree-agent-a8385d293d42c913a'246/// The request under the workspace its first segment is an alias of
247/// (identity's aliases.rs: `g1t` for `flagon-io`), if it is one. Answered
248/// in place rather than redirected: a push does not follow a redirect.
249pub async fn aliased(git: &GitRequest, identity: &Fetcher) -> Result<Option<GitRequest>> {
250 let slug: Option<String> = g1t_kit::call(
251 identity,
252 "resolve_alias",
253 &g1t_contracts::identity::SlugArgs {
254 slug: git.path.namespace.clone(),
255 },
256 )
257 .await?;
258 Ok(slug.map(|slug| git.under(&slug)))
259}
260
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look261/// `url` with its repository, the first two path segments, replaced by
262/// `to`: where a request for a transferred repository's old path goes.
263/// Keeps whether the old address ended in `.git`.
264pub fn transferred(url: &Url, to: &RepoPath) -> Option<String> {
265 let path = url.path().strip_prefix('/')?;
266 let mut segments = path.splitn(3, '/');
267 let (_, name, rest) = (segments.next()?, segments.next()?, segments.next()?);
268 let suffix = if name.ends_with(".git") { ".git" } else { "" };
269 let mut moved = url.clone();
270 moved.set_path(&format!("/{}/{}{suffix}/{rest}", to.namespace, to.name));
271 Some(moved.to_string())
272}
273
Agents and memory, checks and conflicts, profiles, slug renames, custom domains274/// A permanent redirect: 301 for git's first request for refs, which it
275/// follows and then uses the new address for the rest; 308 for the
276/// others, so a POST stays a POST.
277pub fn moved(location: &str, get: bool) -> Result<Response> {
278 let mut response = Response::empty()?.with_status(if get { 301 } else { 308 });
279 response.headers_mut().set("location", location)?;
280 Ok(response)
281}
282
g1t's agents only for listed workspaces, whatever the state of billing283const ZERO_ID: &str = "0000000000000000000000000000000000000000";
284const HEADS: &str = "refs/heads/";
GitHub Actions on g1t, part one: reading workflows285const TAGS: &str = "refs/tags/";
g1t's agents only for listed workspaces, whatever the state of billing286
287/// One ref a push asks to change.
288struct Command {
289 old: String,
290 new: String,
291 name: String,
292}
293
294/// The commands at the start of a receive-pack request, and the
295/// capabilities the client sent with the first of them.
296fn commands(body: &[u8]) -> (Vec<Command>, String) {
297 let mut commands = Vec::new();
298 let mut capabilities = String::new();
299 let mut position = 0;
300 // Commands are pkt-lines; a flush packet ends them and the pack follows.
301 while let Some(length) = body
302 .get(position..position + 4)
303 .and_then(|hex| std::str::from_utf8(hex).ok())
304 .and_then(|hex| usize::from_str_radix(hex, 16).ok())
305 {
306 if length < 4 || position + length > body.len() {
307 break;
308 }
309 let line = &body[position + 4..position + length];
310 position += length;
311 // `<old> <new> <ref>`, and on the first command a NUL then capabilities.
312 let mut halves = line.splitn(2, |byte| *byte == 0);
313 let command = halves.next().unwrap_or_default();
314 if let Some(rest) = halves.next() {
315 capabilities = String::from_utf8_lossy(rest).trim().to_owned();
316 }
317 let Ok(command) = std::str::from_utf8(command) else {
318 continue;
319 };
320 let mut parts = command.trim_end().splitn(3, ' ');
321 if let (Some(old), Some(new), Some(name)) = (parts.next(), parts.next(), parts.next()) {
322 commands.push(Command {
323 old: old.to_owned(),
324 new: new.to_owned(),
325 name: name.to_owned(),
326 });
327 }
328 }
329 (commands, capabilities)
330}
331
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put332/// The bytes of the pack a receive-pack request carries: everything after
333/// the flush packet that ends its commands. Zero for a push that only
334/// deletes refs.
335pub(crate) fn pack_bytes(body: &[u8]) -> u64 {
336 let mut position = 0;
337 while let Some(length) = body
338 .get(position..position + 4)
339 .and_then(|hex| std::str::from_utf8(hex).ok())
340 .and_then(|hex| usize::from_str_radix(hex, 16).ok())
341 {
342 if length == 0 {
343 return (body.len() - position - 4) as u64;
344 }
345 if length < 4 || position + length > body.len() {
346 break;
347 }
348 position += length;
349 }
350 0
351}
352
g1t's agents only for listed workspaces, whatever the state of billing353fn pkt_line(payload: &[u8]) -> Vec<u8> {
354 let mut line = format!("{:04x}", payload.len() + 4).into_bytes();
355 line.extend_from_slice(payload);
356 line
357}
358
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge359/// The branches and tags a push asks to change, as rules see them: the
360/// full ref, where it pointed (`None`: it is created) and where it will
361/// (`None`: it is deleted).
362pub(crate) fn ref_updates(body: &[u8]) -> Vec<(String, Option<String>, Option<String>)> {
363 commands(body)
364 .0
365 .into_iter()
366 .filter(|command| command.name.starts_with(HEADS) || command.name.starts_with(TAGS))
367 .map(|Command { old, new, name }| (name, (old != ZERO_ID).then_some(old), (new != ZERO_ID).then_some(new)))
368 .collect()
369}
370
371/// A report-status answer, as git expects it.
372pub(crate) fn report_response(report: Vec<u8>) -> Result<Response> {
373 let headers = Headers::new();
374 headers.set("content-type", "application/x-git-receive-pack-result")?;
375 headers.set("cache-control", "no-cache")?;
376 Ok(Response::from_bytes(report)?.with_headers(headers))
377}
378
g1t's agents only for listed workspaces, whatever the state of billing379/// What git is told when a push would change a protected branch: every ref
380/// in it is declined, with the reason against the protected one, so that
381/// git prints it beside the branch. `None` if the push leaves the branch
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge382/// alone, or creates it in a repository that does not have it yet. Only
383/// where rulesets cannot be read (an installation without the work
384/// service); rulesets decide everywhere else (rules.rs).
385pub(crate) fn refusal(body: &[u8], protected: &str) -> Option<Vec<u8>> {
g1t's agents only for listed workspaces, whatever the state of billing386 let (commands, capabilities) = commands(body);
387 let reference = format!("{HEADS}{protected}");
388 if !commands
389 .iter()
390 .any(|command| command.name == reference && command.old != ZERO_ID)
391 {
392 return None;
393 }
394 let mut report = pkt_line(b"unpack ok\n");
395 for command in &commands {
396 let reason = if command.name == reference {
397 format!("{protected} is protected: push a branch and open a pull request")
398 } else {
399 format!("not pushed, because the same push would change {protected}")
400 };
401 report.extend(pkt_line(
402 format!("ng {} {reason}\n", command.name).as_bytes(),
403 ));
404 }
405 report.extend_from_slice(b"0000");
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API406 Some(framed(report, &capabilities, &[]))
407}
408
409/// A report-status as git expects it: inside channel 1 when the client
410/// asked for side-band, after `messages` on channel 2, which git prints as
411/// `remote:` lines. Without side-band the messages cannot be shown.
412fn framed(report: Vec<u8>, capabilities: &str, messages: &[String]) -> Vec<u8> {
g1t's agents only for listed workspaces, whatever the state of billing413 let sideband = capabilities
414 .split(' ')
415 .any(|capability| capability.starts_with("side-band"));
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API416 if !sideband {
417 return report;
418 }
419 let mut body = Vec::new();
420 for message in messages {
421 let mut packet = vec![2u8];
422 packet.extend_from_slice(message.as_bytes());
423 packet.push(b'\n');
424 body.extend(pkt_line(&packet));
425 }
426 // side-band (not -64k) packets carry at most 1000 bytes.
427 for chunk in report.chunks(990) {
428 let mut packet = vec![1u8];
429 packet.extend_from_slice(chunk);
430 body.extend(pkt_line(&packet));
431 }
432 body.extend_from_slice(b"0000");
433 body
434}
435
436/// Declines every ref in a push with `reason`, explaining why in
437/// `messages`: what push protection answers when a push adds a secret.
438pub fn declined(body: &[u8], reason: &str, messages: &[String]) -> Result<Response> {
439 let (commands, capabilities) = commands(body);
440 let mut report = pkt_line(b"unpack ok\n");
441 for command in &commands {
442 report.extend(pkt_line(format!("ng {} {reason}\n", command.name).as_bytes()));
443 }
444 report.extend_from_slice(b"0000");
445 let headers = Headers::new();
446 headers.set("content-type", "application/x-git-receive-pack-result")?;
447 headers.set("cache-control", "no-cache")?;
448 Ok(Response::from_bytes(framed(report, &capabilities, messages))?.with_headers(headers))
g1t's agents only for listed workspaces, whatever the state of billing449}
450
GitHub Actions on g1t, part one: reading workflows451/// A branch or tag a push asks to move.
452#[derive(Debug, PartialEq, Eq)]
453pub struct Pushed {
454 /// The full ref: `refs/heads/main`, `refs/tags/v1`.
455 pub git_ref: String,
456 /// Where it pointed before; `None` for a new ref.
457 pub before: Option<String>,
458 pub after: String,
459}
460
461impl Pushed {
462 pub fn branch(&self) -> Option<&str> {
463 self.git_ref.strip_prefix(HEADS)
464 }
465}
466
467/// The branches and tags a push asks to move, read from the commands at the
468/// start of a receive-pack request. Deletions and other refs are left out.
469fn pushed_branches(body: &[u8]) -> Vec<Pushed> {
g1t's agents only for listed workspaces, whatever the state of billing470 commands(body)
471 .0
472 .into_iter()
473 .filter(|command| command.new != ZERO_ID)
GitHub Actions on g1t, part one: reading workflows474 .filter(|command| command.name.starts_with(HEADS) || command.name.starts_with(TAGS))
475 .map(|Command { old, new, name }| Pushed {
476 git_ref: name,
477 before: (old != ZERO_ID).then_some(old),
478 after: new,
g1t's agents only for listed workspaces, whatever the state of billing479 })
480 .collect()
481}
482
483/// The git store's answer, and what the request asked it to change.
484pub struct Forwarded {
485 pub response: Response,
GitHub Actions on g1t, part one: reading workflows486 /// For a push: the branches and tags it asks to move, and the commits
487 /// to move them to. Whether each moved is for the caller to confirm.
488 pub pushed: Vec<Pushed>,
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put489 /// For a push: the size of the pack it sent, for the storage meter.
490 pub pack_bytes: u64,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily491 /// The bytes sent to the store.
492 pub sent: u64,
493 /// Whether the answer is the store's own (not g1t's, for a store that
494 /// was busy).
495 pub from_store: bool,
496 /// For a push: whether it was too large to scan for secrets first and
497 /// was streamed to the store unscanned (`LargePushes::Unscanned`). Its
498 /// `git.push` events say so, and security scans it after it lands.
499 pub unscanned: bool,
g1t's agents only for listed workspaces, whatever the state of billing500}
501
502/// What became of a git request.
503pub enum Push {
504 Forwarded(Forwarded),
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge505 /// A push the rules of its branches or tags refuse (rules.rs), answered
506 /// here without reaching the store.
g1t's agents only for listed workspaces, whatever the state of billing507 Refused(Response),
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API508 /// A push that adds a secret nobody allowed, answered the same way.
509 Blocked(Response),
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily510 /// A push the store could not hold: an object or the repository too
511 /// large, or too large to check. With the reason, for the audit log.
512 Declined(Response, String),
513}
514
515/// What a push may bring, checked as it arrives (pack_limits.rs).
516#[derive(Clone, Copy, Debug)]
517pub struct PushLimits {
518 /// The largest object the store holds.
519 pub max_object: u64,
520 /// What the repository holds now, as g1t counts it.
521 pub held: u64,
522 /// The most a repository may hold.
523 pub repo_limit: u64,
524 /// The largest push that is read whole and scanned for secrets.
525 pub scan_cap: usize,
526 /// What happens to a larger one.
527 pub large: LargePushes,
528}
529
530impl Default for PushLimits {
531 fn default() -> Self {
532 PushLimits {
533 max_object: crate::pack_limits::MAX_OBJECT_BYTES,
534 held: 0,
535 repo_limit: crate::pack_limits::DEFAULT_REPO_LIMIT_BYTES,
536 scan_cap: crate::secret_scan::MAX_SCANNED_PUSH,
537 large: LargePushes::Refuse,
538 }
539 }
540}
541
542/// What happens to a push larger than [`PushLimits::scan_cap`]: set by
543/// `LARGE_PUSHES`.
544#[derive(Clone, Copy, Debug, PartialEq, Eq)]
545pub enum LargePushes {
546 /// Declined (the default): push protection cannot read it, so it does
547 /// not let it in.
548 Refuse,
549 /// Streamed to the store without a scan for secrets; the size limits are
550 /// still checked as it passes.
551 Unscanned,
552}
553
554impl LargePushes {
555 pub fn from_var(value: Option<&str>) -> Self {
556 match value.map(str::trim) {
557 Some("unscanned") => LargePushes::Unscanned,
558 _ => LargePushes::Refuse,
559 }
560 }
561}
562
563/// A push the store could not hold.
564#[derive(Clone, Debug, PartialEq, Eq)]
565pub enum SizeViolation {
566 Object { size: u64 },
567 Repository { held: u64, incoming: u64, limit: u64 },
568 Unscannable { size: u64, cap: usize },
569}
570
571/// Why a push is declined for its size, as git shows it: the `ng` reason,
572/// and the lines printed as `remote:`.
573pub fn size_refusal(violation: &SizeViolation) -> (String, Vec<String>) {
574 use crate::pack_limits::{MAX_OBJECT_BYTES, PLATFORM_BODY_LIMIT_BYTES, megabytes};
575 match violation {
576 SizeViolation::Object { size } => (
577 format!("a file of {} is over the {} limit", megabytes(*size), megabytes(MAX_OBJECT_BYTES)),
578 vec![
579 format!("g1t stores files of up to {} each; this push has one of {}.", megabytes(MAX_OBJECT_BYTES), megabytes(*size)),
580 "Take it out of the commits (git rm --cached, then amend or rebase), and keep large".to_owned(),
581 "files elsewhere: https://docs.g1t.sh/guides/git/#size-limits. Nothing was pushed.".to_owned(),
582 ],
583 ),
584 SizeViolation::Repository { held, incoming, limit } => (
585 "the repository would be over its size limit".to_owned(),
586 vec![
587 format!(
588 "This repository holds about {} and the push adds {}, past the {} a repository may hold.",
589 megabytes(*held),
590 megabytes(*incoming),
591 megabytes(*limit)
592 ),
593 "Delete what you no longer need, or split it: https://docs.g1t.sh/guides/git/#size-limits.".to_owned(),
594 "Nothing was pushed.".to_owned(),
595 ],
596 ),
597 SizeViolation::Unscannable { size, cap } => (
598 "the push is too large to check for secrets".to_owned(),
599 vec![
600 format!(
601 "g1t checks every push for secrets and reads up to {} at once; this one is {}.",
602 megabytes(*cap as u64),
603 megabytes(*size)
604 ),
605 "Push in parts, oldest commits first, then push as usual:".to_owned(),
606 " git rev-list --reverse HEAD | awk 'NR % 500 == 0' | xargs -I{} git push origin {}:refs/heads/main".to_owned(),
607 format!("A push over {} is refused by the network before it reaches g1t (HTTP 413).", megabytes(PLATFORM_BODY_LIMIT_BYTES)),
608 "See https://docs.g1t.sh/guides/git/#size-limits. Nothing was pushed.".to_owned(),
609 ],
610 ),
611 }
612}
613
614/// Feeds the next chunk of a push to the size check; the first violation.
615fn check_size(sizer: &mut Option<PackSizer>, chunk: &[u8], limits: &PushLimits) -> Option<SizeViolation> {
616 let walker = sizer.as_mut()?;
617 match walker.feed(chunk) {
618 Ok(()) => {}
619 Err(Violation::ObjectTooLarge { size }) => return Some(SizeViolation::Object { size }),
620 Err(Violation::Malformed(why)) => {
621 // Not for g1t to judge: the store will say.
622 worker::console_error!("push not checked for size: {why}");
623 *sizer = None;
624 return None;
625 }
626 }
627 let incoming = walker.pack_bytes();
628 crate::pack_limits::over_repo_limit(limits.held, incoming, limits.repo_limit).then_some(SizeViolation::Repository {
629 held: limits.held,
630 incoming,
631 limit: limits.repo_limit,
632 })
633}
634
635/// A request body that streams from `stream`, for `fetch`.
636pub(crate) fn stream_body<S>(stream: S) -> Result<JsValue>
637where
638 S: futures_util::TryStream + 'static,
639 S::Ok: Into<Vec<u8>>,
640 S::Error: Into<worker::Error>,
641{
642 let response: worker::web_sys::Response = Response::from_stream(stream)?.into();
643 Ok(response.body().map_or(JsValue::NULL, Into::into))
644}
645
646/// What git is told when the store is busy: 429 or 503, with when to try
647/// again (resilience.rs).
648pub fn busy_response(busy: Busy) -> Result<Response> {
649 let response = Response::error(busy.message(), busy.status())?;
650 response.headers().set("retry-after", &busy.retry_after.to_string())?;
651 Ok(response)
652}
653
654/// The rest of a request's body, read and thrown away so that git hears
655/// the answer; how many bytes it was.
656async fn drain(stream: &mut worker::ByteStream) -> Result<u64> {
657 let mut size = 0;
658 while let Some(chunk) = stream.next().await {
659 size += chunk?.len() as u64;
660 }
661 Ok(size)
g1t's agents only for listed workspaces, whatever the state of billing662}
663
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look664/// One packet of a pkt-line stream: data, or a flush (`0000`), delimiter
665/// (`0001`) or response-end (`0002`) packet, kept as its four bytes.
666#[derive(Debug, PartialEq, Eq)]
667enum Packet {
668 Data(Vec<u8>),
669 Special([u8; 4]),
670}
671
672/// The packets in `bytes`, or `None` if it is not a whole pkt-line stream.
673fn packets(bytes: &[u8]) -> Option<Vec<Packet>> {
674 let mut out = Vec::new();
675 let mut position = 0;
676 while position < bytes.len() {
677 let header = bytes.get(position..position + 4)?;
678 let length = usize::from_str_radix(std::str::from_utf8(header).ok()?, 16).ok()?;
679 if length < 4 {
680 out.push(Packet::Special(header.try_into().ok()?));
681 position += 4;
682 continue;
683 }
684 out.push(Packet::Data(bytes.get(position + 4..position + length)?.to_vec()));
685 position += length;
686 }
687 Some(out)
688}
689
690fn encode(packets: &[Packet]) -> Vec<u8> {
691 let mut out = Vec::new();
692 for packet in packets {
693 match packet {
694 Packet::Data(data) => out.extend(pkt_line(data)),
695 Packet::Special(bytes) => out.extend_from_slice(bytes),
696 }
697 }
698 out
699}
700
701/// A ref advertisement (`info/refs` for upload-pack) or a protocol v2
702/// `ls-refs` answer with `HEAD` pointing at `branch`, the repository's
703/// default branch as g1t keeps it, so a clone checks it out. The git store
704/// holds the HEAD it was created with; g1t can change the default branch
705/// since. `None` when there is nothing to change: no `HEAD` line, `HEAD`
706/// already names `branch`, or `branch` is not advertised.
707pub fn with_head(body: &[u8], branch: &str) -> Option<Vec<u8>> {
708 let mut packets = packets(body)?;
709 let target = format!("{HEADS}{branch}");
710 let oid = packets.iter().find_map(|packet| {
711 let Packet::Data(data) = packet else { return None };
712 let line = data.split(|byte| *byte == 0).next()?;
713 let line = std::str::from_utf8(line).ok()?.trim_end();
714 let (oid, name) = line.split_once(' ')?;
715 // v2 lines may carry attributes after the name.
716 let name = name.split(' ').next()?;
717 (name == target).then(|| oid.to_owned())
718 })?;
719 let mut changed = false;
720 for packet in &mut packets {
721 let Packet::Data(data) = packet else { continue };
722 let text = String::from_utf8_lossy(data).into_owned();
723 let Some((_, rest)) = text.split_once(' ') else { continue };
724 if !(rest.starts_with("HEAD\0") || rest.starts_with("HEAD\n") || rest.starts_with("HEAD ") || rest == "HEAD") {
725 continue;
726 }
727 let mut line = format!("{oid} {rest}");
728 // v0: `symref=HEAD:refs/heads/<old>` among the capabilities.
729 // v2: `symref-target:refs/heads/<old>` after the name.
730 for marker in ["symref=HEAD:", "symref-target:"] {
731 if let Some(at) = line.find(marker) {
732 let start = at + marker.len();
733 let end = line[start..]
734 .find([' ', '\n', '\0'])
735 .map_or(line.len(), |offset| start + offset);
736 line.replace_range(start..end, &target);
737 }
738 }
739 changed = line != text;
740 if changed {
741 *data = line.into_bytes();
742 }
743 break;
744 }
745 changed.then(|| encode(&packets))
746}
747
748/// Whether a request to the git store is one whose answer names `HEAD`:
749/// the ref advertisement for a fetch, or a protocol v2 `ls-refs`.
750fn names_head(git: &GitRequest, body: Option<&[u8]>) -> bool {
751 if git.service != GitService::UploadPack {
752 return false;
753 }
754 match body {
755 None => git.endpoint == "info/refs",
756 Some(body) => {
757 git.endpoint == "git-upload-pack"
758 && body.windows(b"command=ls-refs".len()).any(|window| window == b"command=ls-refs")
759 }
760 }
761}
762
A fetch whose haves the store does not know gets its pack: g1t ends the store's early answer where git expects763/// Whether a request is a protocol v2 `fetch` still negotiating: it sends
764/// `have` lines and no `done`, so the answer may be acknowledgments only.
765fn negotiating(body: &[u8]) -> bool {
766 let Some(packets) = packets(body) else { return false };
767 let lines: Vec<&[u8]> = packets
768 .iter()
769 .filter_map(|packet| match packet {
770 Packet::Data(data) => Some(data.strip_suffix(b"\n").unwrap_or(data)),
771 Packet::Special(_) => None,
772 })
773 .collect();
774 lines.contains(&b"command=fetch".as_slice())
775 && lines.iter().any(|line| line.starts_with(b"have "))
776 && !lines.contains(&b"done".as_slice())
777}
778
779/// What to do with the start of a store's answer to a negotiating fetch.
780#[derive(Debug, PartialEq, Eq)]
781enum Acknowledged {
782 /// Not enough of it yet to tell.
783 NeedMore,
784 /// Send it on as it is.
785 Whole,
786 /// Acknowledgments without `ready`, followed by more sections: the
787 /// store's answer to keep is these first bytes, ended by a flush.
788 CutAt(usize),
789}
790
791/// How much of the answer to keep. The git store answers a fetch whose
792/// `have`s it does not know with `acknowledgments`, `NAK`, then a pack
793/// anyway; git refuses that ("expected no other sections to be sent after
794/// no 'ready'"), since a server that is not ready must end the response
795/// there and let the client negotiate again. Lines may be `sideband-all`
796/// framed (band 1, `\x01`).
797fn acknowledged(head: &[u8]) -> Acknowledged {
798 let mut position = 0;
799 let mut first = true;
800 loop {
801 let Some(header) = head.get(position..position + 4) else { return Acknowledged::NeedMore };
802 let Some(length) = std::str::from_utf8(header).ok().and_then(|hex| usize::from_str_radix(hex, 16).ok()) else {
803 return Acknowledged::Whole;
804 };
805 if length < 4 {
806 // The acknowledgments section's end: a delimiter means more
807 // sections follow, which only `ready` allows.
808 return match (first, header) {
809 (false, b"0001") => Acknowledged::CutAt(position),
810 _ => Acknowledged::Whole,
811 };
812 }
813 let Some(payload) = head.get(position + 4..position + length) else { return Acknowledged::NeedMore };
814 let line = payload.strip_prefix(b"\x01").unwrap_or(payload);
815 let line = line.strip_suffix(b"\n").unwrap_or(line);
816 if first && line != b"acknowledgments" {
817 return Acknowledged::Whole;
818 }
819 if line == b"ready" {
820 return Acknowledged::Whole;
821 }
822 first = false;
823 position += length;
824 }
825}
826
827/// The answer to a negotiating fetch, with the sections the store sent
828/// after acknowledgments without `ready` left off (see [`acknowledged`]).
829/// Reads only the start of the answer; the rest streams through.
830async fn without_early_pack(mut response: Response) -> Result<Response> {
831 const LOOK: usize = 64 * 1024;
832 let headers = response.headers().clone();
833 headers.delete("content-length")?;
834 let mut stream = response.stream()?;
835 let mut head = Vec::new();
836 loop {
837 match acknowledged(&head) {
838 Acknowledged::CutAt(at) => {
839 head.truncate(at);
The early answer ends with a flush only; git's HTTP transport adds the response-end packet itself840 // A flush ends the acknowledgments and the answer. No
841 // response-end packet: git's HTTP transport adds its own
842 // and refuses one from the server.
843 head.extend_from_slice(b"0000");
A fetch whose haves the store does not know gets its pack: g1t ends the store's early answer where git expects844 return Ok(Response::from_bytes(head)?.with_headers(headers));
845 }
846 Acknowledged::Whole => break,
847 Acknowledged::NeedMore if head.len() >= LOOK => break,
848 Acknowledged::NeedMore => match stream.next().await {
849 Some(chunk) => head.extend_from_slice(&chunk?),
850 None => break,
851 },
852 }
853 }
854 let rest = futures_util::stream::once(async move { Ok::<Vec<u8>, worker::Error>(head) }).chain(stream);
855 Ok(Response::from_stream(rest)?.with_headers(headers))
856}
857
g1t's agents only for listed workspaces, whatever the state of billing858/// Sends the request on to the git store and returns its response as is,
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer859/// unless it is a push `checks` refuse (the workflow gate, the rules, push
860/// protection: push_checks.rs), or one the store could not hold (`limits`,
861/// pack_limits.rs). `checks` is given as much of the push as was read,
862/// whether that is all of it, and whether to scan it for secrets. A
863/// fetch's ref listing has its `HEAD` pointed at `default_branch` (see
864/// [`with_head`]). A POST's body is `read` when the caller has read it
865/// already.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily866///
867/// A push is read as it arrives: up to `limits.scan_cap` is kept, to be
868/// scanned and sent on whole; past it, the push is declined, or streamed
869/// to the store unscanned (`LargePushes`), never held. Reads the store
870/// fails for a moment (429, 5xx) are tried again with backoff; a push never
871/// is. A store still busy after that is answered 429 or 503 with
872/// `Retry-After`.
873#[allow(clippy::too_many_arguments)]
g1t's agents only for listed workspaces, whatever the state of billing874pub async fn forward(
875 mut request: Request,
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms876 read: Option<Vec<u8>>,
g1t's agents only for listed workspaces, whatever the state of billing877 git: &GitRequest,
878 access: &GitAccess,
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer879 checks: impl AsyncFnOnce(&[u8], bool, bool) -> Result<Checks>,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look880 default_branch: Option<&str>,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily881 limits: PushLimits,
A push says where its time went: receiving it, the rules, the secret scan and the git store's answer each have their own Server-Timing step882 timing: &mut Timing,
g1t's agents only for listed workspaces, whatever the state of billing883) -> Result<Push> {
884 let headers = Headers::new();
885 headers.set("authorization", &format!("Bearer {}", access.token))?;
886 for name in FORWARDED_HEADERS {
887 if let Some(value) = request.headers().get(name)? {
888 headers.set(name, &value)?;
889 }
890 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily891 let query = request.url()?.query().map(|query| format!("?{query}")).unwrap_or_default();
892 let url = format!("{}/{}{query}", access.remote, git.endpoint);
893 let method = request.method();
Merge branch 'worktree-agent-a2013627e5ea4ab13'894 // Its own health and breaker: the fallback store's apart from Artifacts'.
895 let namespace = crate::store::health_namespace(&access.remote);
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily896
897 if method == Method::Post && git.endpoint == "git-receive-pack" {
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer898 return push(request, &url, headers, checks, limits, &namespace, timing).await;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily899 }
900
901 // A read: the ref advertisement, `ls-refs`, or a fetch of objects.
902 let body = match (&method, read) {
903 (Method::Post, Some(body)) => Some(body),
904 (Method::Post, None) => Some(request.bytes().await?),
905 _ => None,
906 };
907 let lists_head = match &body {
908 None => method == Method::Get && names_head(git, None),
909 Some(body) => names_head(git, Some(body)),
910 };
911 let sent = body.as_ref().map_or(0, |body| body.len() as u64);
912 let mut attempt = 0;
913 let mut response = loop {
914 let mut init = RequestInit::new();
915 init.with_method(method.clone()).with_headers(headers.clone());
916 if let Some(body) = &body {
917 init.with_body(Some(Uint8Array::from(body.as_slice()).into()));
918 }
919 let started = g1t_kit::now_ms();
920 let answered = Fetch::Request(Request::new_with_init(&url, &init)?).send().await;
921 let ms = g1t_kit::now_ms().saturating_sub(started);
922 let failure = match &answered {
923 Ok(response) => resilience::classify_status(response.status_code()),
924 Err(_) => Some(Failure::Transient),
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms925 };
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily926 let outcome = match failure {
927 None => meters::Outcome::Ok,
928 Some(Failure::RateLimited) => meters::Outcome::RateLimited,
929 Some(_) => meters::Outcome::Failed,
930 };
931 meters::record_health(&namespace, outcome, ms);
932 match (answered, failure) {
933 (Ok(response), None) => break response,
934 (answered, Some(failure)) if resilience::retry(failure, attempt) => {
935 drop(answered);
936 let wait = resilience::backoff_ms(failure, attempt, worker::js_sys::Math::random());
937 worker::Delay::from(std::time::Duration::from_millis(wait)).await;
938 attempt += 1;
g1t's agents only for listed workspaces, whatever the state of billing939 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily940 (_, Some(failure)) => {
Merge branch 'worktree-agent-a2013627e5ea4ab13'941 let busy = Busy { rate_limited: failure == Failure::RateLimited, retry_after: 5, read_only: false };
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily942 return Ok(Push::Forwarded(Forwarded {
943 response: busy_response(busy)?,
944 pushed: Vec::new(),
945 pack_bytes: 0,
946 sent,
947 from_store: false,
948 unscanned: false,
949 }));
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API950 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily951 (Err(error), None) => return Err(error),
g1t's agents only for listed workspaces, whatever the state of billing952 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily953 };
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look954 if let (true, Some(branch)) = (lists_head, default_branch)
955 && response.status_code() == 200
956 {
957 let headers = response.headers().clone();
958 headers.delete("content-length")?;
959 let body = response.bytes().await?;
960 let body = with_head(&body, branch).unwrap_or(body);
961 response = Response::from_bytes(body)?.with_headers(headers);
962 }
A fetch whose haves the store does not know gets its pack: g1t ends the store's early answer where git expects963 if git.endpoint == "git-upload-pack"
964 && response.status_code() == 200
965 && body.as_deref().is_some_and(negotiating)
966 {
967 response = without_early_pack(response).await?;
968 }
g1t's agents only for listed workspaces, whatever the state of billing969 Ok(Push::Forwarded(Forwarded {
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look970 response,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily971 pushed: Vec::new(),
972 pack_bytes: 0,
973 sent,
974 from_store: true,
975 unscanned: false,
976 }))
977}
978
A push says where its time went: receiving it, the rules, the secret scan and the git store's answer each have their own Server-Timing step979/// A receive-pack request; see [`forward`]. Its steps: `recv` (the push
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer980/// read), `checks`, `upload` (the store's answer).
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily981#[allow(clippy::too_many_arguments)]
982async fn push(
983 mut request: Request,
984 url: &str,
985 headers: Headers,
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer986 checks: impl AsyncFnOnce(&[u8], bool, bool) -> Result<Checks>,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily987 limits: PushLimits,
988 namespace: &str,
A push says where its time went: receiving it, the rules, the secret scan and the git store's answer each have their own Server-Timing step989 timing: &mut Timing,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily990) -> Result<Push> {
991 let mut stream = request.stream()?;
992 let mut head: Vec<u8> = Vec::new();
993 let mut sizer = Some(PackSizer::new(limits.max_object));
994 let mut violation = None;
995 let mut ended = false;
996 while head.len() <= limits.scan_cap {
997 match stream.next().await {
998 Some(chunk) => {
999 let chunk = chunk?;
1000 if violation.is_none() {
1001 violation = check_size(&mut sizer, &chunk, &limits);
1002 }
1003 head.extend_from_slice(&chunk);
1004 }
1005 None => {
1006 ended = true;
1007 break;
1008 }
1009 }
1010 }
A push says where its time went: receiving it, the rules, the secret scan and the git store's answer each have their own Server-Timing step1011 timing.mark("recv");
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1012 // The workflow gate and the rules of the branches and tags it changes,
1013 // with push protection alongside for a push read whole that is within
1014 // the size limits (push_checks.rs). What the rules refuse is refused
1015 // whatever else is wrong with it.
1016 let checked = checks(&head, ended, ended && violation.is_none()).await?;
1017 for (part, ms) in checked.spans {
1018 timing.part(part, ms);
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1019 }
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1020 timing.mark("checks");
1021 let blocked = match checked.verdict {
1022 Verdict::Refused(response) => {
1023 if !ended {
1024 drain(&mut stream).await?;
1025 }
1026 return Ok(Push::Refused(response));
1027 }
1028 Verdict::Blocked(response) => Some(response),
1029 Verdict::Clear => None,
1030 };
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1031 if !ended && limits.large == LargePushes::Refuse && violation.is_none() {
1032 let size = head.len() as u64 + drain(&mut stream).await?;
1033 violation = Some(SizeViolation::Unscannable { size, cap: limits.scan_cap });
1034 ended = true;
1035 }
1036 if let Some(violation) = violation {
1037 if !ended {
1038 drain(&mut stream).await?;
1039 }
1040 let (reason, messages) = size_refusal(&violation);
1041 return Ok(Push::Declined(declined(&head, &reason, &messages)?, reason));
1042 }
1043 let pushed = pushed_branches(&head);
1044 let mut init = RequestInit::new();
1045 init.with_method(Method::Post).with_headers(headers);
1046 let started = g1t_kit::now_ms();
1047 let (answered, pack_bytes, sent, unscanned) = if ended {
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1048 if let Some(response) = blocked {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1049 return Ok(Push::Blocked(response));
1050 }
1051 let pack = pack_bytes(&head);
1052 let sent = head.len() as u64;
1053 init.with_body(Some(Uint8Array::from(head.as_slice()).into()));
1054 drop(head);
1055 (Fetch::Request(Request::new_with_init(url, &init)?).send().await, pack, sent, false)
1056 } else {
1057 // Larger than can be scanned, and let through unscanned: streamed,
1058 // with the size limits checked as it passes. A violation ends the
1059 // stream before the pack does, so the store refuses it whole.
1060 worker::console_warn!("a push of more than {} bytes goes to the store unscanned", limits.scan_cap);
1061 let commands = head.iter().take(64 * 1024).copied().collect::<Vec<u8>>();
1062 let found: Rc<RefCell<Option<SizeViolation>>> = Rc::default();
1063 let walked = Rc::new(RefCell::new((sizer, 0u64)));
1064 let rest = {
1065 let found = found.clone();
1066 let walked = walked.clone();
1067 stream.map(move |chunk| {
1068 let chunk = chunk?;
1069 let mut walked = walked.borrow_mut();
1070 walked.1 += chunk.len() as u64;
1071 if let Some(violation) = check_size(&mut walked.0, &chunk, &limits) {
1072 *found.borrow_mut() = Some(violation);
1073 return Err(worker::Error::RustError("push over the size limit".into()));
1074 }
1075 Ok(chunk)
1076 })
1077 };
1078 let first = head.len() as u64;
1079 let body = futures_util::stream::once(async move { Ok::<Vec<u8>, worker::Error>(head) }).chain(rest);
1080 init.with_body(Some(stream_body(body)?));
1081 let answered = Fetch::Request(Request::new_with_init(url, &init)?).send().await;
1082 if let Some(violation) = found.borrow_mut().take() {
1083 let (reason, messages) = size_refusal(&violation);
1084 return Ok(Push::Declined(declined(&commands, &reason, &messages)?, reason));
1085 }
1086 let walked = walked.borrow();
1087 let pack = walked.0.as_ref().map_or_else(|| pack_bytes(&commands), PackSizer::pack_bytes);
1088 (answered, pack, first + walked.1, true)
1089 };
1090 let ms = g1t_kit::now_ms().saturating_sub(started);
A push says where its time went: receiving it, the rules, the secret scan and the git store's answer each have their own Server-Timing step1091 timing.mark("upload");
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1092 let failure = match &answered {
1093 Ok(response) => resilience::classify_status(response.status_code()),
1094 Err(_) => Some(Failure::Transient),
1095 };
1096 meters::record_health(
1097 namespace,
1098 match failure {
1099 None => meters::Outcome::Ok,
1100 Some(Failure::RateLimited) => meters::Outcome::RateLimited,
1101 Some(_) => meters::Outcome::Failed,
1102 },
1103 ms,
1104 );
1105 // A push is never tried again: the store may have taken it.
1106 let response = match (answered, failure) {
1107 (Ok(response), None) => response,
1108 (Ok(response), Some(Failure::RateLimited)) => {
1109 drop(response);
Merge branch 'worktree-agent-a2013627e5ea4ab13'1110 busy_response(Busy { rate_limited: true, retry_after: 5, read_only: false })?
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1111 }
1112 (Ok(response), Some(_)) => response,
1113 (Err(error), _) => {
1114 worker::console_error!("a push did not reach the store: {error}");
Merge branch 'worktree-agent-a2013627e5ea4ab13'1115 busy_response(Busy { rate_limited: false, retry_after: 5, read_only: false })?
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1116 }
1117 };
1118 Ok(Push::Forwarded(Forwarded {
1119 response,
g1t's agents only for listed workspaces, whatever the state of billing1120 pushed,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1121 pack_bytes,
1122 sent,
1123 from_store: true,
1124 unscanned,
g1t's agents only for listed workspaces, whatever the state of billing1125 }))
1126}
1127
1128#[cfg(test)]
1129mod tests {
Merge branch 'worktree-agent-a8385d293d42c913a'1130 use super::{Acknowledged, GitService, Pushed, RepoPath, Url, ZERO_ID, acknowledged, framed, negotiating, pack_bytes, parse, pushed_branches, refusal, server_timing, transferred, with_head, with_namespace};
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms1131
1132 #[test]
1133 fn server_timing_names_each_step_and_the_total() {
1134 assert_eq!(
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1135 server_timing(&[("repo", 12), ("token", 0), ("store", 140)], &[], &[], 153),
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms1136 "repo;dur=12, token;dur=0, store;dur=140, total;dur=153"
1137 );
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1138 assert_eq!(server_timing(&[], &[], &[], 3), "total;dur=3");
1139 // A push's checks run side by side: their parts come after the steps.
1140 assert_eq!(
1141 server_timing(&[("recv", 30), ("checks", 120), ("upload", 300)], &[("read", 40), ("rules", 110), ("scan", 90)], &[], 450),
1142 "recv;dur=30, checks;dur=120, upload;dur=300, read;dur=40, rules;dur=110, scan;dur=90, total;dur=450"
1143 );
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms1144 assert_eq!(
A push is checked once and side by side: its pack is read and its bases fetched once for the rules, the workflow gate and the secret scan, which run together, other services are asked while the pack is read, and cache writes and rule records finish after git has its answer1145 server_timing(&[("repo", 1), ("cache", 2)], &[], &[("refs", "hit-colo")], 4),
Mission control shows where you are needed and what agents landed without you; git answers in about 200ms1146 "repo;dur=1, cache;dur=2, refs;desc=hit-colo, total;dur=4"
1147 );
1148 }
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1149
1150 #[test]
1151 fn a_renamed_repository_redirects_to_its_new_name() {
1152 // A rename keeps the old path in the same table as a transfer, so
1153 // the old remote is sent to the new name the same way.
1154 let to = RepoPath {
1155 namespace: "acme".into(),
1156 name: "booster".into(),
1157 };
1158 let url = Url::parse("https://g1t.sh/acme/rocket.git/info/refs?service=git-upload-pack").unwrap();
1159 assert_eq!(
1160 transferred(&url, &to).as_deref(),
1161 Some("https://g1t.sh/acme/booster.git/info/refs?service=git-upload-pack")
1162 );
1163 }
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put1164
1165 #[test]
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1166 fn head_follows_the_default_branch_in_a_v0_advertisement() {
1167 let main = "1111111111111111111111111111111111111111";
1168 let trunk = "2222222222222222222222222222222222222222";
1169 let body = [
1170 pkt("# service=git-upload-pack\n"),
1171 b"0000".to_vec(),
1172 pkt(&format!("{main} HEAD\0multi_ack symref=HEAD:refs/heads/main agent=git/2\n")),
1173 pkt(&format!("{main} refs/heads/main\n")),
1174 pkt(&format!("{trunk} refs/heads/trunk\n")),
1175 b"0000".to_vec(),
1176 ]
1177 .concat();
1178 let changed = String::from_utf8(with_head(&body, "trunk").unwrap()).unwrap();
1179 assert!(changed.contains(&format!("{trunk} HEAD\0multi_ack symref=HEAD:refs/heads/trunk agent=git/2\n")));
1180 assert!(changed.contains(&format!("{main} refs/heads/main\n")));
1181 assert!(changed.starts_with("001e# service=git-upload-pack\n0000"));
1182 // Already right, or a branch it does not have: left alone.
1183 assert!(with_head(&body, "main").is_none());
1184 assert!(with_head(&body, "gone").is_none());
1185 }
1186
1187 #[test]
1188 fn head_follows_the_default_branch_in_a_v2_listing() {
1189 let main = "1111111111111111111111111111111111111111";
1190 let trunk = "2222222222222222222222222222222222222222";
1191 let body = [
1192 pkt(&format!("{main} HEAD symref-target:refs/heads/main\n")),
1193 pkt(&format!("{main} refs/heads/main\n")),
1194 pkt(&format!("{trunk} refs/heads/trunk\n")),
1195 b"0000".to_vec(),
1196 ]
1197 .concat();
1198 let changed = String::from_utf8(with_head(&body, "trunk").unwrap()).unwrap();
1199 assert!(changed.starts_with(&String::from_utf8(pkt(&format!("{trunk} HEAD symref-target:refs/heads/trunk\n"))).unwrap()));
1200 assert!(changed.ends_with("0000"));
1201 // Without symrefs asked for, only the commit changes.
1202 let plain = [pkt(&format!("{main} HEAD\n")), pkt(&format!("{trunk} refs/heads/trunk\n")), b"0000".to_vec()].concat();
1203 let changed = String::from_utf8(with_head(&plain, "trunk").unwrap()).unwrap();
1204 assert!(changed.starts_with(&format!("0032{trunk} HEAD\n")));
1205 }
1206
1207 #[test]
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put1208 fn a_push_is_measured_by_the_pack_after_its_commands() {
1209 let old = "c71546fcd893ef8b0f57388b65e620d759705dda";
1210 let new = "4807077b296e6edbf410d55e72749d3e1170c291";
1211 let pack = b"PACK\0\0\0\x02\0\0\0\0rest-of-pack";
1212 let body = [
1213 pkt(&format!("{old} {new} refs/heads/main\0 report-status\n")),
1214 b"0000".to_vec(),
1215 pack.to_vec(),
1216 ]
1217 .concat();
1218 assert_eq!(pack_bytes(&body), pack.len() as u64);
1219 // Only deletions: no pack.
1220 let body = [pkt(&format!("{old} {ZERO_ID} refs/heads/gone\n")), b"0000".to_vec()].concat();
1221 assert_eq!(pack_bytes(&body), 0);
1222 assert_eq!(pack_bytes(b"garbage"), 0);
1223 }
Agents and memory, checks and conflicts, profiles, slug renames, custom domains1224
1225 #[test]
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1226 fn a_transferred_repository_keeps_the_rest_of_the_address() {
1227 let to = RepoPath {
1228 namespace: "flagon-io".into(),
1229 name: "g1t".into(),
1230 };
1231 let url = Url::parse("https://g1t.sh/syntaqx/g1t.git/info/refs?service=git-receive-pack").unwrap();
1232 assert_eq!(
1233 transferred(&url, &to).as_deref(),
1234 Some("https://g1t.sh/flagon-io/g1t.git/info/refs?service=git-receive-pack")
1235 );
1236 let url = Url::parse("https://g1t.sh/syntaqx/g1t/git-upload-pack").unwrap();
1237 assert_eq!(
1238 transferred(&url, &to).as_deref(),
1239 Some("https://g1t.sh/flagon-io/g1t/git-upload-pack")
1240 );
1241 }
1242
1243 #[test]
Merge branch 'worktree-agent-a8385d293d42c913a'1244 fn an_alias_is_answered_as_its_workspaces_repository() {
1245 for (address, service) in [
1246 ("https://g1t.sh/g1t/g1t.git/info/refs?service=git-upload-pack", GitService::UploadPack),
1247 ("https://g1t.sh/g1t/g1t.git/git-receive-pack", GitService::ReceivePack),
1248 ("https://g1t.sh/g1t/g1t/git-upload-pack", GitService::UploadPack),
1249 ] {
1250 let git = parse(&Url::parse(address).unwrap()).unwrap();
1251 assert_eq!(git.path.namespace, "g1t", "{address}");
1252 let canonical = git.under("flagon-io");
1253 assert_eq!(
1254 canonical.path,
1255 RepoPath {
1256 namespace: "flagon-io".into(),
1257 name: "g1t".into(),
1258 },
1259 "{address}"
1260 );
1261 assert_eq!(canonical.service, service);
1262 assert_eq!(canonical.endpoint, git.endpoint);
1263 }
1264 }
1265
1266 #[test]
Agents and memory, checks and conflicts, profiles, slug renames, custom domains1267 fn a_renamed_workspace_keeps_the_rest_of_the_address() {
1268 let url = worker::Url::parse(
1269 "https://g1t.sh/acme/rocket.git/info/refs?service=git-upload-pack",
1270 )
1271 .unwrap();
1272 assert_eq!(
1273 with_namespace(&url, "acme-inc").as_deref(),
1274 Some("https://g1t.sh/acme-inc/rocket.git/info/refs?service=git-upload-pack")
1275 );
1276 let bare = worker::Url::parse("https://g1t.sh/acme").unwrap();
1277 assert_eq!(with_namespace(&bare, "acme-inc"), None);
1278 }
g1t's agents only for listed workspaces, whatever the state of billing1279
1280 fn pkt(payload: &str) -> Vec<u8> {
1281 format!("{:04x}{payload}", payload.len() + 4).into_bytes()
1282 }
1283
A fetch whose haves the store does not know gets its pack: g1t ends the store's early answer where git expects1284 fn joined(parts: &[&[u8]]) -> Vec<u8> {
1285 parts.concat()
1286 }
1287
1288 #[test]
1289 fn a_fetch_with_haves_and_no_done_is_negotiating() {
1290 let request = |lines: &[&str]| {
1291 let mut body = joined(&[&pkt("command=fetch\n"), &pkt("object-format=sha1\n"), b"0001"]);
1292 for line in lines {
1293 body.extend(pkt(&format!("{line}\n")));
1294 }
1295 body.extend(b"0000");
1296 body
1297 };
1298 let want = "want 8407eba58b925619274d012258c2b474a5dbf012";
1299 let have = "have 55cd670a89a80df4fa9d9f0244c44fbd2ed1db8b";
1300 assert!(negotiating(&request(&["deepen 1", want, have])));
1301 assert!(!negotiating(&request(&[want, have, "done"])));
1302 assert!(!negotiating(&request(&[want, "done"])));
1303 let ls_refs = joined(&[&pkt("command=ls-refs\n"), b"0001", &pkt("have nothing\n"), b"0000"]);
1304 assert!(!negotiating(&ls_refs));
1305 }
1306
1307 #[test]
1308 fn acknowledgments_without_ready_end_the_answer() {
1309 // What the store sent a shallow fetch whose only `have` it did not
1310 // know, sideband-all framed: a NAK, then a pack anyway.
1311 let answer = joined(&[
1312 &pkt("\x01acknowledgments\n"),
1313 &pkt("\x01NAK\n"),
1314 b"0001",
1315 &pkt("\x01shallow-info\n"),
1316 &pkt("\x01shallow 8407eba58b925619274d012258c2b474a5dbf012\n"),
1317 b"0001",
1318 &pkt("\x01packfile\n"),
1319 ]);
1320 let cut = joined(&[&pkt("\x01acknowledgments\n"), &pkt("\x01NAK\n")]).len();
1321 assert_eq!(acknowledged(&answer), Acknowledged::CutAt(cut));
1322 // Not yet at the section's end.
1323 assert_eq!(acknowledged(&answer[..cut - 2]), Acknowledged::NeedMore);
1324 assert_eq!(acknowledged(&answer[..cut]), Acknowledged::NeedMore);
1325 // Without sideband framing too.
1326 let plain = joined(&[&pkt("acknowledgments\n"), &pkt("ACK abc\n"), b"0001", &pkt("packfile\n")]);
1327 assert!(matches!(acknowledged(&plain), Acknowledged::CutAt(_)));
1328 }
1329
1330 #[test]
1331 fn a_ready_store_or_a_plain_pack_streams_through() {
1332 let ready = joined(&[
1333 &pkt("\x01acknowledgments\n"),
1334 &pkt("\x01ACK bab14ff1b6d9c4918100098009747d776759a967\n"),
1335 &pkt("\x01ready\n"),
1336 b"0001",
1337 &pkt("\x01packfile\n"),
1338 ]);
1339 assert_eq!(acknowledged(&ready), Acknowledged::Whole);
1340 // Acknowledgments only, ended by a flush: already right.
1341 let only = joined(&[&pkt("acknowledgments\n"), &pkt("NAK\n"), b"0000"]);
1342 assert_eq!(acknowledged(&only), Acknowledged::Whole);
1343 let pack = joined(&[&pkt("\x01packfile\n"), b"0000"]);
1344 assert_eq!(acknowledged(&pack), Acknowledged::Whole);
1345 assert_eq!(acknowledged(b"00"), Acknowledged::NeedMore);
1346 }
1347
g1t's agents only for listed workspaces, whatever the state of billing1348 #[test]
1349 fn pushed_branches_are_read_from_the_commands() {
1350 let old = "c71546fcd893ef8b0f57388b65e620d759705dda";
1351 let new = "4807077b296e6edbf410d55e72749d3e1170c291";
1352 let body = [
1353 pkt(&format!(
1354 "{old} {new} refs/heads/main\0 report-status side-band-64k\n"
1355 )),
1356 pkt(&format!("{ZERO_ID} {new} refs/heads/feature/x\n")),
1357 pkt(&format!("{old} {ZERO_ID} refs/heads/gone\n")),
1358 pkt(&format!("{ZERO_ID} {new} refs/tags/v1\n")),
1359 b"0000".to_vec(),
1360 b"PACK\0\0\0\x02\0\0\0\0".to_vec(),
1361 ]
1362 .concat();
1363 assert_eq!(
1364 pushed_branches(&body),
1365 [
GitHub Actions on g1t, part one: reading workflows1366 Pushed {
1367 git_ref: "refs/heads/main".to_owned(),
1368 before: Some(old.to_owned()),
1369 after: new.to_owned()
1370 },
1371 Pushed {
1372 git_ref: "refs/heads/feature/x".to_owned(),
1373 before: None,
1374 after: new.to_owned()
1375 },
1376 Pushed {
1377 git_ref: "refs/tags/v1".to_owned(),
1378 before: None,
1379 after: new.to_owned()
1380 },
g1t's agents only for listed workspaces, whatever the state of billing1381 ]
1382 );
1383 }
1384
1385 #[test]
1386 fn a_push_to_a_protected_branch_is_declined_with_the_reason() {
1387 let old = "c71546fcd893ef8b0f57388b65e620d759705dda";
1388 let new = "4807077b296e6edbf410d55e72749d3e1170c291";
1389 let body = [
1390 pkt(&format!("{old} {new} refs/heads/main\0 report-status\n")),
1391 pkt(&format!("{ZERO_ID} {new} refs/heads/feature\n")),
1392 b"0000".to_vec(),
1393 ]
1394 .concat();
1395 let report = String::from_utf8(refusal(&body, "main").unwrap()).unwrap();
1396 assert!(report.starts_with("000eunpack ok\n"));
1397 assert!(report.contains("ng refs/heads/main main is protected"));
1398 assert!(report.contains("ng refs/heads/feature not pushed"));
1399 assert!(report.ends_with("0000"));
1400 }
1401
1402 #[test]
1403 fn the_report_is_framed_for_a_client_that_asked_for_side_band() {
1404 let old = "c71546fcd893ef8b0f57388b65e620d759705dda";
1405 let body = [
1406 pkt(&format!(
1407 "{old} {ZERO_ID} refs/heads/main\0 report-status side-band-64k\n"
1408 )),
1409 b"0000".to_vec(),
1410 ]
1411 .concat();
1412 let report = refusal(&body, "main").unwrap();
1413 // A length, then channel 1, then the report itself.
1414 assert_eq!(report[4], 1);
1415 assert_eq!(&report[5..18], b"000eunpack ok");
1416 assert!(report.ends_with(b"00000000"));
1417 }
1418
1419 #[test]
1420 fn other_branches_and_a_first_push_are_let_through() {
1421 let old = "c71546fcd893ef8b0f57388b65e620d759705dda";
1422 let new = "4807077b296e6edbf410d55e72749d3e1170c291";
1423 let feature = [
1424 pkt(&format!("{old} {new} refs/heads/feature\0 report-status\n")),
1425 b"0000".to_vec(),
1426 ]
1427 .concat();
1428 assert!(refusal(&feature, "main").is_none());
1429 // An empty repository has to be able to receive its first commits.
1430 let first = [
1431 pkt(&format!(
1432 "{ZERO_ID} {new} refs/heads/main\0 report-status\n"
1433 )),
1434 b"0000".to_vec(),
1435 ]
1436 .concat();
1437 assert!(refusal(&first, "main").is_none());
1438 }
1439
1440 #[test]
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1441 fn a_blocked_push_explains_itself_on_the_progress_channel() {
1442 let report = b"000eunpack ok\n0000".to_vec();
1443 let messages = vec!["g1t found a secret in this push, so nothing was pushed.".to_owned()];
1444 let body = framed(report.clone(), "report-status side-band-64k", &messages);
1445 // Channel 2 first, which git prints as `remote:` lines.
1446 assert_eq!(body[4], 2);
1447 assert!(String::from_utf8_lossy(&body).contains("so nothing was pushed.\n"));
1448 let at = body.windows(5).position(|w| w == b"000eu").unwrap();
1449 assert_eq!(body[at - 1], 1);
1450 assert!(body.ends_with(b"0000"));
1451 // A client without side-band gets the bare report.
1452 assert_eq!(framed(report.clone(), "report-status", &messages), report);
1453 }
1454
1455 #[test]
g1t's agents only for listed workspaces, whatever the state of billing1456 fn a_fetch_request_names_no_branches() {
1457 assert!(
1458 pushed_branches(b"0032want c71546fcd893ef8b0f57388b65e620d759705dda\n0000").is_empty()
1459 );
1460 }
1461}

This file's history is long; its oldest lines are credited to the oldest commit read.