Skip to content

Commit

architecture and such

syntaqxcommitted Parent80567f2Browse files
31 files+3125−680/31 viewed
+2−1
11 # Generated account passwords (services/identity/scripts/create-user.mjs)
22 /.credentials/
3+/.secrets/
34
45 node_modules/
56 target/
1415 wrangler.deploy.json
1516
1617 # Personal notes for the next session
17−/TOMORROW.md
18+/SCRUTINY.md
1819 /TODO.md
+4−2
1717 | How agents behave | `guides/working-with-g1t.md`, and `apps/web/public/llms.txt` |
1818 | Settings, limits or prices | The page that names them, and the table it is in |
1919 | A new feature | A section in the guide that owns it, linked from the docs home if it is a new task |
20+| How g1t is put together: a service, a binding, a store, an event route, the path of a request | Its page in the docs' Architecture section (`apps/docs/src/content/docs/architecture/`), and its diagram in `apps/docs/src/diagrams/architecture.ts`. [How g1t ships](https://docs.g1t.sh/architecture/shipping/#keeping-this-section-true) says which page covers what. |
2021
2122 The style, in short: plain sentences, second person, no marketing words,
2223 sentence-case headings, a numbered list for steps, a table for options, and
112113 Pushes to `main` deploy themselves: `.g1t/workflows/deploy.yml` runs
113114 `scripts/deploy.mjs`, which deploys only the parts that changed, migrations
114115 first. Every deployable part is listed in `deploy/stack.jsonc`; a new service
115−or app goes there, and in the table on
116−[How a self-hosted g1t runs](https://docs.g1t.sh/guides/self-hosting-architecture/#each-part)
116+or app goes there, in the table on
117+[How a self-hosted g1t runs](https://docs.g1t.sh/guides/self-hosting-architecture/#each-part),
118+and in the one on [Services and contracts](https://docs.g1t.sh/architecture/services/#every-unit)
117119 (`npm run test:deploy` says what is missing).
118120 [Deploy g1t to Cloudflare](https://docs.g1t.sh/guides/deploy-to-cloudflare/)
119121 covers the tool, the workflow, rollbacks, adding a unit and first-time
+16−0
210210 ],
211211 },
212212 {
213+ label: 'Architecture',
214+ items: [
215+ { label: 'Overview', slug: 'architecture/overview' },
216+ { label: 'How a request is served', slug: 'architecture/requests' },
217+ { label: 'Services and contracts', slug: 'architecture/services' },
218+ { label: 'Git', slug: 'architecture/git' },
219+ { label: 'Events', slug: 'architecture/events' },
220+ { label: 'Work and the merge queue', slug: 'architecture/work' },
221+ { label: 'Actions, runners and sandboxes', slug: 'architecture/compute' },
222+ { label: 'Deployments and telemetry', slug: 'architecture/deployments' },
223+ { label: 'Data and storage', slug: 'architecture/data' },
224+ { label: 'Identity and access', slug: 'architecture/identity' },
225+ { label: 'How g1t ships', slug: 'architecture/shipping' },
226+ ],
227+ },
228+ {
213229 label: 'Reference',
214230 items: [
215231 { label: 'API overview', slug: 'reference/api' },
+2−1
66 "dev": "astro dev",
77 "build": "astro build",
88 "preview": "astro preview",
9− "deploy": "astro build && wrangler deploy"
9+ "deploy": "astro build && wrangler deploy",
10+ "test": "node --test \"src/diagrams/*.test.ts\""
1011 },
1112 "dependencies": {
1213 "@astrojs/starlight": "^0.42.5",
+72−0
1+---
2+// A diagram from src/diagrams, by name: drawn as an inline SVG in the g1t
3+// tokens, with its caption, a key, and Expand, which opens it at full size
4+// in a dialog. See src/diagrams/engine.ts.
5+import { DIAGRAMS } from '../diagrams/index.ts';
6+import { render } from '../diagrams/engine.ts';
7+import LucideIcon from './LucideIcon.astro';
8+
9+interface Props {
10+ name: string;
11+}
12+
13+const { name } = Astro.props;
14+const spec = DIAGRAMS[name];
15+if (!spec) throw new Error(`No diagram named "${name}". Add it to src/diagrams/index.ts.`);
16+// The dialog's copy has its own ids, so its arrowheads and title never
17+// point at the page's copy.
18+const svg = render(`d-${name}`, spec);
19+const large = render(`d-${name}-full`, spec);
20+const asyncUsed = spec.kind === 'graph' ? spec.edges.some((e) => e.async) : spec.steps.some((s) => 'from' in s && (s.async || s.reply));
21+---
22+
23+<figure class="g1t-diagram not-content" data-diagram>
24+ <div class="g1t-diagram__scroll" tabindex="0" role="group" aria-label={`${spec.caption}. Scrolls sideways on a narrow screen.`}>
25+ <Fragment set:html={svg} />
26+ </div>
27+ <figcaption>
28+ <span>{spec.caption}</span>
29+ <span class="g1t-diagram__tools">
30+ <span class="g1t-diagram__key" aria-hidden="true">
31+ <span><svg width="22" height="8"><line x1="1" y1="4" x2="21" y2="4" class="d-key" /></svg>
32+ {spec.kind === 'sequence' ? 'call' : 'calls'}</span>
33+ {
34+ asyncUsed && (
35+ <span><svg width="22" height="8"><line x1="1" y1="4" x2="21" y2="4" class="d-key d-async" /></svg>
36+ {spec.kind === 'sequence' ? 'answer or event' : 'event, queue or stream'}</span>
37+ )
38+ }
39+ </span>
40+ <button type="button" class="g1t-diagram__expand" data-diagram-open aria-haspopup="dialog">
41+ <LucideIcon name="maximize-2" size="0.875rem" />
42+ Expand
43+ </button>
44+ </span>
45+ </figcaption>
46+ <dialog class="g1t-diagram__dialog" aria-label={spec.caption}>
47+ <div class="g1t-diagram__dialog-bar">
48+ <span>{spec.caption}</span>
49+ <button type="button" class="g1t-diagram__expand" data-diagram-close>
50+ <LucideIcon name="x" size="0.875rem" />
51+ Close
52+ </button>
53+ </div>
54+ <div class="g1t-diagram g1t-diagram__dialog-body">
55+ <div class="g1t-diagram__scroll"><Fragment set:html={large} /></div>
56+ </div>
57+ </dialog>
58+</figure>
59+
60+<script>
61+ // Expand opens the figure's dialog; Close, Escape or a click on the
62+ // backdrop closes it.
63+ for (const figure of document.querySelectorAll<HTMLElement>('[data-diagram]')) {
64+ const dialog = figure.querySelector('dialog');
65+ if (!dialog) continue;
66+ figure.querySelector('[data-diagram-open]')?.addEventListener('click', () => dialog.showModal());
67+ dialog.querySelector('[data-diagram-close]')?.addEventListener('click', () => dialog.close());
68+ dialog.addEventListener('click', (event) => {
69+ if (event.target === dialog) dialog.close();
70+ });
71+ }
72+</script>
+158−0
1+---
2+title: Actions, runners and sandboxes
3+description: How a workflow job is planned and run, on g1t's runners or yours, what walls each job in, how secrets reach a job without the job holding them, and what else runs in a sandbox.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+Everything that runs code you did not write runs in a sandbox: workflow
10+jobs, merge queue builds, deployments, dependency updates, backups. Three
11+pieces share the work.
12+
13+| Piece | Is | Does |
14+| --- | --- | --- |
15+| `services/actions` | A Rust Worker, D1 `g1t-actions` | Reads workflows, starts runs, plans jobs, keeps logs, secrets and self-hosted runners |
16+| `services/runner` | A TypeScript Worker on Cloudflare Containers | Starts a sandbox per job, and owns what leaves it |
17+| `crates/runner` | The `g1t-runner` binary | Runs inside every sandbox and on every self-hosted runner, does one job, and exits |
18+
19+`crates/actions` holds the workflow language itself, with no I/O:
20+syntax, `${{ }}` expressions, filters, matrices and cron. The Worker plans
21+with it and the sandbox runs with it, so the two cannot read a workflow
22+differently.
23+
24+## A workflow job
25+
26+<Diagram name="actions-job" />
27+
28+1. **Trigger.** `actions` reads its event queue, `g1t-events-actions`, ten
29+ at a time (step 1), and matches each event to the workflows read from
30+ the default branch's `.g1t/workflows` (or `.github/workflows`). The
31+ files are GitHub Actions syntax, unchanged; [GitHub
32+ Actions](/guides/actions/) lists what is supported.
33+2. **Plan.** It expands matrices, orders jobs by `needs`, and reports the
34+ run as pending on the commit (steps 2 and 3).
35+3. **Start.** A hosted job is taken by flipping its row from `queued` to
36+ `in_progress` in one statement, so two starters can never both take it.
37+ `actions` asks `runner` for a sandbox (step 4), and `runner` starts a
38+ Container for that job alone (step 5).
39+4. **Run.** The sandbox reports its steps and logs to the API, with the
40+ job's own token (step 6), which passes them to `actions` (step 7). The
41+ answer can tell the job to stop, which is how **Cancel** reaches a
42+ running job (step 8).
43+5. **Finish.** The run's conclusion goes onto the commit as a status
44+ (step 9), where [required checks](/architecture/work/#checks-and-statuses)
45+ find it, and `workflow.completed` is published (step 10).
46+
47+A job that says nothing for 10 minutes is taken as lost. A cron in
48+`actions` runs every minute for schedules, jobs waiting for room, and
49+jobs whose runner went quiet.
50+
51+### Limits
52+
53+| | |
54+| --- | --- |
55+| Hosted jobs running at once | 4 per workspace |
56+| A hosted job's timeout | 60 minutes at most |
57+| A self-hosted job's timeout | 24 hours at most |
58+| Logs | 4 MiB a job, kept 90 days |
59+| Workflows per repository | 50 |
60+| Schedules | Paused after 60 days without a push |
61+
62+### Logs, caches and artifacts
63+
64+Logs are kept in D1, in chunks, deduplicated so a report sent twice is
65+stored once. Cache entries and artifacts go to the R2 bucket
66+`g1t-actions-cache`, uploaded through the API: cache entries under `c/`
67+for 30 days, artifacts under `a/` for 91. `actions` lists them and decides
68+what is kept.
69+
70+## Sandboxes on g1t's runners
71+
72+`runner` defines three Container classes, picked by `runs-on`:
73+
74+| `runs-on` | Class | Size |
75+| --- | --- | --- |
76+| Any other hosted label, and every other kind of work | `AttemptSandbox` | Cloudflare's `standard-1` |
77+| `g1t-2core` | `Sandbox2Core` | 2 vCPU, 8 GiB memory, 16 GB disk |
78+| `g1t-4core` | `Sandbox4Core` | 4 vCPU, 12 GiB memory, 20 GB disk |
79+
80+Every sandbox starts from one image: `services/runner/base/Dockerfile`,
81+which has Node, Python, Go, Rust, Java, .NET, Ruby and Docker, with the
82+`g1t-runner` binary on top. Hosted runners are Linux only.
83+
84+<Diagram name="sandboxes" />
85+
86+### Egress
87+
88+A sandbox starts with no internet. Every HTTP and HTTPS request it makes
89+goes through the runner Worker's outbound handler (`src/egress.ts`,
90+`src/guard.ts`), which decides what may leave. Other protocols have no
91+route out at all.
92+
93+### Secrets a job never holds
94+
95+A secret can be **brokered**: the job sees a stand-in, `g1tsec_…`, and
96+the egress handler swaps in the real value only on a request to a host
97+the secret allows (`src/broker.ts`). On a self-hosted runner, the job sends
98+the request to `POST /actions/jobs/{job}/broker` instead, and g1t makes
99+it, so the value never reaches your machine at all.
100+
101+## Runners of your own
102+
103+A [self-hosted runner](/guides/self-hosted-runners/) only ever calls out,
104+over HTTPS, so it needs no open port:
105+
106+| Call | What it does |
107+| --- | --- |
108+| `POST /runners/register` | Trades a registration token, `g1trt_…`, valid for an hour, for a runner credential, `g1tr_…` |
109+| `POST /runners/{id}/poll` | Waits up to 20 seconds for a job; doubles as the heartbeat |
110+| `POST /runners/{id}/finished` | Says the job is over |
111+| `POST /runners/{id}/remove` | Signs the runner out |
112+
113+The runner credential is stored only as a hash and rotates every day on a
114+poll. A runner that has not polled for 90 seconds is offline; one offline
115+for 14 days is removed. A job goes to a runner that has every one of its
116+`runs-on` labels and whose group allows the repository, and waits up to 24
117+hours for one.
118+
119+### Three walls
120+
121+On your machine, `g1t-runner` puts three walls between a job and the
122+host:
123+
124+1. **A VM.** WSL2 on Windows, with drives, interop and the Windows `PATH`
125+ turned off, vfkit on macOS, or QEMU on Linux (`deploy/sandbox`).
126+2. **A Sysbox container**, the runner itself, with its own dockerd.
127+ It refuses to start if the host's `docker.sock` is mounted
128+ (`deploy/runner`).
129+3. **A container per job**, new each time.
130+
131+Native Windows jobs run in Windows Sandbox instead. Each runner reports
132+its isolation, `nested-container`, `vm`, `container` or `host`, and a
133+`host` runner never gets a public repository's jobs. The [desktop
134+app](/guides/desktop/) runs the sandboxed runner by default while it is
135+open; Docker or direct execution happen only if you choose them.
136+
137+## Other work in sandboxes
138+
139+`g1t-runner` takes a `MODE`, so the same image does more than workflows:
140+
141+| Mode | Started by | Does |
142+| --- | --- | --- |
143+| `actions` | `actions` | A workflow job |
144+| `queue` | `runner`, when the queue changes | Builds a merge queue candidate |
145+| `mergecheck` | `runner`, when a pull request changes | Checks whether a pull request merges cleanly |
146+| `deploy` | `deployments` | Builds an app ([Deployments](/architecture/deployments/)) |
147+| `bump` | `security` | Makes a dependency update |
148+| `backup` | `runner`'s sweep, every 5 minutes | Cuts a repository backup ([Git](/architecture/git/#backups)) |
149+
150+Compute is admitted through billing: a sandbox starts only after
151+`billing` holds its estimated cost (`reserve`), and the hold is settled
152+with the real cost when it ends (`settle`).
153+
154+<Aside type="caution" title="Self-hosted installations">
155+ A [self-hosted g1t](/guides/self-hosting-architecture/) binds `runner`
156+ to a Worker that answers that it is off, so workflows, the merge queue
157+ and deployments do not run there yet.
158+</Aside>
+112−0
1+---
2+title: Data and storage
3+description: Which service owns which database, bucket and Durable Object, the few stores that are shared, how schema changes ship, and the ports that let the same code run on Cloudflare or in Docker.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+## Who owns what
10+
11+Each D1 database is bound by exactly one Worker. If you need another
12+service's data, call that service; never read its tables.
13+
14+| Service | D1 | Other storage |
15+| --- | --- | --- |
16+| `identity` | `g1t` | Writes the avatars KV |
17+| `repos` | `g1t-repos` | The git store (Artifacts), the git cache KV, the `g1t-git-packs` and `g1t-backups` buckets |
18+| `work` | `g1t-work` | |
19+| `events` | `g1t-events` | The `g1t-events` queue, which it produces and consumes, and the subscriber queues it produces |
20+| `actions` | `g1t-actions` | Lists and expires the `g1t-actions-cache` bucket |
21+| `billing` | `g1t-billing` | |
22+| `integrations` | `g1t-integrations` | |
23+| `webhooks` | `g1t-webhooks` | |
24+| `security` | `g1t-security` | |
25+| `search` | `g1t-search` | The `g1t-search-jobs` queue |
26+| `packages` | `g1t-packages` | The `g1t-packages` bucket |
27+| `projects` | `g1t-projects` | Writes project images to the avatars KV |
28+| `chat` | `g1t-chat` | A `ChannelRoom` Durable Object per channel; writes custom emoji to the avatars KV |
29+| `artifacts` | `g1t-docs` | The `g1t-docs-files` bucket, the `g1t-docs` and `g1t-folios` Vectorize indexes, `PageRoom` and `FolioRoom` Durable Objects |
30+| `deployments` | `g1t-deployments` | Writes the domains KV; the `g1t-deployments` dispatch namespace |
31+| `apps/status` | `g1t-status` | |
32+| `notify` | | `Feed`, `Room` and `Topic` Durable Objects |
33+| `telemetry` | | A `TelemetryStore` Durable Object per project |
34+| `runner` | | The sandbox Container classes |
35+| `og` | | The `g1t-screenshots` bucket |
36+| `apps/web` | | The `g1t-downloads` bucket |
37+
38+### The shared stores
39+
40+Three stores are shared on purpose, each with a single writer per key
41+prefix:
42+
43+| Store | Written by | Read by |
44+| --- | --- | --- |
45+| `g1t-avatars` KV | `identity` (people and workspaces), `projects` (`production/`), `chat` (`emoji/`) | The site, `og` |
46+| `g1t-domains` KV | `deployments` | `pages`, `telemetry` |
47+| `g1t-actions-cache` R2 | The API, as jobs upload | `actions`, which decides what is kept |
48+
49+## Durable Objects
50+
51+Durable Objects are used where something has to be live, or where data
52+belongs to one thing and grows on its own: a chat channel, an open doc, a
53+person's open tabs, a project's telemetry. Each object keeps its own
54+SQLite storage. A rule from CONTRIBUTING.md: never wake an object for a
55+read that D1 can answer.
56+
57+## Secrets at rest
58+
59+Secrets that g1t has to use later, such as webhook secrets, Actions
60+secrets, GitHub tokens of linked accounts and two-factor seeds, are sealed
61+with AES-256-GCM by `Sealer` in `crates/secrets`, with the row's id as
62+associated data. A sealed value copied into another row will not open.
63+Each service seals with its own key (`IDENTITY_KEY`, `ACTIONS_KEY`,
64+`REPOS_KEY`, …), set as a Worker secret.
65+
66+Access tokens and session cookies are not stored at all: only their
67+SHA-256 is.
68+
69+## Schema changes
70+
71+Each database's migrations live beside its service, in
72+`services/<name>/migrations`, numbered. A deploy applies every pending
73+migration before any code ships
74+([How g1t ships](/architecture/shipping/)), so for a minute or more the old
75+code runs against the new schema. Three rules follow:
76+
77+1. Add tables and columns. Never rename or drop one that live code reads.
78+2. Change what a value means in two deploys: first code that reads both
79+ forms, then the change.
80+3. Remove what nothing reads anymore only after the code that read it is
81+ gone.
82+
83+## Ports and adapters
84+
85+Each Cloudflare product a service depends on sits behind a port: a trait
86+in Rust, or a binding whose shape a stand-in can copy. g1t.sh binds
87+Cloudflare's adapters. A self-hosted installation, built from the same
88+source, binds open ones, and the service code does not change.
89+
90+<Diagram name="ports-and-adapters" />
91+
92+| Port | Defined in | g1t.sh | Self-hosted |
93+| --- | --- | --- | --- |
94+| `GitStore`, `GitRepo` | `services/repos/src/store.rs` | `ArtifactsStore` | A Worker with the same API as the Artifacts binding (`deploy/self-host/workers/gitstore`), in front of a server of bare repositories using `git http-backend` |
95+| `BlobStore` | `crates/blobstore` | `R2Store` | `S3Store`, with AWS Signature V4, for RustFS, MinIO, Ceph, Garage or AWS S3 |
96+| `PackStore` | `services/repos/src/pack_cache.rs` | R2 | S3, with `PACK_STORE=s3` |
97+| `EMAIL` | A `send_email` binding | Cloudflare Email Sending | A mail Worker that hands each message to Mailpit |
98+| `RUNNER` | A service binding | `services/runner` on Containers | A Worker that answers that the feature is off |
99+| D1, KV, Queues, Durable Objects | Workers bindings | Cloudflare | workerd, with SQLite files on a volume |
100+
101+`deploy/self-host/configs.mjs` writes each Worker's self-hosted config from
102+its `wrangler.jsonc`, so the two never drift apart: it drops routes and
103+placement, binds the stand-ins above, points buckets at S3, and replaces
104+g1t.sh's addresses with yours. What runs and what is off comes from each
105+unit's `self_host` field in `deploy/stack.jsonc`.
106+
107+<Aside title="Adding a dependency on a Cloudflare product">
108+ Put it behind a port first, and give the self-hosted build an adapter
109+ or a stand-in in the same pull request. [How a self-hosted g1t
110+ runs](/guides/self-hosting-architecture/) is the self-hosted side of
111+ this table.
112+</Aside>
+92−0
1+---
2+title: Deployments, packages and telemetry
3+description: How apps are built and served on g1t.page, how the package registries are routed and stored, and where a project's logs, traces and metrics go.
4+---
5+
6+import Diagram from '../../../components/Diagram.astro';
7+
8+## Deployments
9+
10+`deployments` (`services/deployments`, TypeScript, D1 `g1t-deployments`)
11+builds apps and puts them live. `pages` (`services/pages`) serves them.
12+Neither runs your app as a server that waits: each app is a Worker, which
13+runs only while it answers a request, so an app nobody visits costs
14+nothing.
15+
16+<Diagram name="deployment-path" />
17+
18+### Building
19+
20+Every push to the default branch makes a production deployment, and every
21+open pull request gets a preview. `deployments` reads pushes from its
22+event queue and asks `runner` for a sandbox in `deploy` mode
23+([Actions, runners and sandboxes](/architecture/compute/#other-work-in-sandboxes)).
24+The sandbox works out what the app is, in this order:
25+
26+1. a Workers project, from `wrangler deploy --dry-run`;
27+2. a framework it knows, such as Vite, Astro, Next.js or SvelteKit;
28+3. a container, which g1t.sh does not run yet (`CONTAINERS` is `off`);
29+4. a static site.
30+
31+The sandbox is given a key that can upload the app's static files and
32+nothing else. `deployments` uploads the built Worker itself to the Workers
33+for Platforms dispatch namespace `g1t-deployments`. A site with only static
34+files gets a Worker that serves just its assets. A workflow can build
35+instead and hand over the result, with
36+`POST /repos/{owner}/{repo}/deployments/builds` or the `g1t/deploy` action.
37+
38+### Serving
39+
40+| | Hostname |
41+| --- | --- |
42+| Production | `<project>-<workspace>.g1t.page` |
43+| A preview | `<project>-git-<branch>-<workspace>.g1t.page` |
44+| Your domain | A CNAME to `domains.g1t.page`, a custom hostname on g1t.page's zone |
45+
46+`pages` answers every one of those hostnames. It reads the app's name from
47+the hostname, or looks a custom domain up in the `g1t-domains` KV, which
48+`deployments` writes, and hands the request to that app's Worker with
49+`env.APPS.get(app)`. Each request to an app may use 50 ms of CPU and 50
50+subrequests. A preview comes down when its pull request closes, or after it
51+has been idle for a number of days you choose (7 by default).
52+
53+## Packages
54+
55+`packages` (`services/packages`, Rust, D1 `g1t-packages`) is every package
56+registry g1t has. The site recognises a registry's path and passes the
57+request on whole, as it does for git:
58+
59+| Registry | Path on `g1t.sh` |
60+| --- | --- |
61+| Container images (OCI Distribution 1.1) | `/v2/`, with tokens at `/v2/token` |
62+| npm | `/-/npm/` |
63+| Cargo, sparse index | `/-/cargo/<workspace>/` |
64+| Composer | `/-/composer/` |
65+| Maven | `/-/maven/` |
66+| NuGet | `/-/nuget/<workspace>/v3/index.json` |
67+| RubyGems | `/-/rubygems/` |
68+
69+Files are stored in the `g1t-packages` bucket by their digest, so a layer
70+or a tarball pushed twice is kept once. The bucket sits behind the
71+`BlobStore` port, so a self-hosted installation keeps them in any S3
72+store. Large downloads are answered with a signed URL to the bucket.
73+[Packages](/guides/packages/) has each registry's setup.
74+
75+## Telemetry
76+
77+`telemetry` (`services/telemetry`, TypeScript) keeps each project's logs,
78+traces, metrics and errors in a Durable Object of its own,
79+`TelemetryStore`, in that object's SQLite storage. A project's data never
80+shares a database with another's.
81+
82+It has three ways in:
83+
84+| Door | From |
85+| --- | --- |
86+| `otel.g1t.sh/v1/traces`, `/v1/logs`, `/v1/metrics` | Your software, as OpenTelemetry over HTTP, JSON or protobuf, with a `g1to_…` token |
87+| Tail events | Apps on `g1t.page`: `telemetry` is `pages`' tail consumer, and apps name it as their Tail Worker |
88+| `/otlp/v1/*` on the API | The same as `otel.g1t.sh`, passed on as it is; the only door on a self-hosted installation |
89+
90+A store keeps up to 4 GiB and sheds its oldest day past that. You choose
91+how long raw data is kept, from 1 to 30 days; rollups are kept 90 days.
92+[Observability](/guides/observability/) shows what you can do with it.
+158−0
1+---
2+title: Events
3+description: How a state change in one service reaches every other that cares, through one log and a queue per subscriber, and how webhooks, live pages and followers read it.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+Every state change in g1t is published as an event: a push, an issue
10+opened, a pull request merged, a workflow finished, a workspace renamed.
11+Services do not call each other to announce what happened. They publish,
12+and the `events` service (`services/events`, Rust) delivers.
13+
14+## The envelope
15+
16+Every event has the same shape, which follows CloudEvents
17+(`crates/contracts/src/events.rs`, `packages/contracts/src/events.ts`):
18+
19+| Field | What it is |
20+| --- | --- |
21+| `id` | `evt_` and a UUIDv7: sortable by the time it was made |
22+| `type` | Such as `git.push`, `pull.merged`, `workflow.completed` |
23+| `source` | The service that published it |
24+| `time` | When, in RFC 3339 |
25+| `repoId` | The repository it is about, or none |
26+| `actor` | The account that caused it, or none |
27+| `data` | The type's own fields, in its contract |
28+
29+Coordination has its own events, published by `work`: sessions starting
30+and ending, claims, hand-offs, dependencies, overlaps, and the merge
31+queue's verdicts (`integration.failed`, `integration.passed`,
32+`decision.recorded`). They reach webhooks and followers like any other
33+([Work and the merge queue](/architecture/work/#coordination)).
34+
35+Two fields in `data` matter everywhere:
36+
37+- `causedByJob` marks an event caused by a workflow's own token, so it
38+ does not start more workflows.
39+- `truncated: true` marks an event that was too big for a queue message
40+ (96 KB) and had its long strings cut, step by step, until it fit.
41+
42+## From publish to every subscriber
43+
44+<Diagram name="event-fanout" />
45+
46+1. A service calls `events`' `publish` with a batch of events. `events`
47+ gives each its id and time and puts them on the `g1t-events` queue, then
48+ returns. Publishing never waits on a subscriber.
49+2. `events` reads its own queue, 100 messages at a time, and for each
50+ batch:
51+ - writes the events to the log with `INSERT OR IGNORE`, so a batch that
52+ comes round twice is written once;
53+ - sends each subscriber queue the events routed to it;
54+ - tells `notify`, so open pages of that repository refresh, and puts
55+ items in people's inboxes, at most once per person per event.
56+3. If a send to a subscriber fails, `events` notes which queues already
57+ have the batch (`fanout_sent`, kept a day) and throws. The retry skips
58+ those queues.
59+4. After three tries, a batch goes to `g1t-events-dlq`.
60+
61+Every subscriber's consumer is set up the same way: three tries, then the
62+dead-letter queue.
63+
64+### Subscriber queues
65+
66+Each service that reacts to events has its own queue,
67+`g1t-events-<service>`. It gets only the types it needs, as `ROUTES` in
68+`crates/contracts/src/subscribers.rs` says, plus every lifecycle event
69+(renames, transfers and deletions of repositories, workspaces and users),
70+which every subscriber gets.
71+
72+| Queue | Read by | Gets |
73+| --- | --- | --- |
74+| `g1t-events-work` | `work` | `git.push`, `branch.renamed`, `repo.created`, `repo.archived`, `repo.default_branch_changed`, `pull.merged`, `comment.created` |
75+| `g1t-events-runner` | `runner` | Pull request changes, `checks.completed`, `queue.changed`, `comment.created`, issue changes, `repo.archived` |
76+| `g1t-events-actions` | `actions` | `git.push`, `git.deleted`, `pull.*`, `issue.*`, `comment.*`, `release.*`, deployments, `workflow.completed`, `repo.archived` |
77+| `g1t-events-webhooks` | `webhooks` | Everything |
78+| `g1t-events-security` | `security` | `git.push`, pull request changes, `comment.created`, `repo.created`, `repo.visibility_changed` |
79+| `g1t-events-search` | `search` | `git.push`, `repo.*`, `issue.*`, `pull.*`, `user.*`, `workspace.*` |
80+| `g1t-events-deployments` | `deployments` | Pushes, the pull request lifecycle, workspace deletion and restore |
81+| `g1t-events-projects` | `projects` | Repositories, packages and deployments |
82+| `g1t-events-repos` | `repos` | `pull.merged`, `pull.closed`, `pull.reopened`, deletions and restores |
83+| `g1t-events-packages` | `packages` | `git.push`, `repo.visibility_changed`, workspace and team changes |
84+| `g1t-events-integrations` | `integrations` | `issue.closed`, `pull.opened` |
85+| `g1t-events-billing` | `billing` | Lifecycle events only |
86+| `g1t-events-docs` | `artifacts` | `git.push`, `pull.merged` |
87+
88+A subscriber should treat every event as possibly delivered twice, and
89+make its handling idempotent: most do it with a unique key on the event's
90+`id`.
91+
92+## The log
93+
94+`g1t-events`' D1 database keeps the log. Each event gets a `seq` as it is
95+written, one higher than the last; the log has one writer, the consumer
96+above. Ids are made by publishers in many places, so they can arrive out
97+of order. Reading "everything after this event" therefore follows `seq`,
98+not `id`.
99+
100+You can read a repository's log with the API's `list_events`, newest
101+first, 50 a page and up to 200, filtered by type. Status and check events
102+are left out unless you ask for their type. The same database keeps the
103+audit log, for 400 days.
104+
105+### Following
106+
107+Pass `after`, an event id, and `list_events` returns what the log wrote
108+after it, oldest first. Add `wait`, up to 25 seconds, and the call waits
109+for something to arrive:
110+
111+<Diagram name="follow-events" />
112+
113+The API asks the log about once a second while it waits (`apps/api/src/follow.rs`).
114+`g1t work events --follow` runs this in a loop, starting each call from the
115+last event it saw, so it misses nothing between calls. Coding agents get
116+the same over MCP, with the `list_events` tool.
117+
118+## Webhooks
119+
120+`webhooks` reads every event and delivers those each subscription asks
121+for:
122+
123+| | |
124+| --- | --- |
125+| **Request** | `POST` with JSON, to public addresses only, with a 10 second timeout. Anything but a 2xx is a failure. |
126+| **Headers** | `x-g1t-event`, `x-g1t-delivery`, `x-g1t-hook`, and `x-g1t-signature-256: sha256=<HMAC-SHA256 of the body>` |
127+| **Body** | `{ id, type, time, workspace, repository: { id, full_name }, actor: { id, username }, data }`, in snake_case |
128+| **Retries** | After 1 minute, 5 minutes, 30 minutes, 2 hours and 5 hours: six attempts in about seven and a half hours |
129+| **At most once** | A delivery is unique on the subscription and the event's `id` |
130+| **Kept** | 14 days |
131+
132+Each subscription's secret is sealed in D1. [Webhooks](/guides/webhooks/)
133+shows how to check a signature.
134+
135+## Live pages
136+
137+`notify` (TypeScript) keeps three kinds of Durable Object:
138+
139+| Object | One per | Holds |
140+| --- | --- | --- |
141+| `Feed` | Person | Their open tabs' sockets, notifications, counts and push subscriptions |
142+| `Room` | Workspace | Who is here |
143+| `Topic` | Repository | The pages open on it, told by `repo_changed` to refresh |
144+
145+Sockets use the WebSocket Hibernation API, so an idle tab costs nothing
146+while it waits. Browser push uses VAPID, and phones get Expo push; both are
147+sent to live 12 hours.
148+
149+`chat` and `artifacts` work the same way with rooms of their own: a
150+`ChannelRoom` per channel, and a `PageRoom` per doc that syncs edits as a
151+Yjs document and saves them to D1 a few seconds after a burst of typing.
152+
153+<Aside title="Scheduled jobs">
154+ Some work runs on a clock rather than on events: webhook retries every
155+ minute, Actions schedules every minute, fork retirement hourly, backups
156+ nightly, the event log's purge daily. Each service's `wrangler.jsonc`
157+ lists its `crons`.
158+</Aside>
+194−0
1+---
2+title: Git
3+description: Where g1t keeps repositories, what happens on a push and a clone, the caches in front of the git store, pull request forks, merges, and backups.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+Every repository on g1t.sh is an ordinary git repository in Cloudflare
10+Artifacts, a git store. One service, `repos` (`services/repos`, Rust),
11+holds the only binding to it. Everything else that reads code, the site
12+and the API included, asks `repos`.
13+
14+## Where repositories live
15+
16+`repos` talks to the store through a port: the `GitStore` and `GitRepo`
17+traits in `services/repos/src/store.rs`, with `ArtifactsStore` as the
18+adapter for Artifacts. Self-hosted installations bind a different adapter
19+in its place ([Data and storage](/architecture/data/#ports-and-adapters)).
20+Every call to the store goes through one function that meters it, retries
21+reads, and trips a circuit breaker when the store struggles
22+(`resilience.rs`).
23+
24+### Namespaces and sharding
25+
26+An Artifacts namespace takes a fixed number of control requests per 10
27+seconds, and its jurisdiction, US or EU, is set when it is made. So g1t
28+can spread repositories across several namespaces, each its own binding
29+(`services/repos/src/shards.rs`):
30+
31+| Setting | What it decides |
32+| --- | --- |
33+| `GITSTORE_NAMESPACES` | The namespaces and their bindings, such as `GITSTORE_1` and `GITSTORE_EU` |
34+| `GITSTORE_NEW_REPOS` | Where new repositories go |
35+| `GITSTORE_EU_NAMESPACE` | Where an EU workspace's repositories go |
36+
37+A repository's row in D1 records its store key, `namespace/owner--name`.
38+A key with no namespace is in the first, `g1t`. Moving a repository
39+between namespaces is an operator task, in `moves.rs`.
40+
41+### What D1 holds
42+
43+`g1t-repos` keeps what git does not: each repository's row (store key,
44+size, default branch, visibility, `refs_version`), renames and their
45+redirects, releases, stars, stats, backups, moves, the commits each push
46+brought for the contribution calendar, and the meter of every git
47+operation.
48+
49+## A push
50+
51+<Diagram name="push-path" />
52+
53+The site recognises a git path and hands the whole request to `repos`
54+(steps 1 and 2), after the git rate limits. From there, `repos` does the
55+work in `answer_git` (`services/repos/src/lib.rs`):
56+
57+1. **Who is pushing.** HTTP Basic credentials go to `identity`
58+ (steps 3 and 4). A secret with the token prefix is an access token, and
59+ the username is ignored. Anything else is a password, which `identity`
60+ refuses for anyone with two-factor authentication on, so they push
61+ with a token. The repository's row is read at the same time.
62+2. **Which repository.** Renames redirect, transfers resolve, and a pull
63+ request's fork that was retired is made again.
64+3. **Whether they may.** The access check, then the workspace's limits.
65+4. **The rules.** `repos` asks `work` for the rulesets that apply
66+ (step 5).
67+5. **The checks** (step 6). The pack is read once, and four checks run on
68+ it side by side; their verdicts are taken in this order:
69+
70+ | Check | Where | Refuses |
71+ | --- | --- | --- |
72+ | Workflow gate | `workflow_gate.rs` | A token without the workflow permission changing `.g1t/workflows` |
73+ | Rules | `rules.rs`, judged by `crates/rules` | What a ruleset forbids, with a `remote:` line naming the rule |
74+ | Sizes | `pack_limits.rs` | A file over 32 MB, a repository over 950 MB, a push over 100 MB |
75+ | Push protection | `secret_scan.rs`, using `crates/scan` | Secrets, private email addresses, and the workspace's own patterns |
76+
77+6. **The store** (steps 7 and 8). The pack goes to Artifacts with a
78+ short-lived store credential.
79+7. **The answer** (steps 9 and 10). Once the store reports, `refs_version`
80+ goes up, which invalidates every cached ref listing, and only then
81+ does git get its answer.
82+
83+After the answer, in `waitUntil`, `repos` checks with the store's log that
84+each ref really moved, adds the repository's new size, counts commits, and
85+publishes `git.push` for each branch (step 11 and 12). Everything that
86+reacts to a push, from workflows to search to deployments, starts from
87+that event ([Events](/architecture/events/)).
88+
89+<Aside title="Large pushes">
90+ A push up to 24 MiB is scanned before it is stored. A larger one is
91+ streamed straight to the store, its `git.push` events say
92+ `unscanned: true`, and `security` scans it after it lands.
93+</Aside>
94+
95+A few details that keep pushes fast and correct:
96+
97+- **`no-thin`.** g1t advertises that it wants whole packs, so the checks
98+ never have to read objects from the store. A client that sends a thin
99+ pack anyway still works: up to 200 bases are fetched.
100+- **Hidden refs.** `refs/pull/*` belong to g1t. A push to one is refused
101+ for that ref alone.
102+- **Announcements are capped.** A push with more than three tags announces
103+ none of them, and one with more than 1,000 branches announces only the
104+ default branch.
105+
106+### Timing
107+
108+Every git answer carries `Server-Timing`, with a step for each stage:
109+`repo`, `access`, `mint` (a store credential), `store`, `checks` (with
110+`read`, `rules`, `scan` and `gate`), `refs`, `total` and more. Run
111+`GIT_CURL_VERBOSE=1 git push` to see where a slow push spent its time.
112+
113+## A clone or fetch
114+
115+<Diagram name="clone-path" />
116+
117+A fetch is answered from the first place that can:
118+
119+| Order | From | Holds | Kept |
120+| --- | --- | --- | --- |
121+| 1 | R2, `g1t-git-packs` | Whole packs for fresh clones: a request with wants and no haves, shallow ones included | 7 days, or until a ref moves |
122+| 2 | The colo's cache | Ref listings, and trees and blobs up to 1 MiB, which never change | Refs 60 s; objects forever |
123+| 3 | KV, `g1t-repos-git-cache` | Ref listings and store credentials, shared by every isolate | 60 s |
124+| 4 | Artifacts | Everything | |
125+
126+Every cached ref listing is keyed by the repository's `refs_version`, so a
127+push makes the old one unreachable at once. A change can take up to 5
128+seconds to reach every fetch, while a repository's row is reused.
129+
130+**Packs are cached as they stream.** On a miss, the store's answer goes
131+to git and to R2 at the same time, never held whole: in one `put` when
132+small, in 5 MiB parts when large, completed only after a check. A pack
133+over 200 MiB is not cached, at most two fills run at once, and a
134+repository fills at most 30 times a minute. A hit never touches the store.
135+
136+**What is in KV is sealed.** Each value is encrypted with AES-256-GCM under
137+the `REPOS_KEY` secret, with its own key as associated data, so a value
138+copied to another key will not open (`services/repos/src/shared.rs`).
139+
140+When the store is busy, git gets 429 or 503 with `Retry-After`. Pushes are
141+never retried on your behalf.
142+
143+## Pull request forks
144+
145+A pull request made for an issue gets a fork of its own
146+([Forks and branches](/concepts/forks/)). `fork_for_pull` makes it in the
147+same namespace as its source, as `pulls--<pull request id>`, with
148+Artifacts' own fork of the default branch, and it is served at
149+`g1t.sh/pulls/<id>.git`. Anyone who can read the source can read the fork.
150+
151+A day after the pull request merges or closes, an hourly job retires the
152+fork: its head is kept in the source as `refs/pull/<id>/head`, and its data
153+is removed. Reads of a retired fork are answered from the source, and any
154+write makes the fork again first.
155+
156+## Merging
157+
158+The Artifacts binding reads but does not write, so `repos` lands a merge
159+over git itself (`services/repos/src/land.rs`). It asks the fork for a pack
160+with exactly the objects the target lacks, and streams it into a push to
161+the target, without parsing it or holding it whole.
162+
163+`repos` also builds the test merge for every pull request,
164+`refs/pull/<n>/merge`, without a sandbox, using a line-level merge on
165+files up to 1 MiB (`test_merge.rs`). When a pull request is behind and the
166+two sides changed different files, **Catch up with main** is done the same
167+way (`catch_up.rs`). Testing several pull requests together is the merge
168+queue's job ([Work and the merge queue](/architecture/work/#the-merge-queue)).
169+
170+## Backups
171+
172+At 02:53 UTC, `repos` queues up to 200 repositories whose refs moved since
173+their last backup, oldest first. The runner cuts them: each job gets a
174+sandbox that clones with a read-only store credential valid for a few
175+minutes, writes a `git bundle`, and uploads it in 32 MiB parts to the
176+`g1t-backups` bucket. Every thirtieth backup is full and the rest are
177+incremental.
178+
179+| Path | Holds |
180+| --- | --- |
181+| `backups/<repo id>/manifest.json` | The repository's bundles, in order |
182+| `backups/<repo id>/<time>-full.bundle` | Every ref and object |
183+| `backups/<repo id>/<time>-incr.bundle` | What changed since the bundle before |
184+
185+`scripts/ops/backup-restore-drill.mjs` restores from them, and
186+`scripts/ops/restore-to-gitstore.mjs` can rebuild repositories into a
187+self-hosted git store, which g1t.sh can then read from if Artifacts is
188+unavailable (`fallback.rs`).
189+
190+## SSH
191+
192+`crates/sshd` is an SSH front end for git that authenticates your key with
193+`identity` and bridges to the store. It is not deployed: git on g1t is
194+HTTPS only for now. You can already add deploy keys.
+99−0
1+---
2+title: Identity and access
3+description: How every kind of credential becomes one viewer, what the viewer carries, and how each service decides access with the same compiled-in rules instead of asking identity.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+`identity` (`services/identity`, Rust, D1 `g1t`) is the only service that
10+knows about credentials. It holds accounts, sessions, workspaces and their
11+members, teams, repository roles and invitations, access tokens, OAuth,
12+device sign-in and two-factor authentication. Every other service learns
13+who is asking from the viewer it is handed.
14+
15+<Diagram name="identity-flow" />
16+
17+## Credentials
18+
19+| Credential | Looks like | Used by | Resolved with |
20+| --- | --- | --- | --- |
21+| Session cookie | `g1t_session`, 64 hex characters, 30 days | The site | `user_for_session` |
22+| Access token | `g1t_…`, personal or a workspace's, with scopes | The API, MCP, git, the CLI | `user_for_access_token` |
23+| OAuth access token | Issued by `POST /oauth/token`, refreshed with `g1r_…` | MCP clients, the phone app, apps you authorise | `user_for_access_token` |
24+| Job token | `G1T_TOKEN` in a workflow job | Actions, for that job alone | `user_for_access_token` |
25+| Run credential | Made per sandbox run, up to 6 hours | Sandboxes | `user_for_access_token` |
26+| git password | Your account's password | git, only without two-factor authentication | `user_for_git_credentials` |
27+
28+The CLI signs in with the device flow of RFC 8628 (`/device/code`, then
29+`/device/token`) and keeps its token in the system keychain. Session
30+cookies and access tokens are stored as their SHA-256, so neither can be
31+read back.
32+
33+## The viewer
34+
35+`identity` answers with a `User` (`crates/contracts/src/lib.rs`), and the
36+edge hands it to every service it calls, as `viewer` in the call's
37+arguments. It is `null` for someone signed out.
38+
39+| Field | Holds |
40+| --- | --- |
41+| `kind` | `User`, `Workspace`, `Agent` or `System`: what kind of principal this is |
42+| `workspaces` | Each membership: the role, the workspace's base permission, organisation roles such as billing or security manager, and privileges |
43+| `grants` | Roles on repositories, directly or through teams |
44+| `acting` | For a job or run token: the narrower scope it acts within |
45+
46+A service never asks `identity` whether someone may do something. Each
47+calls `can` in `g1t_contracts::access`, the same rules compiled into every
48+Rust service, with `packages/contracts/src/access.ts` as the TypeScript
49+mirror, which a test keeps identical. That takes `identity` out of every
50+request after the first call, and means access is decided by the service
51+that owns the data, next to it.
52+
53+<Aside title="Why the viewer can be trusted">
54+ The viewer is passed as plain JSON, not signed. That is safe because
55+ services can be reached only through service bindings: none is on
56+ `workers.dev`, and the only routes in the core are `telemetry`'s and
57+ `og`'s public endpoints, which take no viewer. A Worker can only be
58+ called by the Workers bound to it, and those are the edge Workers and
59+ other services, which resolved the viewer themselves.
60+</Aside>
61+
62+## Roles
63+
64+| Level | Roles |
65+| --- | --- |
66+| Workspace | Owner and member, plus organisation roles (billing manager, security manager) and member privileges |
67+| Repository | Read, Triage, Write, Maintain and Admin, given to a person, a team, or everyone in the workspace as its base permission |
68+| Outside collaborators | A repository role for someone who is not a member |
69+
70+The rules live in `crates/contracts/src/access.rs`; the grants live in
71+`identity`. [Access and roles](/guides/access-and-roles/) explains each
72+role from a person's side.
73+
74+## Scopes
75+
76+A token can do no more than its owner, and often less. Scopes are written
77+`resource:level`, where `admin` includes `write` and `write` includes
78+`read` (`crates/contracts/src/scopes.rs`). A token's reach is every
79+workspace its owner belongs to, one of them, or none. The API checks the
80+scopes before it calls any service, and the MCP server lists only the
81+tools they allow ([How a request is served](/architecture/requests/#scopes)).
82+
83+## Sign-in hardening
84+
85+| | |
86+| --- | --- |
87+| Two-factor | TOTP (RFC 6238), with ten recovery codes stored as SHA-256. A workspace can require it. |
88+| Sudo mode | Sensitive changes ask you to prove it is you again (`Identity::proof`) |
89+| Lockouts | Wrong passwords are throttled per account and per address |
90+| Unconfirmed email | Can sign in, but cannot push |
91+| GitHub sign-in | A GitHub App's OAuth web flow with PKCE, keyed on your GitHub account's numeric id |
92+| Staff | `sudo.g1t.sh` checks the Cloudflare Access token against a staff list, and is the only caller of `identity`'s admin methods |
93+
94+## Events
95+
96+`identity` publishes `user.*`, `workspace.*`, `team.*` and `invite.*`
97+events. Lifecycle events, such as a workspace being
98+deleted or renamed, go to every subscriber, so each service cleans up or
99+renames its own data ([Events](/architecture/events/#subscriber-queues)).
+124−0
1+---
2+title: Architecture overview
3+description: How g1t is put together, from the four public Workers at the edge to the nineteen services behind them, the event bus that connects them, and the platform each one runs on.
4+---
5+
6+import { CardGrid } from '@astrojs/starlight/components';
7+import Aside from '../../../components/Aside.astro';
8+import Diagram from '../../../components/Diagram.astro';
9+import LinkCard from '../../../components/LinkCard.astro';
10+
11+g1t is a set of small programs, each with one job, that run on Cloudflare
12+Workers. A few face the internet. Almost all the rest can be reached only by the
13+Workers that are bound to them. Every service that keeps data keeps it in its own
14+database, and services learn what happened elsewhere from events instead of
15+calling each other to find out.
16+
17+This section is for developers: people building on g1t, contributing to it,
18+or running it themselves. It names the files, bindings and limits that
19+decide how g1t behaves, so you can read the code next to it.
20+
21+<Diagram name="platform-map" />
22+
23+## The layers
24+
25+| Layer | What is in it | Reached by |
26+| --- | --- | --- |
27+| **Clients** | Browsers, the [desktop](/guides/desktop/) and [phone](/guides/mobile/) apps, git, the [CLI](/guides/cli/), coding agents, [self-hosted runners](/guides/self-hosted-runners/), and visitors to apps deployed on g1t. | The internet |
28+| **Edge** | Four Workers with public routes: the site (`apps/web`), the API and MCP server (`apps/api`), the status page (`apps/status`) and the app dispatcher (`services/pages`). | Their hostnames |
29+| **Core** | Nineteen services, each a Worker with `workers_dev` off. Seventeen have no route at all; `telemetry` takes OTLP at `otel.g1t.sh` and `og` draws social cards at `og.g1t.sh`. | Service bindings from the Workers that name them |
30+| **Platform** | D1, Durable Objects, Queues, R2, KV, Artifacts for git, and Containers for sandboxes. | The one service that owns each, through its bindings |
31+
32+The edge does three things: it works out who is asking, enforces rate
33+limits, and turns a request into calls to the services that own the
34+answer. It keeps no state of its own. The services do the work and own the
35+data.
36+
37+## Where each hostname goes
38+
39+| Hostname | Worker | Serves |
40+| --- | --- | --- |
41+| `g1t.sh` | `g1t` (`apps/web`) | The site, git over HTTPS, and the package registries, which it passes whole to `repos` and `packages` |
42+| `g1tusercontent.com` | `g1t` (`apps/web`) | Raw files, avatars and attachments, on a domain that never sees the session cookie |
43+| `api.g1t.sh`, `mcp.g1t.sh` | `g1t-api` (`apps/api`) | The REST API, the MCP server, OAuth, the runner protocol, and OTLP |
44+| `*.g1t.page` and custom domains | `g1t-pages` (`services/pages`) | Apps deployed on g1t |
45+| `otel.g1t.sh` | `g1t-telemetry` | OpenTelemetry from your software |
46+| `status.g1t.sh` | `g1t-status` (`apps/status`) | The status page, with a database of its own so it answers when the rest does not |
47+| `docs.g1t.sh` | `g1t-docs` (`apps/docs`) | These pages |
48+
49+`sudo.g1t.sh`, g1t's staff console, sits behind Cloudflare Access and is
50+not part of what you can call.
51+
52+## Calls, events and ownership
53+
54+Three rules shape the code:
55+
56+- **A service owns its data.** Each of the sixteen D1 databases is bound
57+ by exactly one Worker. To read another service's data, you call that
58+ service. Three stores are deliberately shared, each with one writer:
59+ the avatars KV, the custom domains KV, and the Actions cache bucket
60+ ([Data and storage](/architecture/data/)).
61+- **Calls are for answers; events are for news.** When the site needs a pull
62+ request, it calls `work`. When a push lands, `repos` does not call the
63+ dozen services that care: it publishes `git.push`, and each of them gets
64+ its own copy on its own queue ([Events](/architecture/events/)).
65+- **The viewer travels with the call.** The edge resolves the person or
66+ token once, and passes the result to every service in the call's
67+ arguments. Each service decides access itself with the same rules,
68+ compiled in, rather than asking `identity` again
69+ ([Identity and access](/architecture/identity/)).
70+
71+A call is an HTTP request over a service binding: `POST /rpc/<method>`
72+with a JSON body. An expected failure, such as "not found" or "not
73+allowed", comes back as a value, `{ "ok": false, "error": { "code",
74+"message" } }`, with status 200. Only something unexpected, a thrown error
75+or a busy store, becomes a non-200 status
76+([Services and contracts](/architecture/services/)).
77+
78+## Rust and TypeScript
79+
80+The services that hold accounts, repositories, work, events, money and the
81+API are written in Rust, compiled to WebAssembly with `worker-build`. The
82+web apps and the services built around Durable Objects and live rooms are
83+TypeScript.
84+
85+| Rust | TypeScript |
86+| --- | --- |
87+| `identity`, `repos`, `work`, `events`, `actions`, `packages`, `security`, `search`, `billing`, `integrations`, `webhooks`, and `apps/api` | `runner`, `deployments`, `projects`, `chat`, `artifacts`, `notify`, `telemetry`, `og`, `pages`, and `apps/web`, `apps/sudo`, `apps/status` |
88+
89+Both sides share their types through contracts: `crates/contracts` in Rust
90+and `packages/contracts` in TypeScript. Where the two must agree exactly,
91+such as the access rules and the scopes, a Rust test reads the TypeScript
92+file and fails if they differ.
93+
94+## One codebase, two homes
95+
96+The same source runs on g1t.sh and in a [self-hosted
97+installation](/guides/self-hosting/). Each Cloudflare product a service
98+uses sits behind a port, and the self-hosted build binds an open
99+adapter in its place: bare repositories served by `git http-backend` for
100+Artifacts, any S3 store for R2, Mailpit for email. The service code does
101+not change. [Data and storage](/architecture/data/#ports-and-adapters)
102+shows each one.
103+
104+<Aside type="tip" title="Read the code next to this">
105+ Every unit g1t deploys is listed in `deploy/stack.jsonc`, with its
106+ language, its stage and what a self-hosted installation does with it.
107+ Each service's `wrangler.jsonc` lists its bindings: its database, the
108+ services it calls, and the queues it reads.
109+</Aside>
110+
111+## In this section
112+
113+<CardGrid>
114+ <LinkCard title="How a request is served" href="/architecture/requests/" icon="route" description="The site and the API, from the first byte to Server-Timing." />
115+ <LinkCard title="Services and contracts" href="/architecture/services/" icon="boxes" description="Every unit, what it owns, what it calls, and how calls work." />
116+ <LinkCard title="Git" href="/architecture/git/" icon="git-branch" description="Where repositories live, the push path, caches and forks." />
117+ <LinkCard title="Events" href="/architecture/events/" icon="radio-tower" description="The bus, the log, fan-out, webhooks and following live." />
118+ <LinkCard title="Work and the merge queue" href="/architecture/work/" icon="git-merge" description="Issues, pull requests, checks, merging and the queue." />
119+ <LinkCard title="Actions, runners and sandboxes" href="/architecture/compute/" icon="cpu" description="Workflow jobs, hosted and self-hosted, and what walls them in." />
120+ <LinkCard title="Deployments and telemetry" href="/architecture/deployments/" icon="rocket" description="Apps on g1t.page, packages, and observability." />
121+ <LinkCard title="Data and storage" href="/architecture/data/" icon="database" description="Who owns what, migrations, and the ports that make self-hosting work." />
122+ <LinkCard title="Identity and access" href="/architecture/identity/" icon="key-round" description="Credentials, the viewer, scopes, and where access is decided." />
123+ <LinkCard title="How g1t ships" href="/architecture/shipping/" icon="ship" description="The monorepo, builds, staged deploys, and keeping these pages true." />
124+</CardGrid>
+181−0
1+---
2+title: How a request is served
3+description: What happens between a request reaching g1t.sh or api.g1t.sh and its answer leaving, step by step, with the deadlines, caches, rate limits and timings along the way.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+Two Workers answer almost everything you send g1t: the site, `apps/web`
10+on `g1t.sh`, and the API, `apps/api` on `api.g1t.sh` and `mcp.g1t.sh`.
11+Neither keeps data. Each works out who is asking, then calls the services
12+that own the answer.
13+
14+## The site
15+
16+`apps/web/workers/app.ts` is the site's entry point. It sorts each request
17+before any page code runs:
18+
19+1. A request for `g1tusercontent.com` goes to `serveUsercontent`, which
20+ never reads or sets the session cookie.
21+2. `go get` lookups get their meta page.
22+3. **git over HTTPS** (paths ending in `info/refs`, `git-upload-pack` or
23+ `git-receive-pack`) passes the git rate limits, then goes to `repos`
24+ whole, through `env.REPOS.fetch`. The site never parses git
25+ ([Git](/architecture/git/)).
26+4. **Package registries** (`/v2/`, `/-/npm/`, `/-/cargo/` and the others)
27+ go to `packages` whole. A 404 is tried once more under the workspace's
28+ current name, so a renamed workspace's old URLs keep working.
29+5. Everything else passes the page rate limit and reaches React Router.
30+
31+### A page
32+
33+<Diagram name="page-request" />
34+
35+The root route's middleware, `viewerMiddleware` in
36+`apps/web/app/lib/session.server.ts`, resolves the viewer once (steps 2
37+and 3). The `g1t_session` cookie is 64 hex characters, `HttpOnly`,
38+`Secure` and `SameSite=Lax`, and lasts 30 days; `identity` stores only its
39+SHA-256. A request can send `Authorization: Bearer` with an access token
40+that has the **website** permission instead of the cookie. The same
41+middleware enforces the email confirmation, workspace and Code access
42+gates.
43+
44+Loaders then call their services together (steps 4 to 7). They use the
45+typed clients in `@g1t/contracts`, built in `app/lib/services.server.ts`,
46+and pass the viewer in each call's arguments. Nothing waits on a call it
47+does not need.
48+
49+### Deadlines
50+
51+Each binding is wrapped by `instrumented()` in `app/lib/perf.server.ts`,
52+which gives every call a deadline (`app/lib/deadline.ts`). A call that
53+outlives it is aborted and throws `DeadlineExceeded`, so one slow service
54+cannot hold a page forever.
55+
56+| Call | Deadline |
57+| --- | --- |
58+| A read | 15 s |
59+| A write, or a read named in `SLOW_READS` | 60 s |
60+| A long job, named in `LONG_CALLS` | 600 s |
61+
62+### Reading your own writes
63+
64+D1 has read replicas. The site uses D1's Sessions API with the services
65+that support it (`identity`, `repos`, `work`, `search`, `billing`,
66+`projects`, `deployments`) by sending an `x-d1-bookmark` header on each
67+call. The latest bookmark is kept in the `g1t_d1` cookie for 5 minutes,
68+and for 30 seconds after a write, reads go to the primary. You never see a
69+page older than a change you just made. The service side is
70+`crates/kit/src/d1.rs`; TypeScript services use
71+`packages/contracts/src/d1.ts`.
72+
73+### The public page cache
74+
75+A signed-out `GET` with no cookie and no `Authorization` header, for a
76+public page, can be answered from the colo's cache. It is fresh for 30
77+seconds, then served stale while it refreshes in the background, for up
78+to 5 minutes. Before a cached repository page is served, the site asks
79+`repos` whether the repository is still public, so making it private
80+takes effect at once.
81+
82+### Server-Timing
83+
84+Every page carries a `Server-Timing` header: the total, each loader, each
85+service's calls and time, and the database time each service reports
86+about itself (`svc`, `db` and `rpc`, from `Served` in
87+`crates/kit/src/d1.rs`). A cached page says
88+`cache;desc="hit, Ns old"`. Open your browser's network panel on any page
89+to see where its time went.
90+
91+<Aside title="Where it runs">
92+ Every Worker runs with placement off, in the data centre nearest the
93+ visitor. `scripts/perf/placement-probe.mjs` measures the alternative,
94+ and CONTRIBUTING.md's **Speed** section has the rules this design
95+ follows: start calls together, add no round trips, and never wake a
96+ Durable Object for a read D1 can answer.
97+</Aside>
98+
99+## The API and MCP
100+
101+`apps/api` serves both `api.g1t.sh` and `mcp.g1t.sh` from one Worker; a
102+host starting `mcp.` is the MCP server. It sorts each request in this
103+order (`apps/api/src/lib.rs`):
104+
105+1. OTLP at `/otlp/v1/*` goes to `telemetry` as it is.
106+2. Requests that bring their own credentials are handled before any g1t
107+ token is read: the Stripe webhook, inbound integration hooks at
108+ `/hooks/<id>`, deployment build jobs, the Actions toolkit and OIDC
109+ issuer, a job's artifacts, cache and brokered requests, and the runner
110+ protocol at `/runners/*`.
111+3. Rate limits.
112+4. The token.
113+5. OAuth.
114+6. MCP, when the host says so.
115+7. The REST routes.
116+
117+<Diagram name="api-request" />
118+
119+A token is sent as `Authorization: Bearer g1t_…` (or `token g1t_…`) and
120+resolved by `identity` (step 3). A wrong or expired one gets a 401 with a
121+`www-authenticate` header that points at the OAuth metadata, as RFC 9728
122+describes, so an MCP client can start signing in by itself.
123+
124+### Scopes
125+
126+Scopes are written `resource:level`, and `admin` includes `write`, which
127+includes `read`. They are defined in `crates/contracts/src/scopes.rs`, and
128+`packages/contracts/src/scopes.ts` mirrors them. The token's scopes decide
129+which operations it may call (step 5), and the MCP server lists only the
130+tools those scopes allow. A workflow job's token carries the job's own,
131+narrower scope.
132+
133+### One table for every operation
134+
135+`apps/api/src/operations.rs` lists every operation once. The REST routes,
136+the MCP tools and the OpenAPI document at
137+[`/openapi.json`](https://api.g1t.sh/openapi.json) are all generated from
138+it; an operation without a description does not compile. The docs'
139+[API reference](/reference/api/) is built from that document, and the
140+CLI's command table from the reference, so the four cannot disagree.
141+
142+### snake_case at the edge
143+
144+Inside g1t, contracts are camelCase. The API's `reply()` turns every
145+answer to snake_case on the way out (`g1t_kit::wire::snake_case`), leaving
146+user-named keys such as inputs, env, secrets and headers as they were
147+written. Errors keep one shape, `{"error": {"code", "message"}}`, with the
148+status that the failure's code maps to.
149+
150+### OAuth 2.1
151+
152+The API is the OAuth issuer. It supports authorization code with PKCE for
153+public clients only, with no client secrets, and dynamic registration at
154+`POST /oauth/register`, whose client id encodes the client's name and
155+redirect addresses. The person approves on the site, at
156+`g1t.sh/oauth/authorize`; the token endpoint, `POST /oauth/token`, asks
157+`identity` to exchange and refresh. Refresh tokens rotate.
158+
159+### MCP
160+
161+The MCP server speaks streamable HTTP with no session state: each `POST`
162+gets one JSON-RPC answer. It supports protocol versions 2025-06-18,
163+2025-03-26 and 2024-11-05. Its tools are resources with an `action`, as
164+[MCP tools](/reference/mcp/) lists.
165+
166+## Rate limits
167+
168+Every limit is a Workers rate-limit binding, and
169+`packages/contracts/src/rate-limits.ts` is the table of record: a test
170+checks every `wrangler.jsonc` against it.
171+
172+| Where | Limit | Counted by |
173+| --- | --- | --- |
174+| The API, without a token | 60 a minute | The client's address |
175+| The API, with a token | 1,000 a minute | The token, hashed |
176+| REST and MCP | Counted separately | |
177+| Sandbox and runner reports | Not counted | |
178+
179+When the limiter itself cannot answer, the request goes through: the
180+limits fail open. The [rate limits page](/reference/rate-limits/) lists
181+every limit you can hit.
+148−0
1+---
2+title: Services and contracts
3+description: Every unit g1t deploys, what each one does, what it owns and what it calls, and how a call between two services works, down to the wire.
4+---
5+
6+g1t deploys twenty-five units, listed once in `deploy/stack.jsonc`. Each is
7+a Worker with its own `wrangler.jsonc`, which names its database, its
8+queues and the services it calls. This page puts all of them in one
9+place.
10+
11+## Every unit
12+
13+**Calls** lists the services each unit binds to, which are the only ones
14+it can reach. A test, `npm run test:deploy`, reads every `wrangler.jsonc`
15+and fails if this table and the bindings disagree.
16+
17+| Unit | Language | Stage | What it does | Owns | Calls |
18+| --- | --- | --- | --- | --- | --- |
19+| `apps/web` | TypeScript | front | The site on `g1t.sh`: pages, git over HTTPS and the package registries passed to their services, and `g1tusercontent.com` | The downloads bucket | `actions`, `artifacts`, `billing`, `chat`, `deployments`, `events`, `identity`, `integrations`, `notify`, `og`, `packages`, `projects`, `repos`, `search`, `security`, `telemetry`, `webhooks`, `work` |
20+| `apps/api` | Rust | edge | REST, MCP, OAuth, the runner protocol, OTLP ingest, Actions toolkit and OIDC | Nothing; writes Actions cache and artifact files | `actions`, `artifacts`, `billing`, `deployments`, `events`, `identity`, `integrations`, `packages`, `projects`, `repos`, `runner`, `search`, `security`, `telemetry`, `webhooks`, `work` |
21+| `apps/status` | TypeScript | edge | `status.g1t.sh`: probes every minute and keeps incidents | D1 `g1t-status` | `billing`, `repos` |
22+| `apps/sudo` | TypeScript | front | g1t.sh's staff console, behind Cloudflare Access | Nothing | `actions`, `billing`, `events`, `identity`, `projects`, `status`, `telemetry` |
23+| `apps/docs` | Astro | front | `docs.g1t.sh`, these pages | Nothing | none |
24+| `services/identity` | Rust | core | Accounts, sessions, workspaces, members, teams, repository roles, tokens, OAuth, device sign-in, 2FA | D1 `g1t`, the avatars KV | `billing`, `events`, `projects`, `repos` |
25+| `services/repos` | Rust | core | Repositories and git: smart HTTP, pre-receive checks, caches, forks, landing merges, releases, backups | D1 `g1t-repos`, the git store, the git cache KV, the packs and backups buckets | `billing`, `events`, `identity`, `security`, `work` |
26+| `services/work` | Rust | core | Issues, pull requests, comments, reviews, labels, milestones, rulesets, checks, statuses, CODEOWNERS, the merge queue | D1 `g1t-work` | `actions`, `events`, `identity`, `repos` |
27+| `services/events` | Rust | core | The event bus and log, the audit log, the inbox | D1 `g1t-events`, the `g1t-events` queue and its subscriber queues | `billing`, `identity`, `notify`, `repos`, `work` |
28+| `services/actions` | Rust | core | Workflows: reading them, triggering runs, planning jobs, secrets, self-hosted runners, logs | D1 `g1t-actions`; lists and expires the Actions cache bucket | `billing`, `deployments`, `events`, `identity`, `notify`, `projects`, `repos`, `runner`, `work` |
29+| `services/runner` | TypeScript | core | Sandboxes on Cloudflare Containers, one per job, with their egress | The Container classes | `actions`, `billing`, `deployments`, `events`, `identity`, `repos`, `work` |
30+| `services/deployments` | TypeScript | core | Builds and serves apps on `g1t.page`, previews, custom domains | D1 `g1t-deployments`, the domains KV, the dispatch namespace | `actions`, `billing`, `events`, `identity`, `og`, `projects`, `repos`, `runner`, `telemetry`, `work` |
31+| `services/pages` | TypeScript | edge | The dispatcher for `*.g1t.page` and custom domains | Nothing; reads the domains KV | none |
32+| `services/packages` | Rust | core | npm, Cargo, Composer, Maven, NuGet, RubyGems, Go and container images | D1 `g1t-packages`, the packages bucket | `billing`, `events`, `identity`, `repos` |
33+| `services/security` | Rust | core | Secret scanning, dependency alerts and updates, code scanning, the supply chain | D1 `g1t-security` | `actions`, `billing`, `events`, `identity`, `repos`, `runner`, `work` |
34+| `services/search` | Rust | core | Site search over repositories, issues, pull requests, people and code | D1 `g1t-search`, its jobs queue | `billing`, `identity`, `repos`, `work` |
35+| `services/projects` | TypeScript | core | Projects and what they depend on | D1 `g1t-projects` | `deployments`, `identity`, `og`, `packages`, `repos` |
36+| `services/chat` | TypeScript | core | Channels, direct messages and threads, live | D1 `g1t-chat`, a Durable Object per channel | `identity`, `notify` |
37+| `services/artifacts` | TypeScript | core | Docs written together, live, with their spaces, sharing and search | D1 `g1t-docs`, its files bucket, two Vectorize indexes, a Durable Object per open doc | `billing`, `events`, `identity`, `notify`, `repos`, `work` |
38+| `services/notify` | TypeScript | core | Notifications, live pages, presence, browser and phone push | A Durable Object per person, per workspace and per repository | `identity` |
39+| `services/telemetry` | TypeScript | core | Logs, traces, metrics and errors for projects, at `otel.g1t.sh` | A Durable Object per project | `billing` |
40+| `services/billing` | Rust | core | Usage, holds and settlements, limits, Stripe | D1 `g1t-billing` | `identity`, `packages`, `repos` |
41+| `services/integrations` | Rust | core | Model providers, alert sources, issue trackers, the GitHub App, Ask g1t | D1 `g1t-integrations` | `identity`, `repos`, `work` |
42+| `services/webhooks` | Rust | core | Webhook subscriptions and signed deliveries with retries | D1 `g1t-webhooks` | `identity`, `repos` |
43+| `services/og` | TypeScript | core | Social cards and page screenshots, at `og.g1t.sh` | The screenshots bucket | `billing`, `identity`, `projects`, `repos`, `work` |
44+
45+Calls go both ways in places, `repos` and `work` for one. That is allowed:
46+bindings only have to respect the deploy stages, and every service is in
47+the same one ([How g1t ships](/architecture/shipping/)).
48+
49+## A call, on the wire
50+
51+A service binding is a direct connection from one Worker to another in the
52+same data centre, with no trip over the public internet. g1t uses it as
53+plain HTTP:
54+
55+```http
56+POST https://work/rpc/get_pull
57+content-type: application/json
58+
59+{"repo": {"namespace": "acme", "name": "app"}, "number": 12, "viewer": {"id": "usr_…", "workspaces": […], …}}
60+```
61+
62+The hostname is ignored; the method is the path. The arguments are that
63+method's struct in the contracts, here `ViewArgs` in
64+`crates/contracts/src/work.rs`. `viewer` is the person or token asking, or
65+`null` for someone signed out. The answer is always JSON:
66+
67+```json
68+{ "ok": true, "value": { … } }
69+```
70+
71+or, for anything the caller should handle, still with status 200:
72+
73+```json
74+{ "ok": false, "error": { "code": "not_found", "message": "No pull request #12." } }
75+```
76+
77+| Status | Means | The caller gets |
78+| --- | --- | --- |
79+| 200 with `ok: true` | It worked | The value |
80+| 200 with `ok: false` | An expected failure, with a code from `FailureCode` | A value to handle, such as `Result` in TypeScript or `Outcome` in Rust |
81+| 404 | No such method | An error |
82+| 503 | A store is busy, with `Retry-After` | An error |
83+| 500 | Something threw | An error |
84+
85+On the Rust side, `g1t_kit::call(&binding, "method", &args)` sends a call
86+and `rpc_method`, `args` and `reply` serve one (`crates/kit/src/lib.rs`).
87+On the TypeScript side, the clients in `packages/contracts/src/clients.ts`
88+do both. Two Workers expose Worker RPC entrypoints instead: `og`'s
89+`Screenshots` and `status`'s `StatusAdmin`.
90+
91+There is no secret between services. A service can be called only
92+through a binding, because none of them is published on `workers.dev`
93+and only `og` and `telemetry` have routes, for their own public endpoints.
94+That makes the binding list in each `wrangler.jsonc` the complete list of
95+who can call it.
96+
97+## Contracts
98+
99+Services share types, never code. Each one depends on the contracts, and
100+none on another service's crate or package.
101+
102+| Package | Holds |
103+| --- | --- |
104+| `crates/contracts` (`g1t-contracts`) | A module per service with its data and its methods' arguments; `Outcome` and `Failure`; the event envelope and every event's data; `ROUTES`, which says which events each subscriber queue gets; `access`, the access rules every service compiles in; `scopes`; ids |
105+| `packages/contracts` (`@g1t/contracts`) | The same for TypeScript: each service's API type (`IdentityApi`, `ReposApi`, …), typed clients, `Result` and `FailureCode`, events, scopes, rate limits, the D1 session helper. TypeScript source, with no build step. |
106+| `crates/kit` (`g1t-kit`) | Plumbing for Rust Workers: the call protocol, D1 sessions and `Server-Timing` (`d1.rs`), rate limits that fail open, billing's platform pause, the snake_case converter, and handlers every service shares for renames, transfers and deletions |
107+
108+Where the two languages must agree exactly, a Rust test reads the
109+TypeScript file and compares: the access table (`access.rs`), scopes, and
110+the shapes that have shared fixtures in JSON. Keep both in the same pull
111+request.
112+
113+### Ids
114+
115+Ids are TypeIDs: a prefix that says what the thing is, then a UUIDv7 in
116+Crockford base32, such as `evt_01jb…` for an event or `usr_…` for a person.
117+They sort by the time they were made, and are monotonic within one process
118+and millisecond (`crates/contracts/src/ids.rs`).
119+
120+## Shared crates
121+
122+| Crate | What it is |
123+| --- | --- |
124+| `crates/rules` | The rulesets engine, with no I/O: which rules apply to a ref, and whether a push or a merge passes them |
125+| `crates/scan` | Detectors for secrets, a git pack reader, lockfiles, OSV, the dependency graph, SPDX and SARIF |
126+| `crates/secrets` | `Sealer`, AES-256-GCM with the row's id as associated data, and HMAC signatures |
127+| `crates/blobstore` | The `BlobStore` port, with R2 and S3 adapters |
128+| `crates/actions` | Workflow syntax, expressions, filters, matrices and cron, with no I/O, used by both the `actions` service and the sandbox |
129+| `crates/runner` | `g1t-runner`, the binary every sandbox and self-hosted runner runs |
130+| `crates/g1t` | [The g1t CLI](/guides/cli/) |
131+| `crates/sshd` | An SSH front end for git, not deployed yet |
132+
133+## Adding a service
134+
135+1. Make it under `services/<name>`, with its `wrangler.jsonc`: its D1
136+ database if it keeps data, and `"workers_dev": false`.
137+2. Put its types in `crates/contracts` or `packages/contracts`, and its
138+ client in `packages/contracts/src/clients.ts` if the site calls it.
139+3. Add it to `deploy/stack.jsonc` with its kind, stage, secrets and
140+ `self_host`, then to the table above and to [How a self-hosted g1t
141+ runs](/guides/self-hosting-architecture/#each-part).
142+4. If it reacts to events, give it a `g1t-events-<name>` queue and a
143+ `SUBSCRIBER_*` binding on `events`, and add its event types to `ROUTES`
144+ ([Events](/architecture/events/#subscriber-queues)).
145+5. Run `npm run test:deploy`. It says what is missing.
146+
147+[Deploy g1t to Cloudflare](/guides/deploy-to-cloudflare/) covers the
148+manifest's fields and a unit's first deploy.
+106−0
1+---
2+title: How g1t ships
3+description: The monorepo's layout, how each kind of unit is built, how a push to main becomes a staged deploy on g1t's own Actions, the tests that guard it, and how this section stays true.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+g1t is built on g1t. Its source is at
10+[g1t.sh/flagon-io/g1t](https://g1t.sh/flagon-io/g1t), every change is a
11+pull request with required checks, and its own workflows deploy it.
12+
13+## The repository
14+
15+| Folder | Holds |
16+| --- | --- |
17+| `apps/` | What people open: `web` (the site), `api`, `status`, `sudo`, `docs`, and the `desktop` (Tauri) and `mobile` (Expo) apps |
18+| `services/` | The services, one folder each, with their `wrangler.jsonc` and `migrations` |
19+| `crates/` | Shared Rust: `contracts`, `kit`, `rules`, `scan`, `secrets`, `blobstore`, `actions`, the `runner` binary and the `g1t` CLI |
20+| `packages/` | Shared TypeScript: `contracts` and `theme` (the design tokens) |
21+| `deploy/` | `stack.jsonc`, the inventory; `self-host`, the Docker Compose installation; `runner` and `sandbox`, the self-hosted runner's images |
22+| `scripts/` | `deploy.mjs` and its helpers, releases, operations |
23+| `plugins/` | The [Claude Code plugin](/guides/claude-code-plugin/), and setup for other coding agents |
24+| `.g1t/workflows/` | CI, deploys and releases, run by g1t Actions |
25+| `conformance/` | Checks that a job's environment matches what workflows expect |
26+
27+## Building
28+
29+`deploy/stack.jsonc` gives each unit a kind, and each kind builds its own
30+way:
31+
32+| Kind | Built by | Units |
33+| --- | --- | --- |
34+| `rust-worker` | `scripts/build-rust-worker.mjs`: `worker-build` to WebAssembly, then `wasm-opt` | The Rust services and `apps/api` |
35+| `ts-worker` | Wrangler's bundler | The TypeScript services |
36+| `react-router` | `vite build`, then Wrangler | `apps/web`, `apps/sudo` |
37+| `astro` | `astro build` | `apps/docs` |
38+
39+The runner's image is built in two layers: a large base with every
40+toolchain, rebuilt only when its folder changes and recorded in
41+`services/runner/base.json`, and a thin layer on top with the static
42+`g1t-runner` binary.
43+
44+## Deploying
45+
46+<Diagram name="deploy-stages" />
47+
48+A push to `main` starts `.g1t/workflows/deploy.yml`, one run at a time,
49+never cancelled:
50+
51+1. **Check and plan.** The manifest is checked against every config and
52+ `npm run test:deploy` runs. Beside it, `scripts/deploy.mjs plan` works
53+ out which units changed since the commit each one is running.
54+2. **Migrations.** Every pending D1 migration, before any code.
55+3. **Core, edge, front.** The units in each stage deploy in parallel; a
56+ stage starts only when the one before succeeded. A unit binds only to
57+ units in its own stage or an earlier one, which a test checks, so new
58+ code never calls a service that has not shipped yet.
59+4. **Smoke.** `scripts/ops/smoke.mjs` checks that sign-in and sign-up still
60+ work, without making an account.
61+
62+Each run is one `production` deployment, shown on the commit as the
63+`deploy / production` status. Only units whose inputs changed are
64+deployed: their folder, or a crate or package they are built from, which
65+`deploy.mjs` reads from Cargo's and npm's workspace metadata.
66+[Deploy g1t to Cloudflare](/guides/deploy-to-cloudflare/) covers the
67+tool, rollbacks and a first deploy.
68+
69+## Tests that guard the architecture
70+
71+Some of what this section describes is enforced by tests, so a change that
72+breaks it fails CI:
73+
74+| Test | Fails when |
75+| --- | --- |
76+| `npm run test:deploy` | A unit is missing from `stack.jsonc`, a binding crosses stages the wrong way, a unit is missing from [How a self-hosted g1t runs](/guides/self-hosting-architecture/), or [Services and contracts](/architecture/services/) disagrees with a `wrangler.jsonc` |
77+| `cargo test -p g1t-contracts` | The Rust and TypeScript access rules or scopes differ |
78+| `cargo test -p g1t-api openapi` | The docs' copy of the OpenAPI document is out of date |
79+| `app/lib/front-door-limits.test.ts` in `apps/web` | A `wrangler.jsonc` and `packages/contracts/src/rate-limits.ts` disagree |
80+| `npm test` in `apps/docs` | A diagram's arrow names a box that does not exist, two boxes overlap, or a label does not fit its box |
81+
82+CI (`.g1t/workflows/ci.yml`) runs the Rust tests, the TypeScript checks and
83+tests, and builds the site, sudo and these docs on every pull request.
84+
85+## Keeping this section true
86+
87+These pages describe the code on `main`. When you change how g1t is put
88+together, change them in the same pull request:
89+
90+| If you change | Update |
91+| --- | --- |
92+| A service's bindings, storage or job | Its row in [Services and contracts](/architecture/services/#every-unit) and [Data and storage](/architecture/data/) |
93+| A request's path, a limit or a deadline | [How a request is served](/architecture/requests/) |
94+| An event type or a route to a subscriber | [Events](/architecture/events/) |
95+| The push, fetch or merge path | [Git](/architecture/git/) and [Work and the merge queue](/architecture/work/) |
96+| Sandboxes, runners or egress | [Actions, runners and sandboxes](/architecture/compute/) |
97+| A new dependency on a Cloudflare product | Its port in [Data and storage](/architecture/data/#ports-and-adapters) |
98+
99+Diagrams are data, in `apps/docs/src/diagrams/architecture.ts`. Change the
100+spec, and both the picture and the page's Markdown copy change with it.
101+
102+<Aside title="Run it yourself">
103+ `docker compose -f deploy/self-host/docker-compose.yml up --build`
104+ builds every unit from source and starts the site on port 8787 and the
105+ API on 8789. [Run g1t yourself](/guides/self-hosting/) has the rest.
106+</Aside>
+175−0
1+---
2+title: Work and the merge queue
3+description: How the work service keeps issues, pull requests, checks and rulesets, coordinates sessions and claims, decides and lands a merge, and tests many changes at once in the merge queue.
4+---
5+
6+import Aside from '../../../components/Aside.astro';
7+import Diagram from '../../../components/Diagram.astro';
8+
9+`work` (`services/work`, Rust) keeps everything about a change that is not
10+the code itself: issues, pull requests, comments, reviews, labels,
11+milestones, rulesets, checks and commit statuses, CODEOWNERS, and the
12+merge queue. Its database is `g1t-work`. The code lives in `repos`, and
13+`work` asks `repos` whenever it needs it.
14+
15+## Issues and pull requests
16+
17+Issues and pull requests share one sequence of numbers per repository
18+(the `counters` table), so `#12` names exactly one of them. A pull request
19+records where its head is: a fork of its own, `pulls--<id>` in the git
20+store, or a branch in the repository ([Git](/architecture/git/#pull-request-forks)).
21+
22+`work` reads every push from its event queue. A push to a pull request's
23+head updates it: the new head, the files it changes, whether it is behind
24+the base, and which other open pull requests change the same files, the
25+overlap shown on each.
26+
27+When a pull request merges, its issue closes and records which pull
28+request resolved it, and the issue's other open pull requests close as
29+superseded ([How g1t works](/concepts/overview/#several-pull-requests-for-one-issue)).
30+
31+## Coordination
32+
33+Many agents and people can work on one repository at once. `work` keeps
34+what lets them see each other, around the issue, which is the unit of
35+work (`services/work/src/coordination/`):
36+
37+| Table | Holds | Module |
38+| --- | --- | --- |
39+| `sessions` | One run of someone's own harness (`claude_code`, `codex`, `cursor`, `copilot` or `other`), attached to an issue, a pull request and a branch | `sessions.rs` |
40+| `session_entries` | What a session recorded: prompts, reasoning, tool calls, notes, each with an idempotency `key` | `sessions.rs` |
41+| `claims` | An advisory, expiring "I'm working on #12" | `claims.rs` |
42+| Issues' `blocked_by` | The issues that have to land first, refused if it would make a cycle | `dependencies.rs` |
43+| `overlap_pairs` | Pairs of open pull requests changing the same files, announced once per pair | `overlaps.rs` |
44+
45+<Diagram name="coordination" />
46+
47+Three choices shape it:
48+
49+- **Nothing is a lock.** A claim is advice: anyone may claim an issue
50+ someone else holds, is told about them, and decides. Nothing is ever
51+ refused because of a claim (step 9 in the diagram).
52+- **Nothing waits on a harness.** No cron sweeps for quiet sessions or
53+ stale claims. The next read or change of a repository's coordination
54+ finds them (`sweep`) and announces them then: a session quiet for 15
55+ minutes is idle, one quiet for 2 hours has ended, and a claim not renewed
56+ within its time (30 minutes unless asked, at most 240) has expired.
57+- **Everything can be sent twice.** Starting a session is idempotent on the
58+ harness's own session id, and each entry on its `key`, so a hook can call
59+ every time without checking first, and a retry after a lost answer does
60+ no harm.
61+
62+`get_work` gathers an issue's whole context in one call (`context.rs`): the
63+issue, its definition of done, the required checks, claims, sessions,
64+pull requests with their checks, overlaps and place in the queue, what it
65+is blocked by and blocks, its decision records, and its latest
66+coordination events. A session's entries on a pull request are what
67+**Blame** reads to show the agent's account of a line ([Sessions and
68+why-blame](/guides/why-blame/)). [Sessions, claims and
69+dependencies](/guides/sessions/) has the calls.
70+
71+## Checks and statuses
72+
73+Anything can report on a commit: a workflow, an integration, g1t itself.
74+Each report is a status with a `context` (its name) and a state, kept per
75+commit in `commit_statuses`; richer reports are check runs and suites.
76+
77+Actions reports one status per run, named `<workflow> / <event>`, such as
78+`CI / pull_request` (`services/actions/src/plan.rs`). A required check
79+named `CI` is met by a status of that name for any event:
80+`check_name` in `crates/contracts/src/work.rs` reads the name before the
81+last ` / `. One that failed fails the check, even if another passed.
82+
83+## Deciding a merge
84+
85+Rulesets are stored in `work`, and judged by `crates/rules`, which does no
86+I/O and so can be tested in full. The same rules run in two places:
87+
88+| When | Who asks | What is judged |
89+| --- | --- | --- |
90+| A push | `repos`, before anything is stored | Which refs may move, and how: deletion, force-push, signed commits, file paths |
91+| A merge | `work` | Required reviews, required checks, CODEOWNERS, up to date, the merge queue |
92+
93+`requirements` gathers every rule that applies to the target branch, and
94+`judge` returns, for each, whether it passes, fails or is still waiting,
95+and whether the person asking may bypass it. A merge happens only when
96+none fails or waits, for people and coding agents alike.
97+
98+When everything passes, `work` asks `repos` to land the pull request's
99+head on the base, and publishes `pull.merged`.
100+
101+## The merge queue
102+
103+Two changes can merge without a conflict and still break each other. A
104+repository whose rules require the merge queue tests each pull request
105+together with the ones ahead of it, and moves `main` only to a state whose
106+required checks passed (`services/work/src/queue.rs`).
107+
108+<Diagram name="merge-queue" />
109+
110+1. **Build.** Up to `max_entries_to_build` entries (4 by default) are
111+ tested at once. For each, a sandbox merges the default branch with that
112+ entry and every entry ahead of it, and pushes the result to
113+ `g1t-queue/<entry>` in the repository.
114+2. **Test.** Workflows that run on `merge_group` run on each of those
115+ commits, and the required checks have to pass there.
116+3. **Land.** Entries land in order. Once everything ahead of it has landed,
117+ an entry moves the default branch to the very commit that was tested.
118+4. **Eject.** An entry whose checks fail leaves the queue with a failed
119+ check run on its pull request, and the entries behind it are tested
120+ again without it.
121+
122+| State | Means |
123+| --- | --- |
124+| `waiting` | In the queue, not yet being built |
125+| `testing` | Built, its checks running |
126+| `passed` | Its checks passed; it lands when those ahead have |
127+| `failed` | Its checks failed; it leaves the queue |
128+| `landed` | On the default branch |
129+| `removed` | Taken out by hand |
130+
131+### Who answers for a failure
132+
133+A combined state can fail because of the entry itself, or because of an
134+entry ahead of it. `integration.rs` works out which:
135+
136+- **Held, not blamed.** A state that fails while an entry ahead is still
137+ being tested is held. If that entry fails too, the held one is tested
138+ again without it and is not blamed. Only once everything ahead has
139+ passed does the held entry answer for the failure, so a broken change is
140+ blamed on itself, never on what was queued behind it.
141+- **Narrowed down.** An entry whose own checks pass but whose combination
142+ fails breaks *with* some of the entries ahead. `work` builds the default
143+ branch with half of them and the entry, and halves again, within three
144+ extra builds (`PROBE_BUDGET`), recorded in `queue_probes`. The builds run
145+ on `merge_group` like any queued state and never land. What is left is
146+ reported as the `suspects`.
147+- **Told everywhere.** The answer goes on the pull request's and the
148+ issue's timelines, into the pull request's session, as a decision record
149+ (`integration_failure`, confidence `tested`, in `decisions`), and as the
150+ `integration.failed` event. A passing combination is recorded as
151+ `integration.passed`.
152+- **Rejoining.** A pull request the queue took out keeps its merge
153+ request. Once a new commit's checks pass, it is merged again, into the
154+ queue, as whoever queued it, by the rules as they are then.
155+
156+[The merge queue](/guides/merge-queue/#who-answers-for-it) has the table of
157+outcomes from a person's side.
158+
159+<Aside title="What a merge queue needs">
160+ Building candidates takes sandboxes, so the merge queue needs a paid or
161+ trial workspace, or the open-source pool for a public repository.
162+ [The merge queue](/guides/merge-queue/) has the settings.
163+</Aside>
164+
165+## Events it publishes and reads
166+
167+`work` publishes what happens to issues, pull requests, comments, checks
168+and the queue (`issue.*`, `pull.*`, `comment.*`, `status.created`,
169+`queue.changed` and the like), and the coordination events:
170+`session.started`, `session.ended`, `work.claimed`, `work.released`,
171+`work.claim_expired`, `work.handoff_requested`, `dependency.changed`,
172+`overlap.detected`, `integration.failed`, `integration.passed` and
173+`decision.recorded`. It reads
174+`git.push` and a few repository events from `g1t-events-work`
175+([Events](/architecture/events/#subscriber-queues)).
+1−0
218218 opened, a pull request being merged, a session growing. Events are delivered
219219 to the services that react to them and are kept as a timeline per
220220 repository, which you can read through the [API](/reference/api/).
221+[Events](/architecture/events/) in the Architecture section shows how.
221222
222223 ## What is not built yet
223224
+2−1
66 [Run g1t yourself](/guides/self-hosting/) starts g1t with Docker Compose.
77 This page is for the person who keeps that installation running: what is
88 inside it, what stands in for each Cloudflare service g1t.sh uses, and
9−where the limits are.
9+where the limits are. How the services themselves work, wherever they
10+run, is in the [Architecture](/architecture/overview/) section.
1011
1112 ## The shape
1213
+1−0
117117 | Storage | Databases and buckets your teams can create, at a level each team is given, with copies of production data for anything not yet reviewed. | <Status is="coming" /> |
118118 | [Audit log](/guides/audit-log/) | Every change to the workspace, with export. | <Status is="live" /> |
119119 | [Run g1t yourself](/guides/self-hosting/) | The core forge in Docker. Deployments are off in this version. | <Status is="preview" /> |
120+| [Architecture](/architecture/overview/) | How g1t is built, service by service, for developers building on it, contributing to it or running it. | <Status is="live" /> |
120121
121122 ## Bring a repository
122123
+516−0
Loading the diff…
+53−0
Loading the diff…
+449−0
Loading the diff…
+10−0
Loading the diff…
+16−0
Loading the diff…
+287−0
Loading the diff…
+1−0
Loading the diff…
+4−1
Loading the diff…
+12−20
Loading the diff…
+94−0
Loading the diff…
+20−42
Loading the diff…
+16−0
Loading the diff…