Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 1 | # Self-hosting g1t |
| 2 | ||
| 3 | The goal (2026-10-05): g1t should not be locked to Cloudflare. Anyone should | |
| 4 | be able to run it on their own machine with `docker compose up`. The free | |
| 5 | core is MIT and self-hostable; managed hosting at g1t.sh is the paid | |
| 6 | product, and it stays on Cloudflare. Self-hosting must never make hosted | |
| 7 | g1t worse, so hosted code paths do not change to make room for it. | |
| 8 | ||
| 9 | This document covers: | |
| 10 | ||
| 11 | 1. An inventory of every Cloudflare dependency in the code. | |
| 12 | 2. The design: ports and adapters, with the runtime choice weighed. | |
| 13 | 3. What phase 1 ships today: `deploy/self-host/`, and what was verified. | |
| 14 | 4. The phased plan, with estimates. | |
| 15 | 5. The risks. | |
| 16 | ||
| 17 | The user guide is `apps/docs/src/content/docs/guides/self-hosting.md`. | |
| 18 | ||
| 19 | ## The short version | |
| 20 | ||
| 21 | - **g1t is already shaped for this.** Every service talks to every other | |
| 22 | over plain HTTP: `POST /rpc/<method>` with a JSON body (`crates/kit`, | |
| 23 | `packages/contracts/src/clients.ts`). Every database is SQLite (D1). | |
| 24 | Git storage sits behind a `GitStore` port (`services/repos/src/store.rs`), | |
| 25 | and everything past that port speaks git's smart HTTP to a remote URL | |
| 26 | with a bearer token. | |
| 27 | - **The fastest credible path is to run the Workers themselves in | |
| 28 | workerd**, the open-source Workers runtime, not to port them. Phase 1 | |
| 29 | runs every core Worker unchanged (the same WebAssembly and the same | |
| 30 | bundles that deploy to Cloudflare) in one workerd process under | |
| 31 | `wrangler dev`. D1, KV and Queues are kept on a volume as SQLite files. | |
| 32 | - **Three things replace Cloudflare-only bindings**, as small Workers bound | |
| 33 | in their place, with no change to the services: | |
| 34 | - `ARTIFACTS` becomes an Artifacts-compatible shim in front of a git | |
| 35 | store, which keeps plain bare repositories on disk and serves them | |
| 36 | with `git http-backend`. | |
| 37 | - `EMAIL` becomes a shim that logs each message and hands it to Mailpit, | |
| 38 | which can relay to any SMTP server. | |
| 39 | - The runner and the context hub are bound to an "off" Worker, which | |
| 40 | answers "agents are off" instead of failing. | |
| 41 | - **Proven on this machine with Docker:** sign up, confirm the email | |
| 42 | through Mailpit, create a workspace and a repository, push and clone over | |
| Docs: the self-hosted object store is RustFS | 43 | HTTP (the second clone from the clone pack cache in RustFS), open an |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 44 | issue, and browse code, commits and files in the site; then the REST API, |
| Docs: the self-hosted object store is RustFS | 45 | OAuth metadata and MCP on their own port, an npm package published and |
| 46 | installed and a container image pushed and pulled through RustFS, pull | |
| 47 | requests from a branch and from a fork merged onto `main`, the merge queue taking a pull request and | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 48 | giving it back, and every cron handler the scheduler runs. All of it runs |
| 49 | against local storage. See [Phase 1: what works today](#3-phase-1-what-works-today). | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 50 | - **Long term:** keep workerd as the runtime, because it is what hosted |
| 51 | runs. Replace `wrangler dev` with a production workerd configuration. | |
| 52 | Move the binding shims into code-level ports in `g1t_kit` and a TS | |
| 53 | `@g1t/platform` package, so each primitive has a hosted and a | |
| 54 | self-hosted adapter behind one interface. | |
| 55 | ||
| 56 | ## 1. Inventory | |
| 57 | ||
| 58 | The sources are every `wrangler.jsonc` plus a grep of the code. Coupling | |
| 59 | is graded: | |
| 60 | ||
| 61 | - **thin**: one call site or a config switch; | |
| 62 | - **adapter**: already behind a port, or easy to put behind one; | |
| 63 | - **woven**: the logic is shaped around the product. | |
| 64 | ||
| 65 | ### By primitive | |
| 66 | ||
| 67 | | Primitive | Where | Coupling | Self-hosted equivalent | | |
| 68 | | --- | --- | --- | --- | | |
| 69 | | **Workers runtime**, service bindings | Every service. Rust through `worker` 0.8 (`#[event(fetch\|queue\|scheduled)]`, `Env`, `Fetcher`); TS as `export default { fetch, queue, scheduled }` | woven (as a runtime), thin (as an API) | **workerd**: the same runtime, open source. Service bindings work as they do hosted. Calls are HTTP (`POST /rpc/<method>`), so a native port could use plain HTTP clients. | | |
| 70 | | **Workers RPC** (JS methods across a binding) | Only `RUNNER`: `RunnerService extends WorkerEntrypoint` (`services/runner/src/index.ts:626`). `apps/web` calls `env.RUNNER.enabled/run/plan/...` directly in 11 routes. | thin | workerd supports it. A native port needs these on `/rpc/*` as well; the runner already has a `fetch` shim for Rust callers. | | |
| 71 | | **D1** | System of record for 13 services. Rust: `env.d1("DB")`; TS: `D1Database`; `db.batch()` in `crates/kit` `rename` | woven (SQL), thin (API) | **SQLite files**. workerd/Miniflare implements D1 on SQLite, and the same `migrations/` apply with `wrangler d1 migrations apply --local`. A native port would need a `Database` port over `rusqlite`/`better-sqlite3`; the SQL is already SQLite, including FTS5. | | |
| Docs: the self-hosted object store is RustFS | 72 | | **KV** | `BLOBS`: Actions artifacts and cache (`apps/api/src/blobs.rs`, `apps/web/app/lib/artifacts.server.ts`). `AVATARS`: `services/identity/src/avatars.rs`, `apps/web/workers/app.ts`, `services/og`. `DOMAINS`: `services/deployments/src/domains.ts`, `services/pages` | thin | Miniflare KV on disk (SQLite plus blob files). Natively: a `BlobStore` port on the filesystem or S3. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 73 | | **Queues**: the event bus | Producer: `services/events` `BUS.sendBatch` (`lib.rs:67`). The consumer writes the log, then fans out to every binding named `SUBSCRIBER_*` (`lib.rs:161`). Twelve consumers, one queue each. Private job queues in search (`g1t-search-jobs`) and context (`g1t-context-jobs`); consumers branch on the queue name. | woven | Miniflare Queues: in-process and persisted, which works today. Natively: a `Bus` port with a SQLite outbox and a poller per subscriber, or NATS/Redis Streams. At-least-once delivery and idempotent consumers are already the contract. | |
| 74 | | **Durable Objects** | Only `AttemptSandbox` (runner), as the containers library's base class. Uses `ctx.storage.get/put/delete`, `schedule()` (alarm), `idFromName`/`idFromString`, DO RPC (`run`, `destroy`, `noteBlocked`). **Not used:** WebSocket hibernation, raw `alarm()`, `ctx.storage.sql`, `ctx.exports`. | woven, in the runner only | workerd supports Durable Objects (on-disk SQLite). Runner state can move to the sandbox supervisor (phase 2). | | |
| 75 | | **Containers** (`@cloudflare/containers`) | `services/runner`: one sandbox per agent run, Actions job and deploy build. `sleepAfter`, `start({ envVars, enableInternet })`, `onStop`. Image: `services/runner/Dockerfile` (node 24, git, toolchains, Claude Code, `g1t-runner`). | woven | **Docker or Podman** through the socket, with the same image. Wrangler can already run Containers locally through Docker; whether that covers outbound interception has to be tested. | | |
| Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look | 76 | | **Outbound interception** (guardrails egress) | `services/runner/src/guard.ts` (`egress()`, `abuse()`), `egress.ts` (`sandboxHosts`, `buildHosts`, `EGRESS_CA = /etc/cloudflare/certs/cloudflare-containers-ca.crt`), `AttemptSandbox.outboundHandlers`, `interceptHttps = true`, `setOutboundHandler("egress", { hosts })`, `setOutboundByHost("sandbox.g1t.internal", "abuse")` (a sandbox reporting it stopped itself for mining; `crates/runner/src/abuse.rs`) | woven | The sandbox joins an internal network with no route out, and gets `HTTP(S)_PROXY` pointing at an allow-list proxy that checks the `CONNECT` host. The CA bundle is then not needed, because nothing is re-signed. Blocked hosts are reported to the runner as `noteBlocked` does today. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 77 | | **Artifacts** (git storage) | Only `services/repos/src/store.rs` (`ArtifactsStore`, behind the `GitStore`/`GitRepo` traits). Methods used: `create`, `get`; then `info`, `createToken`, `log`, `readCommit`, `readTree`, `readBlob`, `readFile`, `fork`, and `[Symbol.dispose]`. Everything else (push, fetch, landing, catch-up, import, ref listing) is smart HTTP to `info().remote` with `Bearer <token>`: `land.rs`, `catch_up.rs`, `refs.rs`, `import.rs`, `git_http.rs`. | adapter | **Bare repositories on disk plus `git http-backend`.** Built in phase 1: `deploy/self-host/gitstore`. Forks are local clones with hard links. Tokens are HMAC-signed, scoped, and expire. | |
| 78 | | **Cache API** | `services/repos/src/store.rs` (trees and blobs up to 1 MiB, by hash), `apps/web/workers/app.ts` (avatars), `services/og` | thin, optional | Miniflare's cache, or none. Every use tolerates a miss. | | |
| 79 | | **Vectorize** | `services/context/src/index.ts` (`VECTORS.upsert/deleteByIds/query`); optional, guarded by `if (!AI \|\| !VECTORS)` | thin | **sqlite-vec** (default: one file, next to D1), pgvector or Qdrant behind a `VectorIndex` port; or off, which already degrades to keyword search. | | |
| 80 | | **Workers AI** | `services/context` only: `@cf/baai/bge-base-en-v1.5` embeddings (768 dims) | thin | An OpenAI-compatible `/v1/embeddings` endpoint (Ollama, vLLM, LM Studio, or a hosted API) behind an `Embedder` port. Changing models means re-embedding (a backfill job already exists). | | |
| Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier | 81 | | **AI Gateway** | `services/runner/src/model-env.ts` (`modelEnv`), `services/models/src/route.ts:70` (gateway URL, `cf-aig-*` headers), `services/billing/src/keeper.rs` (reads gateway logs to settle) | thin | Optional already: an empty `AI_GATEWAY_ID` goes straight to the provider. Any Anthropic- or OpenAI-compatible base URL works for a workspace's own provider. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 82 | | **Workers for Platforms** | `services/pages` (dispatcher: `env.APPS.get(script).fetch`), `services/deployments/src/cloudflare.ts` (script and asset upload through the REST API) | woven | Phase 1: off. Later: a self-hosted app host in workerd, using the Worker Loader binding to load uploaded scripts, or a workerd per app (see [Deployments](#deployments)). | |
| 83 | | **Cloudflare for SaaS** (custom hostnames) | `services/deployments/src/custom-hostnames.ts` (`/zones/{id}/custom_hostnames`) | thin | Caddy with on-demand TLS, asking g1t whether a hostname is allowed. | | |
| 84 | | **Cloudflare REST API** | deployments: script upload, list, delete, assets, GraphQL usage. Billing keeper: AI Gateway logs, `billable-usage`, GraphQL container usage. Ops scripts in `scripts/`. | thin (deployments), woven (keeper pricing) | Deployments: the app-host adapter. Keeper: off when self-hosted, because there is no bill to reconcile. | | |
| 85 | | **Email Sending** | `services/identity/src/email.rs` (`EMAIL.send({to, from, subject, text, html})`); callers: verification, password reset, `admin.rs` limit warnings | thin | Built in phase 1: a shim that logs and hands mail to Mailpit, which relays over SMTP. Later: a `Mailer` port with an SMTP adapter. | | |
| 86 | | **Cloudflare Access** | `apps/sudo/app/lib/access.ts` (verifies `Cf-Access-Jwt-Assertion` against `/cdn-cgi/access/certs`, `ACCESS_AUD`, `STAFF_EMAILS`) | woven, in sudo only | A local admin flag: `G1T_ADMINS` usernames checked against the normal g1t session. Self-hosters rarely need sudo, which is about billing. | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 87 | | **Cron Triggers** | actions (every minute), webhooks (every minute), identity (`*/15`), security (`*/30`), repos and packages (hourly), events (daily), billing (`*/15` and daily), deployments (`*/10`), runner (`*/5`) | thin | workerd runs `scheduled()` when asked, but never on its own. `deploy/self-host/scheduler.mjs` asks: once a minute, inside the g1t container, it runs each due cron through Wrangler's local API (`POST /cdn-cgi/local/explorer/api/local/scheduled?worker=<name>`, answered only on localhost), the same handler Cron Triggers run. The services and crons come from `schedules.json`, which `configs.mjs` writes from each `wrangler.jsonc` for the services in its `SELF_HOST_CRONS`: repos, events, identity, security, webhooks and packages. Not run: actions (it would start scheduled workflows with no runner), billing (Cloudflare and Stripe) and deployments (Cloudflare's API). A handler still running from the minute before is not started again, and one is given up on after ten minutes. `scheduler.mjs --once` runs every cron of every service now and exits non-zero if one failed (smoke.sh uses it). The status page has its own loop (`status.sh`). | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 88 | | **`cloudflare:workers` imports** | `apps/web` (`env` in 15 files), `apps/sudo`, `services/runner` (`WorkerEntrypoint`) | thin | Provided by workerd. A Node port would pass `env` through context instead. | |
| 89 | | **Static Assets** | `apps/web` (Vite plugin build), `apps/docs`, `apps/sudo` (`run_worker_first`) | thin | workerd serves them. | | |
| 90 | | **`placement`, `observability`, routes, custom domains** | every `wrangler.jsonc` | config only | Dropped by `deploy/self-host/configs.mjs`. | | |
| 91 | | **`cf-ray`** | Used as an audit request id, with a fallback: `services/repos/src/run_access.rs:131`, `apps/api/src/audit.rs:37` | thin | Falls back already. | | |
| Docs: the self-hosted object store is RustFS | 92 | | **R2** | `services/packages` (`BLOBS`: container layers and other package files), `services/repos` (`BACKUPS`: nightly backup bundles; `GIT_PACKS`: the clone pack cache), the API's Actions cache (`ACTIONS_CACHE`), the runner's downloads | thin | **S3-compatible storage**: the `BlobStore` port in `crates/blobstore` has an R2 adapter and an S3 one (`s3.rs`, SigV4 over fetch); each service names its own bucket (`BLOB_STORE`/`S3_BUCKET` for packages, `BACKUP_STORE`/`BACKUP_S3_BUCKET` for backups, `PACK_STORE`/`PACK_S3_BUCKET` for clone packs), run against RustFS in the compose file. | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 93 | | **Not used** | Hyperdrive, Workflows, Analytics Engine, Browser Rendering, Images, Turnstile, Secrets Store, `connect()`, HTMLRewriter, `request.cf` | — | — | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 94 | |
| 95 | ### By service | |
| 96 | ||
| Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow | 97 | The deployable units, their stage and what each is built from are listed in |
| 98 | `deploy/stack.jsonc` (see `docs/DEPLOYING.md`); its `self_host` field is | |
| 99 | what `deploy/self-host/configs.mjs` runs, turns off or leaves out. A test | |
| 100 | checks this table names every unit. | |
| 101 | ||
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 102 | | Service | Runs on | Cloudflare dependencies beyond Workers and D1 | Phase 1 self-hosted | |
| 103 | | --- | --- | --- | --- | | |
| 104 | | `apps/web` | TS Worker plus assets | KV (`BLOBS`, `AVATARS`), Cache API, `cloudflare:workers` `env`, RPC to `RUNNER` | Runs unchanged | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 105 | | `apps/api` | Rust Worker | KV `BLOBS`, R2 `ACTIONS_CACHE`; `api.g1t.sh`/`mcp.g1t.sh` addresses (now the `API_URL`, `MCP_URL` and `SITE_URL` settings, hosted defaults when unset: `src/addresses.rs`) | Runs in a second workerd on its own port (`API_PORT`, 8789; `self_host: "separate"`), started by `start.sh` once the first is up. Its service bindings reach the other Workers through Wrangler's dev registry. `API_URL` (default: `PUBLIC_URL`'s host on `API_PORT`) is its OAuth issuer; MCP is the path `/mcp` on it (`MCP_URL`). Its KV is its own, apart from the site's: Actions artifacts need the runner, which is off | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 106 | | `apps/sudo` | TS Worker plus assets | Access JWT | Not run | |
| 107 | | `apps/docs` | Static | — | Not run (docs.g1t.sh serves them) | | |
| Deploys as code: a manifest of every Worker, a deploy tool that ships only what changed in parallel stages, and a g1t Actions workflow | 108 | | `apps/status` | TS Worker | Email Sending, cron; bound only to billing | Runs in a process of its own (`status.sh`), so it stays up when the site does not | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 109 | | `services/identity` | Rust | Email Sending, KV `AVATARS` | Runs unchanged; `EMAIL` goes to the mail shim | |
| Docs: the self-hosted object store is RustFS | 110 | | `services/repos` | Rust | **Artifacts**, **R2** (`BACKUPS`), Cache API, optional KV `GIT_CACHE` with `REPOS_KEY`, optional R2 `GIT_PACKS` | Runs unchanged; `ARTIFACTS` goes to the git store, backups to the `g1t-backups` bucket in RustFS (`BACKUP_STORE=s3`), and the clone pack cache to `g1t-git-packs` (`PACK_STORE=s3`: the `PackStore` port in `src/pack_cache.rs` over the shared `BlobStore`, multipart, an object only once whole; `storage-setup` gives the bucket a lifecycle rule that deletes packs after 7 days and aborts uploads unfinished after a day). Without `GIT_CACHE` and `REPOS_KEY`, credentials and ref listings are kept per isolate only. Its nightly cron queues backups, but bundles are cut by the runner, which is off in phase 1: none are made yet | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 111 | | `services/work` | Rust | Queue consumer | Runs unchanged | |
| 112 | | `services/events` | Rust | Queues (producer and fan-out) | Runs unchanged; the off services' queues are not produced to | | |
| 113 | | `services/projects` | TS | Queue consumer | Runs unchanged | | |
| 114 | | `services/search` | Rust | Queues (events and its own jobs); FTS5 | Runs unchanged | | |
| 115 | | `services/billing` | Rust | Cron, Cloudflare REST API (keeper), Stripe | Runs with `FREE_WHILE_BUILDING=true` and no Stripe key: nothing is charged | | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 116 | | `services/security` | Rust | Queue, cron | Runs unchanged; its cron through `scheduler.mjs` | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 117 | | `services/actions` | Rust | Queue, cron, `ACTIONS_KEY` | Runs; jobs need the runner, which is off | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 118 | | `services/webhooks` | Rust | Queue, cron, `WEBHOOKS_KEY` | Runs; retries through `scheduler.mjs` | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 119 | | `services/integrations` | Rust | Queue, `INTEGRATIONS_KEY` | Runs unchanged | |
| Docs: the self-hosted object store is RustFS | 120 | | `services/packages` | Rust | **R2** (`BLOBS`), cron, queue, `PACKAGES_TOKEN_SECRET` | Runs with `BLOB_STORE=s3` against the compose file's RustFS (`deploy/self-host/configs.mjs`); no request size limit (`MAX_REQUEST_BYTES` 0 means none) | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 121 | | `services/deployments` | TS | Workers for Platforms, REST API, KV `DOMAINS`, cron | Runs with no API token: nothing deploys | |
| 122 | | `services/runner` | TS | **Containers**, Durable Objects, outbound interception, AI Gateway, cron | Off: bound to the off Worker | | |
| 123 | | `services/context` | TS | **Vectorize**, **Workers AI**, Queues | Off: bound to the off Worker | | |
| 124 | | `services/models` | TS | AI Gateway; public at `models.g1t.sh` | Not run (only sandboxes call it) | | |
| 125 | | `services/pages` | TS | Dispatch namespace, wildcard routes, KV | Not run | | |
| 126 | | `services/og` | TS | Cache API | Not run (social cards are optional) | | |
| 127 | | `crates/runner` | native, in the sandbox | Talks to `https://api.g1t.sh` and `https://g1t.sh` (hard-coded in the runner Worker); `wrangler deploy --dry-run` for builds | Phase 2 | | |
| 128 | | `crates/sshd` | native | Not deployed; calls `/_internal/ssh/*` endpoints that do not exist yet | Phase 3 | | |
| 129 | ||
| 130 | ### Hard-coded hosted addresses | |
| 131 | ||
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 132 | Self-hosting needs one setting, `PUBLIC_URL`, in place of these, and |
| 133 | `configs.mjs` derives the rest from it. These are settings now, each with | |
| 134 | the hosted address as its default, so hosted g1t sets nothing: | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 135 | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 136 | - The site (`apps/web/app/lib/addresses.ts`): `SITE_URL`, `API_URL`, |
| 137 | `MCP_URL` and `OG_URL` (empty: no social card tags). The root loader | |
| 138 | hands them to the page; `meta.ts`, the clone box, agent setup, the | |
| 139 | pull request and merge box remotes, the tokens page and the OAuth | |
| 140 | consent's `iss` read them. | |
| 141 | - The API (`apps/api/src/addresses.rs`): `SITE_URL`, `API_URL` (the OAuth | |
| 142 | issuer) and `MCP_URL` (the protected resource; a path on the API's host | |
| 143 | self-hosted). | |
| 144 | - Identity's mail (`services/identity/src/email.rs`): `SITE_URL` for | |
| 145 | links and the logo, `MAIL_FROM` for the sender. The mail shim still | |
| 146 | rewrites any `https://g1t.sh` link left in a message. | |
| 147 | ||
| 148 | Still hard-coded, for phase 2: | |
| 149 | ||
| 150 | - `apps/web/workers/app.ts` (`DOCS`); defaults in | |
| 151 | `components/invites-section.tsx`, `components/runners.tsx`, | |
| 152 | `lib/invites.ts` and `lib/legal.ts`. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 153 | - `services/runner/src/index.ts` (`G1T_API`, `GIT_REMOTE` and the other |
| 154 | remotes handed to sandboxes, in 12 places). | |
| 155 | - `services/billing` (`stripe.rs`, `accounts.rs`, `limits.rs`). | |
| 156 | - `crates/sshd` (`G1T_API` default). | |
| 157 | ||
| 158 | There are about 150 occurrences of `g1t.sh` in non-test code. Most are | |
| 159 | docs links and copy, and need no change. | |
| 160 | ||
| 161 | ## 2. Design | |
| 162 | ||
| 163 | ### Principles | |
| 164 | ||
| 165 | 1. **Hosted is the reference.** Hosted code does not change behaviour to | |
| 166 | make room for self-hosting. A self-hosted adapter is added beside the | |
| 167 | hosted one, and the hosted one stays the default. | |
| 168 | 2. **Swap at the narrowest seam that exists.** A binding-shaped seam (a | |
| 169 | Worker that offers the same methods as the Cloudflare binding) needs no | |
| 170 | code change, and is how phase 1 works. A code-level port (a trait or | |
| 171 | interface with two adapters) is cleaner and testable, and is the long-term | |
| 172 | shape. Each primitive moves from the first to the second when it is next | |
| 173 | touched. | |
| 174 | 3. **One runtime, two hosts.** The Workers stay Workers. workerd runs them | |
| 175 | self-hosted, so one build serves both and there is no second code path | |
| 176 | to keep correct. | |
| 177 | 4. **Off is a real mode.** Every optional subsystem (agents, context, | |
| 178 | deployments, billing) has an "off" answer that pages already handle. | |
| 179 | ||
| 180 | ### The ports | |
| 181 | ||
| 182 | | Port | Hosted adapter | Self-hosted adapter | Lives in | Status | | |
| 183 | | --- | --- | --- | --- | --- | | |
| 184 | | `GitStore` / `GitRepo` | `ArtifactsStore` | Git store (bare repos, `git http-backend`) through an Artifacts-compatible shim; later a `LocalGitStore` that calls the git store's HTTP API from Rust directly | `services/repos/src/store.rs` (exists) | **Built** (binding level) | | |
| 185 | | `Mailer` | Email Sending binding | SMTP (through Mailpit relay now; a direct SMTP adapter later) | `g1t_kit::mail` | **Built** (binding level) | | |
| 186 | | `Sandbox` | `AttemptSandbox` (Containers, DO) | `DockerSandbox`: a supervisor that starts the runner image through the Docker API | `services/runner` (TS) | Phase 2 | | |
| 187 | | `Egress` | Containers outbound handler | Allow-list HTTP(S) proxy on an internal network | `services/runner` | Phase 2 | | |
| 188 | | `ModelRoute` | AI Gateway, or direct | Any Anthropic/OpenAI-compatible base URL | `services/models`, `runner/model-env.ts` | Exists (env-switchable) | | |
| 189 | | `Embedder` | Workers AI | OpenAI-compatible `/v1/embeddings` | `services/context` | Phase 3 | | |
| 190 | | `VectorIndex` | Vectorize | sqlite-vec / pgvector / Qdrant | `services/context` | Phase 3 | | |
| 191 | | `AppHost` | Workers for Platforms plus REST API | workerd app host (Worker Loader) | `services/deployments` | Phase 3 | | |
| 192 | | `Domains` | Cloudflare for SaaS | Caddy on-demand TLS | `services/deployments` | Phase 3 | | |
| 193 | | `AdminAuth` | Cloudflare Access | `G1T_ADMINS` plus the normal session | `apps/sudo` | Phase 4 | | |
| 194 | | `Bus` | Queues | Miniflare Queues now; SQLite outbox later | `services/events`, `g1t_kit` | Works (runtime) | | |
| 195 | | `Database` | D1 | SQLite files through workerd | — | Works (runtime) | | |
| 196 | | `Blobs` | KV | Miniflare KV now; filesystem/S3 later | — | Works (runtime) | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 197 | | `Scheduler` | Cron Triggers | `scheduler.mjs`: a ticker that calls `scheduled()` through Wrangler's local API | `deploy/self-host` | **Built** for the services in `SELF_HOST_CRONS`; each handler checked by `smoke.sh` | |
| 198 | | `PackStore` | R2 `GIT_PACKS` | S3 (`PACK_STORE=s3`) through `crates/blobstore` | `services/repos/src/pack_cache.rs` | **Built** | | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 199 | | `UsageKeeper` | Cloudflare bill plus AI Gateway logs | None (billing off) | `services/billing` | Off | |
| 200 | ||
| 201 | For Rust, the code-level ports go in `crates/kit` as traits (`g1t_kit::ports`), | |
| 202 | with the Cloudflare adapters next to them. For TypeScript, they go in a new | |
| 203 | `packages/platform` package, with each adapter in its own module so a | |
| 204 | hosted bundle never pulls in a self-hosted adapter. The adapter is chosen at | |
| 205 | startup from one setting, `G1T_MODE=hosted|self`, never per request. | |
| 206 | ||
| 207 | ### The runtime: workerd, or native binaries | |
| 208 | ||
| 209 | | | **workerd (phase 1: `wrangler dev`; later a plain workerd config)** | **Native: Rust on axum/hyper, TS on Node** | | |
| 210 | | --- | --- | --- | | |
| 211 | | Effort to first boot | **Done.** About a day, almost all of it shims and config. | Weeks. The Rust services use `worker::*` throughout: `Env`, `Fetcher`, `D1Database`, the `#[event]` macros and `js_sys` interop. Each needs an abstraction layer before it compiles natively. TS needs an `env` provider in place of `cloudflare:workers`, plus Queues, D1 and KV clients. | | |
| 212 | | Fidelity to hosted | **The same runtime and the same bundles.** A bug self-hosted is a bug hosted. | A second implementation of every platform API, with its own bugs. | | |
| 213 | | Performance | Good for one node: WebAssembly in V8, SQLite on local disk. Single-threaded per isolate; plenty for a team. | Better per core, and multi-threaded. That matters only at a scale where people use g1t.sh. | | |
| 214 | | Maintenance | Low. New hosted features run self-hosted for free, unless they add a new Cloudflare-only binding. The config generator then drops it or binds a stand-in. | High. Every feature is built twice, or behind a port that both sides keep honest. | | |
| 215 | | Scale-out | One process, one disk. D1-on-SQLite is single-writer. | Could use Postgres and many processes. | | |
| 216 | | Operational risk | `wrangler dev` is a development tool (see risks). Moving to `workerd serve` with a generated config, or a small Miniflare-API launcher, removes the dev-tool surface. | Conventional. | | |
| 217 | ||
| 218 | **Recommendation.** | |
| 219 | ||
| 220 | - **Phase 1:** workerd under `wrangler dev`, as built in `deploy/self-host/`. | |
| 221 | It needs no change to any service and is proven end to end. | |
| 222 | - **Phase 2:** replace `wrangler dev` with a launcher that drives Miniflare's | |
| 223 | API directly, or a generated `workerd` config. It should expose only the | |
| 224 | site and the API, with no dev endpoints, run cron, and run as a proper | |
| 225 | service. | |
| 226 | - **Long term:** keep workerd as the runtime and push the remaining | |
| 227 | Cloudflare-only bindings behind code-level ports. Compile to native only | |
| 228 | what already is native (the runner binary, sshd), or a service where | |
| 229 | workerd is a real limit, case by case. | |
| 230 | ||
| 231 | A full native port is not worth it. It would double the maintenance of | |
| 232 | every feature, which is the opposite of what makes self-hosting | |
| 233 | sustainable for a small team. | |
| 234 | ||
| 235 | ### Sandboxes (phase 2) | |
| 236 | ||
| 237 | - **Engine.** Docker or Podman through the socket, mounted into a small | |
| 238 | supervisor (`g1t-sandboxd`), not into the workerd container. The | |
| 239 | supervisor exposes the runner's `Sandbox` port over HTTP: start with | |
| 240 | env, stop, status, and the exit report that becomes `onStop`. It uses | |
| 241 | the **same image** (`services/runner/Dockerfile`). | |
| 242 | - **Runner.** The runner Worker runs in workerd with `AttemptSandbox` | |
| 243 | replaced by a `DockerSandbox` adapter. Run state that lives in | |
| 244 | `ctx.storage` moves to the supervisor's SQLite; `schedule()`/`timeUp` | |
| 245 | becomes a supervisor timer. | |
| 246 | - **Egress guardrails.** Each restricted sandbox joins an internal Docker | |
| 247 | network with no default route. Its `HTTP_PROXY`/`HTTPS_PROXY` point at an | |
| 248 | allow-list proxy (for example a small Go or Node `CONNECT` proxy) that | |
| 249 | admits the hosts `sandboxHosts()` computes and reports refusals. | |
| 250 | Nothing is intercepted, so no CA is injected. `EGRESS=off` puts the | |
| 251 | sandbox on a normal network. | |
| 252 | - **Addresses.** `G1T_API`, `GIT_REMOTE` and the rest become | |
| 253 | `PUBLIC_URL`-derived settings. Sandboxes reach the site and API on the | |
| 254 | compose network. | |
| 255 | - **Shortcut to evaluate first.** Wrangler can already run Containers | |
| 256 | classes locally through Docker. If its local Containers support | |
| 257 | `setOutboundHandler`, the runner could run nearly unchanged. Test this | |
| 258 | before building the supervisor. | |
| 259 | ||
| 260 | ### Git storage | |
| 261 | ||
| 262 | Artifacts gives g1t: named repositories, scoped short-lived tokens, a smart | |
| 263 | HTTP remote, typed reads (commit, tree, blob, file, log), copy-on-write | |
| 264 | forks, and jurisdictions. g1t uses everything except jurisdictions. | |
| 265 | ||
| 266 | The self-hosted git store (`deploy/self-host/gitstore/server.mjs`, about 450 | |
| 267 | lines of Node with no dependencies) provides the same: | |
| 268 | ||
| 269 | | Artifacts | Git store | | |
| 270 | | --- | --- | | |
| 271 | | `create(name, { setDefaultBranch, description })` | `git init --bare --initial-branch`, plus `g1t.json` beside it for metadata | | |
| 272 | | `get(name)` then `info()` | Metadata, `HEAD`, and the last push time; `remote` is `GITSTORE_URL/git/<name>.git` | | |
| 273 | | `createToken(scope, ttl)` | HMAC-SHA256 over `{ key, scope, expiry }` with the shared secret | | |
| 274 | | Smart HTTP remote | `git http-backend`; a read token cannot push | | |
| 275 | | `log`, `readCommit`, `readTree`, `readBlob`, `readFile` | `git rev-list --first-parent`, `cat-file`, `ls-tree` | | |
| 276 | | `fork(name, { defaultBranchOnly })` | `git clone --bare [--single-branch]` with hard-linked objects | | |
| 277 | ||
| 278 | Hosted is untouched: `ArtifactsStore` is still the only adapter compiled | |
| 279 | into `services/repos`. Self-hosted, the `ARTIFACTS` binding is a service | |
| 280 | binding to `deploy/self-host/workers/artifacts`, which offers Artifacts' | |
| 281 | methods and calls the git store. Long term, a `LocalGitStore` adapter in | |
| 282 | Rust should call the git store's API directly, which removes the shim. The | |
| 283 | git store can later gain `git gc` scheduling and object-store-backed packs | |
| 284 | for large installations. | |
| 285 | ||
| 286 | ### Search and context | |
| 287 | ||
| 288 | - **Site search** (`services/search`) is D1 FTS5. It works self-hosted as | |
| 289 | is (verified: `/search?q=hello` answers 200). | |
| 290 | - **Context hub** (`services/context`) needs an `Embedder` and a | |
| 291 | `VectorIndex`: | |
| 292 | - Default: **sqlite-vec** in the context service's own SQLite, with | |
| 293 | embeddings from an **OpenAI-compatible endpoint**. Ollama's | |
| 294 | `nomic-embed-text` has the same 768 dimensions as today's | |
| 295 | `bge-base-en-v1.5`. | |
| 296 | - Alternatives: pgvector or Qdrant, for installations that already run | |
| 297 | them. | |
| 298 | - Off: the service already degrades to keyword search when `AI` or | |
| 299 | `VECTORS` is missing (`index.ts:919`), so phase 3 can first run context | |
| 300 | with neither bound. | |
| 301 | ||
| 302 | ### Models | |
| 303 | ||
| 304 | Already portable. The model proxy (`services/models`) and the runner's | |
| 305 | model environment take any Anthropic-compatible base URL, and an empty | |
| 306 | `AI_GATEWAY_ID` skips AI Gateway. Self-hosted: | |
| 307 | ||
| 308 | - a workspace connects its own provider under Integrations (Anthropic, or | |
| 309 | any Anthropic-compatible endpoint, including a local gateway in front of | |
| 310 | OpenAI-compatible models); | |
| 311 | - "hosted models" are off, because there is no g1t key to spend. | |
| 312 | ||
| 313 | ### Deployments | |
| 314 | ||
| 315 | - **Phase 1–2: off.** `services/deployments` runs, but with no | |
| 316 | `CLOUDFLARE_API_TOKEN` nothing deploys. Its pages and settings still | |
| 317 | render. | |
| 318 | - **Phase 3: an app host on workerd.** The self-hosted `pages` dispatcher | |
| 319 | uses workerd's **Worker Loader** binding to load each uploaded app's | |
| 320 | modules on demand, keyed by deployment id, from the blob store. Static | |
| 321 | assets are served from the same store. Builds run in Docker sandboxes | |
| 322 | (phase 2), which already bundle with `wrangler deploy --dry-run`. | |
| 323 | Wildcard hosts (`*.apps.example.com`) and custom domains go through | |
| 324 | Caddy with on-demand TLS. The alternative, one workerd process per app, | |
| 325 | is simpler to isolate but heavier. | |
| 326 | ||
| 327 | ### Billing and the feature map | |
| 328 | ||
| 329 | Self-hosted billing is **off by default**: no Stripe, no usage limits, | |
| 330 | no keeper. Billing still runs, because the shell reads `billing.account` | |
| 331 | on every page, but with `FREE_WHILE_BUILDING=true` and no Stripe key. | |
| 332 | ||
| 333 | | Feature | Hosted (g1t.sh) | Self-hosted default | Self-hosted, when turned on | | |
| 334 | | --- | --- | --- | --- | | |
| 335 | | Accounts, workspaces, repos, git over HTTP | On | On | — | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 336 | | Issues, pull requests, review | On | On | — | |
| 337 | | Merge queue | On | Takes pull requests; testing and landing them needs sandboxes | Phase 2 | | |
| 338 | | Bringing a pull request up to date before it lands (catch-up) | On | Off: needs a sandbox | Phase 2 | | |
| Docs: the self-hosted object store is RustFS | 339 | | Clone pack cache | R2 | RustFS (`g1t-git-packs`) | — | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 340 | | Site search (FTS5) | On | On | — | |
| 341 | | Email | Email Sending | Mailpit, logged | SMTP relay | | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 342 | | Webhooks, integrations | On | On (retries through `scheduler.mjs`) | — | |
| g1t is one name: its agent's work, commits and comments show as @g1t, and nobody can claim g1t or g1t-agent | 343 | | g1t's agent | On | Off | Phase 2: Docker sandboxes plus your own model provider | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 344 | | Guardrails egress | Containers interception | n/a | Phase 2: allow-list proxy | |
| 345 | | Hosted models (g1t's key) | On (billed) | Off | Never: bring your own | | |
| 346 | | Context hub semantic search | Vectorize plus Workers AI | Off | Phase 3: sqlite-vec plus an OpenAI-compatible embedder | | |
| 347 | | Deployments | Workers for Platforms | Off | Phase 3: workerd app host | | |
| 348 | | Custom domains | Cloudflare for SaaS | Off | Phase 3: Caddy on-demand TLS | | |
| 349 | | Billing, limits, Stripe, keeper | On | Off | Not planned | | |
| 350 | | sudo (staff console) | Access | Off | Phase 4: `G1T_ADMINS` | | |
| 351 | | Git over SSH | Not yet | Off | Phase 3 (`crates/sshd`, which is native already) | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 352 | | REST API, OAuth, MCP | On | On, on `API_PORT` | — | |
| 353 | | CLI | On | Off | Phase 2 | | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 354 | |
| 355 | ### Auth, Access and email | |
| 356 | ||
| 357 | - `apps/sudo` checks Cloudflare Access. Self-hosted, it should check a | |
| 358 | normal g1t session against `G1T_ADMINS`. That needs an `AdminAuth` port in | |
| 359 | `workers/app.ts` with two adapters. It is phase 4, because sudo is about | |
| 360 | billing. | |
| 361 | - Sessions are random tokens stored hashed in D1, with no signing key, so | |
| 362 | nothing to configure. The cookie is `Secure`: fine on `localhost`, but any | |
| 363 | other address needs HTTPS. The compose file should gain an optional Caddy | |
| 364 | service in phase 2. | |
| 365 | - **Email verification is required** before creating anything, and there is | |
| 366 | no bypass in code. That is why the proof ships a working mail path | |
| 367 | (Mailpit) rather than "email off". An admin "mark verified" or | |
| 368 | `G1T_SKIP_EMAIL_VERIFICATION` belongs with `G1T_ADMINS`. | |
| 369 | ||
| 370 | ### Configuration, upgrades and backups | |
| 371 | ||
| 372 | - **Today:** environment variables in the compose file (`PUBLIC_URL`, | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 373 | `G1T_PORT`, `API_PORT`, `API_URL`, `MCP_URL`, `MAIL_URL`, `MAIL_FROM`, |
| 374 | the S3 store and its buckets). Keys (`ACTIONS_KEY`, | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 375 | `INTEGRATIONS_KEY`, `WEBHOOKS_KEY`, the git store secret) are generated on |
| 376 | first start and kept on volumes. | |
| 377 | - **Phase 2:** one `g1t.toml`, read by the launcher and turned into | |
| 378 | bindings and variables: | |
| 379 | ||
| 380 | ```toml | |
| 381 | public_url = "https://git.example.com" | |
| 382 | ||
| 383 | [mail] | |
| 384 | smtp = "smtp://user:pass@smtp.example.com:587" | |
| 385 | from = "g1t <git@example.com>" | |
| 386 | ||
| 387 | [agents] # off when absent | |
| 388 | docker = "unix:///var/run/docker.sock" | |
| 389 | egress = "enforce" | |
| 390 | ||
| 391 | [context] # off when absent | |
| 392 | embeddings = "http://ollama:11434/v1" | |
| 393 | model = "nomic-embed-text" | |
| 394 | ||
| 395 | [admins] | |
| 396 | usernames = ["alice"] | |
| 397 | ``` | |
| 398 | ||
| 399 | - **Upgrades.** The container applies every service's D1 migrations to its | |
| 400 | SQLite file on start (`wrangler d1 migrations apply --local`). Applied | |
| 401 | migrations are recorded in `d1_migrations`, so this is idempotent. Hosted | |
| 402 | and self-hosted run the same migration files, which keeps them | |
| 403 | forward-compatible. Rule to keep: migrations stay additive, or come with | |
| 404 | a backfill a self-hoster's start can run. | |
| 405 | - **Backups.** Phase 1: stop, then tar the `g1t-data` and `g1t-git` | |
| 406 | volumes (documented in the guide). Phase 2: online backups with | |
| Merge branch 'worktree-agent-ac5b181a013e54348' | 407 | `sqlite3 .backup` per database, or Litestream for continuous |
| 408 | replication. The repositories get hosted g1t's nightly bundles | |
| 409 | (docs/ARTIFACTS.md, R11) once the runner runs: the storage is already | |
| 410 | configured (`BACKUP_STORE=s3`, the `g1t-backups` bucket that | |
| Docs: the self-hosted object store is RustFS | 411 | `storage-setup` makes, `BACKUP_S3_BUCKET` to choose another), and the |
| Merge branch 'worktree-agent-ac5b181a013e54348' | 412 | restore drill reads a copy of that bucket |
| Docs: the self-hosted object store is RustFS | 413 | (`aws --endpoint-url <S3_ENDPOINT> s3 sync s3://g1t-backups ./copy`, then |
| Merge branch 'worktree-agent-ac5b181a013e54348' | 414 | `node scripts/ops/backup-restore-drill.mjs --bundles ./copy --repo-id <id> --live <bare repository>`). |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 415 | |
| 416 | ## 3. Phase 1: what works today | |
| 417 | ||
| 418 | Everything is in `deploy/self-host/`: | |
| 419 | ||
| 420 | | File | What it is | | |
| 421 | | --- | --- | | |
| Docs: the self-hosted object store is RustFS | 422 | | `docker-compose.yml` | `g1t` (every core Worker in one workerd on 8787, and the API in a second on 8789), `status`, `gitstore` (bare repositories), `rustfs` (S3-compatible storage for packages, backups and clone packs; `RUSTFS_IMAGE`, default `rustfs/rustfs:1.0.1`), `storage-setup` (the AWS CLI, `AWS_CLI_IMAGE`, default `amazon/aws-cli:2.37.10`: makes the three buckets and puts the packs' bucket's lifecycle rule, which expires `packs/` after 7 days and aborts multipart uploads unfinished after a day; RustFS's scanner applies it), and `mailpit` (mail). Volumes: `g1t-data`, `g1t-git`, `g1t-objects`, `g1t-status`, `g1t-secrets`. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 423 | | `Dockerfile` | Compiles the ten Rust services to WebAssembly with `worker-build`, as hosted does. Builds the site with React Router. The runtime image has Node, Wrangler, workerd and the built Workers. | |
| 424 | | `Dockerfile.dockerignore` | Build-context rules for this image only (the root `.dockerignore` leaves out the site). | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 425 | | `start.sh` | Makes the sealing keys once, writes the configs, applies migrations, runs `wrangler dev` with every config on `0.0.0.0:8787`, persisting to `/data/state`, and the API's `wrangler dev` on `0.0.0.0:8789` once the first answers. Starts `scheduler.mjs`. | |
| 426 | | `scheduler.mjs` | The cron ticker (see Cron Triggers above). | | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 427 | | `configs.mjs` | Derives each self-hosted Wrangler config from the hosted `wrangler.jsonc`. It drops routes, account and placement, rebinds `ARTIFACTS`/`EMAIL` and the off services, and rewrites hosted URLs. Derived, so it cannot drift. | |
| 428 | | `gitstore/server.mjs`, `gitstore/Dockerfile` | The git store. | | |
| 429 | | `workers/artifacts/index.js` | The `ARTIFACTS` binding, implemented against the git store. | | |
| 430 | | `workers/mail/index.js` | The `EMAIL` binding: logs, then sends to Mailpit. | | |
| 431 | | `workers/off/index.js` | The runner and the context hub when they are off. | | |
| Docs: the self-hosted object store is RustFS | 432 | | `smoke.sh` | The end-to-end check, including the clone pack cache, the API, an npm package published and installed, pull requests, the merge queue and (with `SCHEDULER_ONCE`) every cron handler. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 433 | |
| 434 | Workers running: the site; identity, repos, work, events, projects, search, | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 435 | billing, security, actions, webhooks, integrations, packages and |
| 436 | deployments; the artifacts, mail and two off stand-ins; and, in a second | |
| 437 | workerd, the API. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 438 | |
| 439 | ### Verified | |
| 440 | ||
| 441 | On this machine (Windows 11, Docker Desktop 29.8, engine on Linux): | |
| 442 | ||
| 443 | 1. **Without Docker, with local processes.** The git store ran under Node | |
| 444 | on Windows and every Worker ran under `wrangler dev`, using the configs | |
| 445 | from `configs.mjs` and the existing Rust builds. `smoke.sh` passed every | |
| 446 | step: sign-up, confirmation through the logged link, workspace, repo, | |
| 447 | `git push` and `git clone` over HTTP, issue, and the code, blob and | |
| 448 | commits pages. Seventeen other pages answered 200: home, workspace, | |
| 449 | repo overview, issues, pulls, settings, agents, actions, deployments, | |
| 450 | security, people, usage, explore, search, account settings and tree. | |
| 451 | The workspace context page answered 403 from the off stand-in, as | |
| 452 | intended. | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 453 | 2. **With Docker Compose** (2026-10-07, `docker compose up --build`, with |
| Docs: the self-hosted object store is RustFS | 454 | `API_PORT=18789` because 8789 was taken on this machine; the store is |
| 455 | RustFS 1.0.1, its buckets made by `storage-setup`). `smoke.sh` | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 456 | passed every step: the ones above; a second clone answered from the pack |
| Docs: the self-hosted object store is RustFS | 457 | cache (`Server-Timing: pack;desc=hit`), with the packs in RustFS's |
| 458 | `g1t-git-packs` and its lifecycle rule in place (packs expire after 7 | |
| 459 | days, unfinished uploads are aborted after 1); an access token made in the | |
| 460 | site; an npm package published to the installation's registry and | |
| 461 | installed back, its tarball in `g1t-packages`; `GET /user`, the API index (`mcp_url`, `git_url`), the OAuth | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 462 | metadata (`issuer` the API's address, `authorization_endpoint` on the |
| 463 | site), MCP's 401 challenge and `tools/list`; a pull request from a branch | |
| 464 | and one from a fork (`create_pull_request` without a branch: a fork in | |
| 465 | the git store, pushed to with the token, marked ready) merged onto | |
| 466 | `main`; the merge queue turned on, a pull request merged into it and | |
| 467 | shown `waiting`, taken out (`unqueue`), the queue turned off and the | |
| 468 | pull request merged; and `scheduler.mjs --once`, every handler `ok`. | |
| 469 | The repository page's clone box and MCP line named the installation's | |
| Docs: the self-hosted object store is RustFS | 470 | own addresses, with no social card tags. By hand against the same |
| 471 | stack: `docker push` and `docker pull` of an image with a 20 MB layer | |
| 472 | (uploaded in 10 MiB parts), the layer read back from `/v2/.../blobs/` | |
| 473 | with a matching digest, the `g1t-backups` bucket present (empty: the | |
| 474 | runner cuts bundles), no unfinished upload in any bucket, and the | |
| 475 | guide's upgrade copy from an old MinIO volume into RustFS. | |
| 476 | 3. **The clone pack cache against RustFS without the stack.** | |
| 477 | `node services/repos/dev/clone-check.mjs --s3` (RustFS in Docker, the | |
| 478 | bucket and lifecycle rule made with the AWS CLI): misses then hits for | |
| 479 | full and shallow clones over protocol v2 and v0, a miss after the refs | |
| 480 | version moves, five whole packs in the bucket (12 MB each, multipart | |
| 481 | objects of 3 parts), each reading back at its listed size, no | |
| 482 | unfinished upload, and both lifecycle rules. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 483 | |
| 484 | ### Not verified, or not working yet | |
| 485 | ||
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 486 | - The merge queue past `waiting`, and catch-up (bringing a pull request |
| 487 | up to date before it lands): both need a sandbox, and the runner is off. | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 488 | - Actions schedules (`on: schedule`), and billing's and deployments' crons. |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 489 | - The CLI, and git over SSH. |
| 490 | - An OAuth sign-in from start to finish (the metadata and issuer are | |
| 491 | checked, the consent flow is not). | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 492 | - Anything on an address other than `localhost` without HTTPS (the |
| 493 | session cookie is `Secure`). | |
| 494 | - Restart durability beyond one restart, upgrades across schema changes, | |
| 495 | and backup and restore. | |
| 496 | ||
| 497 | ## 4. Phased plan | |
| 498 | ||
| 499 | Estimates are for one engineer who knows the codebase, working with | |
| 500 | agents. They include docs and tests. | |
| 501 | ||
| 502 | | Phase | Scope | Estimate | | |
| 503 | | --- | --- | --- | | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 504 | | **1. Core forge** | **Done:** compose stack, git store, Artifacts/Email shims, off stand-ins, config generator, smoke test, guide; the API (REST, MCP, OAuth) on its own port with a `PUBLIC_URL`-derived issuer; `PUBLIC_URL`-derived settings in identity mail, the site's meta tags, clone box and agent setup, and the API; the cron ticker; the clone pack cache on S3; pull requests from branches and forks and the merge queue's enqueue and removal in `smoke.sh`. **Left:** optional Caddy for HTTPS; a CI job that builds the images and runs `smoke.sh`; the remaining hard-coded addresses listed above. | 2–3 days left | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 505 | | **2. Agents with Docker sandboxes** | Test Wrangler's local Containers first. Otherwise: `g1t-sandboxd` supervisor, `DockerSandbox` adapter in the runner, egress allow-list proxy and internal network, runner addresses from `PUBLIC_URL`, models through a workspace's own provider, `g1t.toml` and a launcher that replaces `wrangler dev`. | 2–3 weeks | |
| 506 | | **3. Search, context and deployments** | `Embedder` (OpenAI-compatible) and `VectorIndex` (sqlite-vec first) ports in context; app host on workerd with Worker Loader; Caddy on-demand TLS for app and custom domains; git over SSH through `crates/sshd` plus the missing `/_internal/ssh/*` endpoints. | 3–4 weeks | | |
| 507 | | **4. Parity and upgrade path** | Code-level ports in `g1t_kit` / `@g1t/platform` replacing the binding shims (`LocalGitStore` in Rust, `Mailer` with SMTP); `AdminAuth` for sudo; online backups (Litestream or `.backup`); versioned releases with published images; an upgrade test in CI that migrates a snapshot of the previous release; a self-host column in the docs for every feature. | 3–4 weeks | | |
| 508 | ||
| 509 | Total to parity: about 10–13 weeks. Phase 1 alone is already a credible | |
| 510 | "run it yourself" for the core forge. | |
| 511 | ||
| 512 | ## 5. Risks | |
| 513 | ||
| 514 | - **`wrangler dev` is a development tool.** It exposes Miniflare's dev | |
| 515 | endpoints (`/cdn-cgi/...`, including a local data explorer) on the same | |
| 516 | port as the site. Treat phase 1 as **localhost or a trusted private | |
| 517 | network only** until the launcher in phase 2 replaces it. Its flags and | |
| 518 | behaviour can also change between Wrangler releases. Pin the Wrangler | |
| 519 | version, as the lockfile already does. | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 520 | - **The API reaches the other Workers through Wrangler's dev registry.** |
| 521 | Two `wrangler dev` processes in one container find each other through | |
| 522 | a registry directory, a development feature like the rest. If the API | |
| 523 | starts and a binding says `[not connected]`, restart the container. The | |
| 524 | phase 2 launcher serves both from one workerd. | |
| Docs: the self-hosted object store is RustFS | 525 | - **The object store.** The compose file runs RustFS (Apache-2.0), |
| 526 | pinned to a release tag, and makes its buckets with the AWS CLI | |
| 527 | (Apache-2.0). Any S3-compatible store can take its place | |
| 528 | (`S3_ENDPOINT`), given the same buckets and the packs' lifecycle rule. | |
| 529 | Installations started before 2026-10-07 kept these files in MinIO, in | |
| 530 | the `g1t-packages` volume; the guide's "Upgrade" section copies them | |
| 531 | across. | |
| Packages, with a container registry on g1t.sh; workspaces deleted whole and kept 30 days; Members for every member | 532 | - **Cron goes through Wrangler's local API.** `scheduler.mjs` asks |
| 533 | `/cdn-cgi/local/explorer/api/local/scheduled`, a development endpoint | |
| 534 | that may change between Wrangler releases (pinned by the lockfile). | |
| 535 | Actions schedules, billing and deployments are not run. | |
| Running g1t yourself: the design, a docker compose proof, and a guide to what works today | 536 | - **New Cloudflare-only bindings break self-hosting silently.** |
| 537 | `configs.mjs` passes unknown keys through untouched. A new binding type | |
| 538 | could make `wrangler dev` reach for a remote resource (for example | |
| 539 | `remote: true`, AI or Vectorize) and prompt for a login. Mitigation: CI | |
| 540 | that builds the compose stack and runs `smoke.sh` on every change, and an | |
| 541 | allow-list in `configs.mjs` that fails on unknown binding types. | |
| 542 | - **Artifacts semantics drift.** The shim copies the Artifacts methods g1t | |
| 543 | uses today. If repos starts using another method (`import`, | |
| 544 | `listTokens`, `revokeToken`), self-hosted fails at runtime. Mitigation: a | |
| 545 | contract test that runs `services/repos` against the git store, and the | |
| 546 | long-term `LocalGitStore` port. | |
| 547 | - **Error codes over RPC.** `ArtifactsStore::create` and `fork` tolerate | |
| 548 | `ALREADY_EXISTS` by reading the thrown error's `code`. Workers RPC may not | |
| 549 | carry custom error properties across a service binding. If it does not, | |
| 550 | retrying a half-finished create fails self-hosted where it would succeed | |
| 551 | hosted. This was not seen in testing, because creates were never retried. | |
| 552 | - **Single node, single writer.** SQLite (D1 local) and one workerd | |
| 553 | process suit a team, not a large organisation. Scaling out means | |
| 554 | Postgres behind a `Database` port, which is a large change and is not | |
| 555 | planned. | |
| 556 | - **The `Secure` cookie.** It needs HTTPS anywhere but `localhost`. A LAN | |
| 557 | install over plain HTTP cannot sign in. | |
| 558 | - **Building from a working tree that is mid-change.** The image compiles | |
| 559 | every Rust service, including ones other work is changing. A service that | |
| 560 | does not compile breaks the whole image. Released images (phase 4) fix | |
| 561 | this. | |
| 562 | - **Image size and build time.** The first build compiles ten Rust crates | |
| 563 | to WebAssembly and installs the site's dependencies. Expect minutes and | |
| 564 | several GB. Published images remove this for users. | |
| Merge branch 'worktree-agent-aaf03bdceac799c89' | 565 | - **Hard-coded hosted URLs.** The few left (listed above) point at the |
| 566 | hosted service until they read a setting: the docs link, invite and | |
| 567 | runner defaults, and the runner's remotes. |
This file's history is long; its oldest lines are credited to the oldest commit read.