Sign in

flagon-io/g1t

public

Commit

Docs: the self-hosted object store is RustFS

The self-hosting guide names RUSTFS_IMAGE and AWS_CLI_IMAGE, the g1t-objects volume, the npm step of smoke.sh, and how an installation started before 2026-10-07 copies its packages and backups out of the old MinIO volume (tried against a copy of one). docs/SELF_HOSTING.md records what was run against RustFS 1.0.1: clone-check --s3, smoke.sh end to end, a container push and pull with a 20 MB layer. Comments in the blob store, packages and repos services, ARTIFACTS.md, PACKAGES.md and the restore drill follow.

syntaqxcommitted Parent666a910Browse files
12 files+94−500/12 viewed
+3−2
180180 Then open http://localhost:8787 and sign up. The confirmation mail is in
181181 Mailpit at http://localhost:8025. Repositories, push and clone, issues,
182182 pull requests and code browsing work, and the API and MCP server answer at
183−http://localhost:8789; agents, deployments and context search are off in
184−this version.
183+http://localhost:8789; packages and container images are kept in the
184+bundled S3-compatible store, RustFS. Agents, deployments and context search
185+are off in this version.
185186 [docs/SELF_HOSTING.md](docs/SELF_HOSTING.md) says what works and what is next.
186187
187188 ### On Cloudflare
+37−9
1717 | --- | --- |
1818 | Sign up, sign in, email confirmation | Works. Mail goes to the bundled Mailpit inbox. |
1919 | Workspaces, members, access tokens | Works |
20−| Repositories: create, push and clone over HTTP, browse code, commits | Works. A fresh clone's pack is kept in the bundled MinIO, so the next clone of the same commit is served from there. |
20+| Repositories: create, push and clone over HTTP, browse code, commits | Works. A fresh clone's pack is kept in the bundled object store, so the next clone of the same commit is served from there. |
2121 | Issues, comments, labels | Works |
2222 | Pull requests from a branch or from a fork, merged onto the default branch | Works, when the pull request is up to date with the default branch. Bringing one up to date first needs g1t's agent, which is off. |
2323 | The merge queue | Takes pull requests and shows them waiting. Testing and landing them needs g1t's agent, which is off: take a pull request out of the queue, or turn the queue off, to merge it. |
2424 | [The REST API](/reference/api/), OAuth and [MCP](/reference/mcp/) | Work, on a port of their own: `http://localhost:8789`, with the MCP server at `http://localhost:8789/mcp` |
25−| [Container images](/guides/containers/): `docker login`, push and pull at your `PUBLIC_URL` | Works, kept in the bundled MinIO, with no limit on a layer's size or on pulls |
25+| [Container images](/guides/containers/): `docker login`, push and pull at your `PUBLIC_URL` | Works, kept in the bundled object store, with no limit on a layer's size or on pulls |
2626 | Site search | Works |
2727 | A status page of your own | Works, at `http://localhost:8788` ([below](#the-status-page)) |
2828 | Webhooks, integrations | Work, retries included |
116116 the site.
117117 2. It makes an access token and calls the API, the OAuth metadata and the
118118 MCP server with it.
119−3. It opens a pull request from a branch and one from a fork, through the
119+3. It publishes an npm package to your installation's registry and
120+ installs it back.
121+4. It opens a pull request from a branch and one from a fork, through the
120122 API, and merges both onto `main`.
121−4. It turns the merge queue on, merges a pull request into it, takes it
123+5. It turns the merge queue on, merges a pull request into it, takes it
122124 out again, and merges it with the queue off.
123125
124126 ```sh
135137 ```
136138
137139 It prints `All checks passed` when every step worked. It needs `curl`,
138−`git` and `node`.
140+`git`, `node` and `npm`; `PACKAGES=off` skips the npm package.
139141
140142 ## Settings
141143
155157 | `REGISTRATION_MODE` | `open` | `open`: anyone can make an account. `invite`: every new account needs an [invite](/guides/authentication/#invites), as on g1t.sh. |
156158 | `INVITES_PER_USER` | `5` | How many invites each person can have out, while `REGISTRATION_MODE` is `invite` |
157159 | `WAITLIST_NOTIFY_EMAIL` | (none) | Where a summary of new access requests goes, at most every 15 minutes. Empty sends none; requests still wait for you in the database. |
158−| `S3_ENDPOINT`, `S3_BUCKET`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` | the bundled MinIO, bucket `g1t-packages` | Where packages' files are kept: any S3-compatible store. Change the two keys before first start; MinIO is made with them. |
160+| `S3_ENDPOINT`, `S3_BUCKET`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` | the bundled RustFS, bucket `g1t-packages` | Where packages' files are kept: any S3-compatible store. Change the two keys before first start; RustFS is made with them. |
159161 | `S3_PUBLIC_ENDPOINT` | (none) | The store's address as clients reach it. When set, large layers are downloaded from it directly with a signed URL. |
160−| `PACK_S3_BUCKET` | `g1t-git-packs` | The bucket on the same store that packs for fresh clones are kept in, so the next clone of the same commit is not built again. The bundled MinIO deletes packs after 7 days; on another store, give the bucket a rule that expires objects under `packs/` and unfinished multipart uploads. |
161−| `MINIO_IMAGE` | `pgsty/minio:latest` | The MinIO server image the bundled store runs. MinIO no longer publishes its own images; this is a community build of the same server. |
162+| `PACK_S3_BUCKET` | `g1t-git-packs` | The bucket on the same store that packs for fresh clones are kept in, so the next clone of the same commit is not built again. The bundled store deletes packs after 7 days, and uploads left unfinished after a day; on another store, give the bucket a lifecycle rule that does the same. |
163+| `RUSTFS_IMAGE` | `rustfs/rustfs:1.0.1` | The image the bundled object store runs: [RustFS](https://rustfs.com), an S3-compatible server. |
164+| `AWS_CLI_IMAGE` | `amazon/aws-cli:2.37.10` | The image `storage-setup` makes the buckets and the packs' lifecycle rule with. |
162165 | `BACKUP_S3_BUCKET` | `g1t-backups` | The bucket on the same store that nightly repository backups (a `git bundle` of each repository whose branches or tags changed) are kept in. The bundles are cut by g1t's runner, which this installation does not run yet, so the bucket stays empty for now: copy the volumes, as below. |
163166 | `STATUS_PORT` | `8788` | The port the status page is published on |
164167 | `STATUS_PROBE_REPO` | (none) | A public repository, `workspace/repo`, whose branches the status page lists every minute as a clone would. Empty: git is not checked. |
238241 | --- | --- |
239242 | `g1t_g1t-data` | Accounts, workspaces, issues and every other record, as SQLite files; the keys that seal stored secrets (`keys.env`) |
240243 | `g1t_g1t-git` | Your repositories, one bare git repository each |
241−| `g1t_g1t-packages` | Container images' layers and other package files, the `g1t-backups` bucket and the clone packs in `g1t-git-packs` (MinIO) |
244+| `g1t_g1t-objects` | The bundled object store (RustFS): container images' layers and other package files in `g1t-packages`, the `g1t-backups` bucket and the clone packs in `g1t-git-packs` |
242245 | `g1t_g1t-secrets` | The key the site and the git store share |
243246
244247 To back up, stop g1t and copy the volumes:
263266 docker compose -f deploy/self-host/docker-compose.yml up --build -d
264267 ```
265268
269+### Installations started before 7 October 2026
270+
271+These kept packages' files and backups in MinIO, in the `g1t_g1t-packages`
272+volume. The bundled store is now RustFS, in `g1t_g1t-objects`, and starts
273+empty. After the upgrade above, copy the old objects across (use your own
274+`S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY` if you changed them):
275+
276+```sh
277+docker run -d --name g1t-old-store --network g1t_default \
278+ -v g1t_g1t-packages:/data -e MINIO_ROOT_USER=g1t \
279+ -e MINIO_ROOT_PASSWORD=g1t-packages-secret pgsty/minio server /data
280+docker run --rm --network g1t_default -e AWS_ACCESS_KEY_ID=g1t \
281+ -e AWS_SECRET_ACCESS_KEY=g1t-packages-secret -e AWS_DEFAULT_REGION=us-east-1 \
282+ --entrypoint sh amazon/aws-cli:2.37.10 -c '
283+ for b in g1t-packages g1t-backups; do
284+ aws --endpoint-url http://g1t-old-store:9000 s3 sync "s3://$b" "/tmp/$b" &&
285+ aws --endpoint-url http://rustfs:9000 s3 sync "/tmp/$b" "s3://$b"
286+ done'
287+docker rm -f g1t-old-store
288+```
289+
290+The clone packs are not copied: they are a cache, and are made again on
291+the next clone. Once your images and packages pull, remove the old volume
292+with `docker volume rm g1t_g1t-packages`.
293+
266294 ## Stop and remove
267295
268296 ```sh
+1−1
33 version = "0.1.0"
44 edition.workspace = true
55 license.workspace = true
6−description = "Object storage behind one port: R2 on Cloudflare, any S3-compatible store (MinIO) when self-hosted."
6+description = "Object storage behind one port: R2 on Cloudflare, any S3-compatible store (RustFS) when self-hosted."
77
88 [dependencies]
99 g1t-contracts.workspace = true
+1−1
11 //! Object storage behind one port, `BlobStore`: R2 on Cloudflare, and any
2−//! S3-compatible store (MinIO in the self-host compose file) elsewhere.
2+//! S3-compatible store (RustFS in the self-host compose file) elsewhere.
33 //!
44 //! Every service that keeps objects names its own [`Config`]: the variable
55 //! that chooses the store (`r2`, the default, or `s3`), the R2 bucket
+3−3
11 //! The S3 adapter, for self-hosted installations: any S3-compatible store
2−//! (MinIO, Ceph, Garage, AWS) over fetch, signed with SigV4, path-style.
2+//! (RustFS, Ceph, Garage, AWS) over fetch, signed with SigV4, path-style.
33 //! S3_ENDPOINT, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY and S3_REGION say
44 //! where, and the service's own variable (`Config::s3_bucket`) which
55 //! bucket. Its public endpoint variable, when it names one and that is
1313 use crate::{BlobStore, Config, Got, Part, Wanted, var};
1414
1515 pub struct S3Store {
16− /// `http://minio:9000`, without a trailing slash.
16+ /// `http://rustfs:9000`, without a trailing slash.
1717 endpoint: String,
1818 /// Where clients reach the same store, for signed URLs.
1919 public_endpoint: Option<String>,
249249 let xml = "<InitiateMultipartUploadResult><Bucket>b</Bucket><UploadId>abc-123</UploadId></InitiateMultipartUploadResult>";
250250 assert_eq!(xml_value(xml, "UploadId"), Some("abc-123"));
251251 assert_eq!(xml_value(xml, "Key"), None);
252− assert_eq!(host_of("http://minio:9000"), "minio:9000");
252+ assert_eq!(host_of("http://rustfs:9000"), "rustfs:9000");
253253 assert_eq!(host_of("https://s3.example.com/base"), "s3.example.com");
254254 }
255255 }
+3−3
367367 | R10 | Built in repos; work unchanged | `divergence` works out the target's side once per target head per isolate (`coalesce.rs`: the head under the refs version, then the history by hash, kept 60 s), and what the target changed between two trees once per pair (10 minutes). `readCommit` and logs by hash come from the cache. Work's fan-out (`after_push`, up to 100 pull requests) is unchanged: its 100 `divergence` calls now cost one walk of the target instead of 100. |
368368 | R7 | Groundwork | `shards.rs`: bindings named in `ARTIFACTS_NAMESPACES` (JSON, binding → namespace; `ARTIFACTS` → `g1t` always there), a repository's namespace kept in its `store` column as `<namespace>/<key>` (no prefix means the `ARTIFACTS` namespace, so every existing key reads the same), new repositories placed by `ARTIFACTS_NEW_REPOS` (comma-separated, spread by an FNV hash of the repository id; names not bound are skipped), forks always in their repository's namespace, `ARTIFACTS_EU_NAMESPACE` reserved for EU residency (no workspace setting yet). Works with only `ARTIFACTS` bound, as today. |
369369 | R8 | Built | `crates/runner/src/clone.rs`: every sandbox clones at `--depth=1` (a full g1t clone took 5.4 s, depth 1 took 3.8 s). Work that merges (catch-up, the merge queue, merge checks, a review's diff) deepens 50, 500, then 5000 commits until the two sides share one, and fetches everything only as the last resort (`share_history`). `G1T_CLONE_DEPTH` (0 or `full` for everything) and `G1T_CLONE_FILTER=blob:none` change it per runner. |
370−| R6 | Built; the bucket must exist before it deploys | `pack_cache.rs`: an upload-pack POST with wants and no `have` or `shallow` lines (a fresh clone, the sandboxes' `deepen 1` ones included), uncompressed and at most 1 MiB, is keyed `packs/<repo id>/<refs_version>/<sha256>` over the request normalized: protocol v2 capabilities without `agent=`/`session-id=` and its arguments, each sorted and deduplicated; v0/v1 wants sorted, the first want's capabilities split off, sorted and without `agent=`, then `deepen`/`filter` lines, a flush and `done`. Only while `refs_cache::usable` (the version known, and no push credential out of g1t's hands), so never across a refs change. Looked up after authorization, alongside the free-workspace limits and the kept refs answer; a hit streams from the bucket (`Server-Timing` `pack;desc=hit`). A miss streams the store's 200 to git through a tee that copies it to a fill in `ctx.wait_until` (at most 5 MiB queued between them, 2 fills per isolate, one per key): under 5 MiB it is one `put` once it all arrived; larger, 5 MiB multipart parts completed only after the last part and a check that it is one whole side-band pack (well-formed pkt-lines, `PACK` on channel 1, no `ERR` or channel 3, a closing flush). Over 200 MB, a queue that falls behind, git going away or the store's stream failing lets the fill go and aborts the upload; nothing partial can be read. Meters `pack_cache.hit` (with the bytes served) and `pack_cache.miss` (counted with `record`, bytes added at the end), neither an operation by default; a hit records no `git.fetch`. Storage is behind the `PackStore` port, whose adapter puts the shared `BlobStore` (`crates/blobstore`) behind it: R2 hosted (`GIT_PACKS`, bucket `g1t-git-packs`, lifecycle: packs deleted after 7 days, unfinished uploads after 1), or with `PACK_STORE=s3` the S3 bucket `PACK_S3_BUCKET` names (self-hosted: MinIO's `g1t-git-packs`, packs deleted after 7 days, unfinished uploads by MinIO after 24 hours; `node services/repos/dev/clone-check.mjs --s3` checks it). With neither, nothing is kept. |
370+| R6 | Built; the bucket must exist before it deploys | `pack_cache.rs`: an upload-pack POST with wants and no `have` or `shallow` lines (a fresh clone, the sandboxes' `deepen 1` ones included), uncompressed and at most 1 MiB, is keyed `packs/<repo id>/<refs_version>/<sha256>` over the request normalized: protocol v2 capabilities without `agent=`/`session-id=` and its arguments, each sorted and deduplicated; v0/v1 wants sorted, the first want's capabilities split off, sorted and without `agent=`, then `deepen`/`filter` lines, a flush and `done`. Only while `refs_cache::usable` (the version known, and no push credential out of g1t's hands), so never across a refs change. Looked up after authorization, alongside the free-workspace limits and the kept refs answer; a hit streams from the bucket (`Server-Timing` `pack;desc=hit`). A miss streams the store's 200 to git through a tee that copies it to a fill in `ctx.wait_until` (at most 5 MiB queued between them, 2 fills per isolate, one per key): under 5 MiB it is one `put` once it all arrived; larger, 5 MiB multipart parts completed only after the last part and a check that it is one whole side-band pack (well-formed pkt-lines, `PACK` on channel 1, no `ERR` or channel 3, a closing flush). Over 200 MB, a queue that falls behind, git going away or the store's stream failing lets the fill go and aborts the upload; nothing partial can be read. Meters `pack_cache.hit` (with the bytes served) and `pack_cache.miss` (counted with `record`, bytes added at the end), neither an operation by default; a hit records no `git.fetch`. Storage is behind the `PackStore` port, whose adapter puts the shared `BlobStore` (`crates/blobstore`) behind it: R2 hosted (`GIT_PACKS`, bucket `g1t-git-packs`, lifecycle: packs deleted after 7 days, unfinished uploads after 1), or with `PACK_STORE=s3` the S3 bucket `PACK_S3_BUCKET` names (self-hosted: RustFS's `g1t-git-packs`, with the same lifecycle rule, put by the compose file's `storage-setup`; `node services/repos/dev/clone-check.mjs --s3` checks it). With neither, nothing is kept. |
371371 | R11 | Built; not yet deployed | Nightly `git bundle` backups to the `g1t-backups` R2 bucket, and a restore drill. Migration `0013_backups.sql` (`repo_backups`, and an `operation_mapping` row). See "R11: backups and the restore drill" below. |
372372
373373 ### R1: reading `scripts/ops/artifacts-usage.mjs`
540540
541541 Storage, through the `BlobStore` port in `crates/blobstore` (the adapters packages already used):
542542 the `BACKUPS` binding (bucket `g1t-backups`) with `BACKUP_STORE=r2`; any S3-compatible store with
543−`BACKUP_STORE=s3` and `BACKUP_S3_BUCKET` (self-hosted: MinIO). Without either, backups are off and
543+`BACKUP_STORE=s3` and `BACKUP_S3_BUCKET` (self-hosted: RustFS). Without either, backups are off and
544544 the nightly cron does nothing.
545545
546546 ```text
582582 at random, the repository is one whose refs have not moved since its last backup, so any
583583 difference is the backup's. Run it after the first night, then monthly, and after any change to
584584 `backups.rs` or `backup.rs`. `--bundles <dir>` reads a local copy of the bucket instead
585−(self-hosted: `mc mirror local/g1t-backups <dir>`), with `--repo-id` and `--live <url or path>`.
585+(self-hosted: `aws --endpoint-url <S3_ENDPOINT> s3 sync s3://g1t-backups <dir>`), with `--repo-id` and `--live <url or path>`.
586586 `npm run test:ops` runs it against bundles cut with git.
587587
588588 **A real restore into the store**, as it can be done today:
+1−1
3636 `packages/contracts`. Other services talk to it over RPC and hear from it through events.
3737 - **Cloudflare in production, anything in a self-hosted install.** Files go through a
3838 `BlobStore` port. In production its adapter is R2; self-hosted it is S3-compatible storage
39− (MinIO in the compose file) or a local directory. Upload URLs (`presign`) are part of the port:
39+ (RustFS in the compose file) or a local directory. Upload URLs (`presign`) are part of the port:
4040 R2 and S3 sign them, the disk adapter answers with a path back through the service.
4141 Metadata is D1, which self-hosting already runs (workerd's D1 over SQLite).
4242 - **Content-addressed.** Every file is stored once by its SHA-256 (`blobs/sha256/<hex>`). A
+40−25
4040 answers "agents are off" instead of failing.
4141 - **Proven on this machine with Docker:** sign up, confirm the email
4242 through Mailpit, create a workspace and a repository, push and clone over
43− HTTP (the second clone from the clone pack cache in MinIO), open an
43+ HTTP (the second clone from the clone pack cache in RustFS), open an
4444 issue, and browse code, commits and files in the site; then the REST API,
45− OAuth metadata and MCP on their own port, pull requests from a branch and
46− from a fork merged onto `main`, the merge queue taking a pull request and
45+ OAuth metadata and MCP on their own port, an npm package published and
46+ installed and a container image pushed and pulled through RustFS, pull
47+ requests from a branch and from a fork merged onto `main`, the merge queue taking a pull request and
4748 giving it back, and every cron handler the scheduler runs. All of it runs
4849 against local storage. See [Phase 1: what works today](#3-phase-1-what-works-today).
4950 - **Long term:** keep workerd as the runtime, because it is what hosted
6869 | **Workers runtime**, service bindings | Every service. Rust through `worker` 0.8 (`#[event(fetch\|queue\|scheduled)]`, `Env`, `Fetcher`); TS as `export default { fetch, queue, scheduled }` | woven (as a runtime), thin (as an API) | **workerd**: the same runtime, open source. Service bindings work as they do hosted. Calls are HTTP (`POST /rpc/<method>`), so a native port could use plain HTTP clients. |
6970 | **Workers RPC** (JS methods across a binding) | Only `RUNNER`: `RunnerService extends WorkerEntrypoint` (`services/runner/src/index.ts:626`). `apps/web` calls `env.RUNNER.enabled/run/plan/...` directly in 11 routes. | thin | workerd supports it. A native port needs these on `/rpc/*` as well; the runner already has a `fetch` shim for Rust callers. |
7071 | **D1** | System of record for 13 services. Rust: `env.d1("DB")`; TS: `D1Database`; `db.batch()` in `crates/kit` `rename` | woven (SQL), thin (API) | **SQLite files**. workerd/Miniflare implements D1 on SQLite, and the same `migrations/` apply with `wrangler d1 migrations apply --local`. A native port would need a `Database` port over `rusqlite`/`better-sqlite3`; the SQL is already SQLite, including FTS5. |
71−| **KV** | `BLOBS`: Actions artifacts and cache (`apps/api/src/blobs.rs`, `apps/web/app/lib/artifacts.server.ts`). `AVATARS`: `services/identity/src/avatars.rs`, `apps/web/workers/app.ts`, `services/og`. `DOMAINS`: `services/deployments/src/domains.ts`, `services/pages` | thin | Miniflare KV on disk (SQLite plus blob files). Natively: a `BlobStore` port on the filesystem or S3/MinIO. |
72+| **KV** | `BLOBS`: Actions artifacts and cache (`apps/api/src/blobs.rs`, `apps/web/app/lib/artifacts.server.ts`). `AVATARS`: `services/identity/src/avatars.rs`, `apps/web/workers/app.ts`, `services/og`. `DOMAINS`: `services/deployments/src/domains.ts`, `services/pages` | thin | Miniflare KV on disk (SQLite plus blob files). Natively: a `BlobStore` port on the filesystem or S3. |
7273 | **Queues**: the event bus | Producer: `services/events` `BUS.sendBatch` (`lib.rs:67`). The consumer writes the log, then fans out to every binding named `SUBSCRIBER_*` (`lib.rs:161`). Twelve consumers, one queue each. Private job queues in search (`g1t-search-jobs`) and context (`g1t-context-jobs`); consumers branch on the queue name. | woven | Miniflare Queues: in-process and persisted, which works today. Natively: a `Bus` port with a SQLite outbox and a poller per subscriber, or NATS/Redis Streams. At-least-once delivery and idempotent consumers are already the contract. |
7374 | **Durable Objects** | Only `AttemptSandbox` (runner), as the containers library's base class. Uses `ctx.storage.get/put/delete`, `schedule()` (alarm), `idFromName`/`idFromString`, DO RPC (`run`, `destroy`, `noteBlocked`). **Not used:** WebSocket hibernation, raw `alarm()`, `ctx.storage.sql`, `ctx.exports`. | woven, in the runner only | workerd supports Durable Objects (on-disk SQLite). Runner state can move to the sandbox supervisor (phase 2). |
7475 | **Containers** (`@cloudflare/containers`) | `services/runner`: one sandbox per agent run, Actions job and deploy build. `sleepAfter`, `start({ envVars, enableInternet })`, `onStop`. Image: `services/runner/Dockerfile` (node 24, git, toolchains, Claude Code, `g1t-runner`). | woven | **Docker or Podman** through the socket, with the same image. Wrangler can already run Containers locally through Docker; whether that covers outbound interception has to be tested. |
8889 | **Static Assets** | `apps/web` (Vite plugin build), `apps/docs`, `apps/sudo` (`run_worker_first`) | thin | workerd serves them. |
8990 | **`placement`, `observability`, routes, custom domains** | every `wrangler.jsonc` | config only | Dropped by `deploy/self-host/configs.mjs`. |
9091 | **`cf-ray`** | Used as an audit request id, with a fallback: `services/repos/src/run_access.rs:131`, `apps/api/src/audit.rs:37` | thin | Falls back already. |
91−| **R2** | `services/packages` (`BLOBS`: container layers and other package files), `services/repos` (`BACKUPS`: nightly backup bundles; `GIT_PACKS`: the clone pack cache), the API's Actions cache (`ACTIONS_CACHE`), the runner's downloads | thin | **S3-compatible storage**: the `BlobStore` port in `crates/blobstore` has an R2 adapter and an S3 one (`s3.rs`, SigV4 over fetch); each service names its own bucket (`BLOB_STORE`/`S3_BUCKET` for packages, `BACKUP_STORE`/`BACKUP_S3_BUCKET` for backups, `PACK_STORE`/`PACK_S3_BUCKET` for clone packs), run against MinIO in the compose file. |
92+| **R2** | `services/packages` (`BLOBS`: container layers and other package files), `services/repos` (`BACKUPS`: nightly backup bundles; `GIT_PACKS`: the clone pack cache), the API's Actions cache (`ACTIONS_CACHE`), the runner's downloads | thin | **S3-compatible storage**: the `BlobStore` port in `crates/blobstore` has an R2 adapter and an S3 one (`s3.rs`, SigV4 over fetch); each service names its own bucket (`BLOB_STORE`/`S3_BUCKET` for packages, `BACKUP_STORE`/`BACKUP_S3_BUCKET` for backups, `PACK_STORE`/`PACK_S3_BUCKET` for clone packs), run against RustFS in the compose file. |
9293 | **Not used** | Hyperdrive, Workflows, Analytics Engine, Browser Rendering, Images, Turnstile, Secrets Store, `connect()`, HTMLRewriter, `request.cf` | — | — |
9394
9495 ### By service
106107 | `apps/docs` | Static | — | Not run (docs.g1t.sh serves them) |
107108 | `apps/status` | TS Worker | Email Sending, cron; bound only to billing | Runs in a process of its own (`status.sh`), so it stays up when the site does not |
108109 | `services/identity` | Rust | Email Sending, KV `AVATARS` | Runs unchanged; `EMAIL` goes to the mail shim |
109−| `services/repos` | Rust | **Artifacts**, **R2** (`BACKUPS`), Cache API, optional KV `GIT_CACHE` with `REPOS_KEY`, optional R2 `GIT_PACKS` | Runs unchanged; `ARTIFACTS` goes to the git store, backups to MinIO's `g1t-backups` bucket (`BACKUP_STORE=s3`), and the clone pack cache to `g1t-git-packs` (`PACK_STORE=s3`: the `PackStore` port in `src/pack_cache.rs` over the shared `BlobStore`, multipart, an object only once whole; `minio-setup` gives the bucket a rule that deletes packs after 7 days, and MinIO removes unfinished uploads after 24 hours). Without `GIT_CACHE` and `REPOS_KEY`, credentials and ref listings are kept per isolate only. Its nightly cron queues backups, but bundles are cut by the runner, which is off in phase 1: none are made yet |
110+| `services/repos` | Rust | **Artifacts**, **R2** (`BACKUPS`), Cache API, optional KV `GIT_CACHE` with `REPOS_KEY`, optional R2 `GIT_PACKS` | Runs unchanged; `ARTIFACTS` goes to the git store, backups to the `g1t-backups` bucket in RustFS (`BACKUP_STORE=s3`), and the clone pack cache to `g1t-git-packs` (`PACK_STORE=s3`: the `PackStore` port in `src/pack_cache.rs` over the shared `BlobStore`, multipart, an object only once whole; `storage-setup` gives the bucket a lifecycle rule that deletes packs after 7 days and aborts uploads unfinished after a day). Without `GIT_CACHE` and `REPOS_KEY`, credentials and ref listings are kept per isolate only. Its nightly cron queues backups, but bundles are cut by the runner, which is off in phase 1: none are made yet |
110111 | `services/work` | Rust | Queue consumer | Runs unchanged |
111112 | `services/events` | Rust | Queues (producer and fan-out) | Runs unchanged; the off services' queues are not produced to |
112113 | `services/projects` | TS | Queue consumer | Runs unchanged |
116117 | `services/actions` | Rust | Queue, cron, `ACTIONS_KEY` | Runs; jobs need the runner, which is off |
117118 | `services/webhooks` | Rust | Queue, cron, `WEBHOOKS_KEY` | Runs; retries through `scheduler.mjs` |
118119 | `services/integrations` | Rust | Queue, `INTEGRATIONS_KEY` | Runs unchanged |
119−| `services/packages` | Rust | **R2** (`BLOBS`), cron, queue, `PACKAGES_TOKEN_SECRET` | Runs with `BLOB_STORE=s3` against the compose file's MinIO (`deploy/self-host/configs.mjs`); no request size limit (`MAX_REQUEST_BYTES` 0 means none) |
120+| `services/packages` | Rust | **R2** (`BLOBS`), cron, queue, `PACKAGES_TOKEN_SECRET` | Runs with `BLOB_STORE=s3` against the compose file's RustFS (`deploy/self-host/configs.mjs`); no request size limit (`MAX_REQUEST_BYTES` 0 means none) |
120121 | `services/deployments` | TS | Workers for Platforms, REST API, KV `DOMAINS`, cron | Runs with no API token: nothing deploys |
121122 | `services/runner` | TS | **Containers**, Durable Objects, outbound interception, AI Gateway, cron | Off: bound to the off Worker |
122123 | `services/context` | TS | **Vectorize**, **Workers AI**, Queues | Off: bound to the off Worker |
335336 | Issues, pull requests, review | On | On | — |
336337 | Merge queue | On | Takes pull requests; testing and landing them needs sandboxes | Phase 2 |
337338 | Bringing a pull request up to date before it lands (catch-up) | On | Off: needs a sandbox | Phase 2 |
338−| Clone pack cache | R2 | MinIO (`g1t-git-packs`) | — |
339+| Clone pack cache | R2 | RustFS (`g1t-git-packs`) | — |
339340 | Site search (FTS5) | On | On | — |
340341 | Email | Email Sending | Mailpit, logged | SMTP relay |
341342 | Webhooks, integrations | On | On (retries through `scheduler.mjs`) | — |
407408 replication. The repositories get hosted g1t's nightly bundles
408409 (docs/ARTIFACTS.md, R11) once the runner runs: the storage is already
409410 configured (`BACKUP_STORE=s3`, the `g1t-backups` bucket that
410− `minio-setup` makes, `BACKUP_S3_BUCKET` to choose another), and the
411+ `storage-setup` makes, `BACKUP_S3_BUCKET` to choose another), and the
411412 restore drill reads a copy of that bucket
412− (`mc mirror local/g1t-backups ./copy`, then
413+ (`aws --endpoint-url <S3_ENDPOINT> s3 sync s3://g1t-backups ./copy`, then
413414 `node scripts/ops/backup-restore-drill.mjs --bundles ./copy --repo-id <id> --live <bare repository>`).
414415
415416 ## 3. Phase 1: what works today
418419
419420 | File | What it is |
420421 | --- | --- |
421−| `docker-compose.yml` | `g1t` (every core Worker in one workerd on 8787, and the API in a second on 8789), `status`, `gitstore` (bare repositories), `minio` and `minio-setup` (packages, backups and clone packs, with the packs' expiry rule), and `mailpit` (mail). Volumes: `g1t-data`, `g1t-git`, `g1t-packages`, `g1t-status`, `g1t-secrets`. MinIO no longer publishes `minio/minio` or `minio/mc` images; `MINIO_IMAGE` (default `pgsty/minio`, a community build with `mc` in it) is the server. |
422+| `docker-compose.yml` | `g1t` (every core Worker in one workerd on 8787, and the API in a second on 8789), `status`, `gitstore` (bare repositories), `rustfs` (S3-compatible storage for packages, backups and clone packs; `RUSTFS_IMAGE`, default `rustfs/rustfs:1.0.1`), `storage-setup` (the AWS CLI, `AWS_CLI_IMAGE`, default `amazon/aws-cli:2.37.10`: makes the three buckets and puts the packs' bucket's lifecycle rule, which expires `packs/` after 7 days and aborts multipart uploads unfinished after a day; RustFS's scanner applies it), and `mailpit` (mail). Volumes: `g1t-data`, `g1t-git`, `g1t-objects`, `g1t-status`, `g1t-secrets`. |
422423 | `Dockerfile` | Compiles the ten Rust services to WebAssembly with `worker-build`, as hosted does. Builds the site with React Router. The runtime image has Node, Wrangler, workerd and the built Workers. |
423424 | `Dockerfile.dockerignore` | Build-context rules for this image only (the root `.dockerignore` leaves out the site). |
424425 | `start.sh` | Makes the sealing keys once, writes the configs, applies migrations, runs `wrangler dev` with every config on `0.0.0.0:8787`, persisting to `/data/state`, and the API's `wrangler dev` on `0.0.0.0:8789` once the first answers. Starts `scheduler.mjs`. |
428429 | `workers/artifacts/index.js` | The `ARTIFACTS` binding, implemented against the git store. |
429430 | `workers/mail/index.js` | The `EMAIL` binding: logs, then sends to Mailpit. |
430431 | `workers/off/index.js` | The runner and the context hub when they are off. |
431−| `smoke.sh` | The end-to-end check, including the API, pull requests, the merge queue and (with `SCHEDULER_ONCE`) every cron handler. |
432+| `smoke.sh` | The end-to-end check, including the clone pack cache, the API, an npm package published and installed, pull requests, the merge queue and (with `SCHEDULER_ONCE`) every cron handler. |
432433
433434 Workers running: the site; identity, repos, work, events, projects, search,
434435 billing, security, actions, webhooks, integrations, packages and
450451 The workspace context page answered 403 from the off stand-in, as
451452 intended.
452453 2. **With Docker Compose** (2026-10-07, `docker compose up --build`, with
453− `API_PORT=18789` because 8789 was taken on this machine). `smoke.sh`
454+ `API_PORT=18789` because 8789 was taken on this machine; the store is
455+ RustFS 1.0.1, its buckets made by `storage-setup`). `smoke.sh`
454456 passed every step: the ones above; a second clone answered from the pack
455− cache (`Server-Timing: pack;desc=hit`), with the packs in MinIO's
456− `g1t-git-packs` and its 7-day rule in place; an access token made in the
457− site; `GET /user`, the API index (`mcp_url`, `git_url`), the OAuth
457+ cache (`Server-Timing: pack;desc=hit`), with the packs in RustFS's
458+ `g1t-git-packs` and its lifecycle rule in place (packs expire after 7
459+ days, unfinished uploads are aborted after 1); an access token made in the
460+ site; an npm package published to the installation's registry and
461+ installed back, its tarball in `g1t-packages`; `GET /user`, the API index (`mcp_url`, `git_url`), the OAuth
458462 metadata (`issuer` the API's address, `authorization_endpoint` on the
459463 site), MCP's 401 challenge and `tools/list`; a pull request from a branch
460464 and one from a fork (`create_pull_request` without a branch: a fork in
463467 shown `waiting`, taken out (`unqueue`), the queue turned off and the
464468 pull request merged; and `scheduler.mjs --once`, every handler `ok`.
465469 The repository page's clone box and MCP line named the installation's
466− own addresses, with no social card tags.
467−3. **The clone pack cache against MinIO without the stack.**
468− `node services/repos/dev/clone-check.mjs --s3` (MinIO in Docker):
469− misses then hits for full and shallow clones over protocol v2 and v0, a
470− miss after the refs version moves, five whole packs in the bucket (12 MB
471− each, so uploaded in parts), no unfinished upload, and the expiry rule.
470+ own addresses, with no social card tags. By hand against the same
471+ stack: `docker push` and `docker pull` of an image with a 20 MB layer
472+ (uploaded in 10 MiB parts), the layer read back from `/v2/.../blobs/`
473+ with a matching digest, the `g1t-backups` bucket present (empty: the
474+ runner cuts bundles), no unfinished upload in any bucket, and the
475+ guide's upgrade copy from an old MinIO volume into RustFS.
476+3. **The clone pack cache against RustFS without the stack.**
477+ `node services/repos/dev/clone-check.mjs --s3` (RustFS in Docker, the
478+ bucket and lifecycle rule made with the AWS CLI): misses then hits for
479+ full and shallow clones over protocol v2 and v0, a miss after the refs
480+ version moves, five whole packs in the bucket (12 MB each, multipart
481+ objects of 3 parts), each reading back at its listed size, no
482+ unfinished upload, and both lifecycle rules.
472483
473484 ### Not verified, or not working yet
474485
511522 a registry directory, a development feature like the rest. If the API
512523 starts and a binding says `[not connected]`, restart the container. The
513524 phase 2 launcher serves both from one workerd.
514−- **The MinIO image.** MinIO stopped publishing `minio/minio` and
515− `minio/mc`. The compose file uses a community build (`MINIO_IMAGE`,
516− `pgsty/minio`); any S3-compatible store can take its place.
525+- **The object store.** The compose file runs RustFS (Apache-2.0),
526+ pinned to a release tag, and makes its buckets with the AWS CLI
527+ (Apache-2.0). Any S3-compatible store can take its place
528+ (`S3_ENDPOINT`), given the same buckets and the packs' lifecycle rule.
529+ Installations started before 2026-10-07 kept these files in MinIO, in
530+ the `g1t-packages` volume; the guide's "Upgrade" section copies them
531+ across.
517532 - **Cron goes through Wrangler's local API.** `scheduler.mjs` asks
518533 `/cdn-cgi/local/explorer/api/local/scheduled`, a development endpoint
519534 that may change between Wrangler releases (pinned by the lockfile).
+2−2
2121 // backup's.
2222 // --repo-id <id> the repository by id (no database needed with --bundles).
2323 // --bundles <dir> read the bucket from a local copy (`backups/<id>/...`
24−// under it, as `mc mirror` or `rclone copy` leave it)
25−// instead of R2, e.g. a self-hosted MinIO's.
24+// under it, as `aws s3 sync` or `rclone copy` leave it)
25+// instead of R2, e.g. a self-hosted store's.
2626 // --live <url or path> the live repository to compare with; default
2727 // https://g1t.sh/<workspace>/<name>.git.
2828 // --keep keep the temporary directory, and say where it is.
+1−1
11 //! Where packages' files are kept: the `BlobStore` port (crates/blobstore),
2−//! with R2 behind it on Cloudflare and any S3-compatible storage (MinIO in
2+//! with R2 behind it on Cloudflare and any S3-compatible storage (RustFS in
33 //! the compose file) when self-hosted. BLOB_STORE chooses: `r2` (the
44 //! default) or `s3`.
55 //!
+1−1
77 //! again after [`Settings::full_every`] incremental ones, so a restore
88 //! never reads a long chain. Bundles and a manifest that lists the chain
99 //! are kept in object storage through the `BlobStore` port: the BACKUPS R2
10−//! bucket hosted, any S3-compatible store (MinIO in the compose file)
10+//! bucket hosted, any S3-compatible store (RustFS in the compose file)
1111 //! self-hosted, as BACKUP_STORE says.
1212 //!
1313 //! ```text
+1−1
8989 /// Where packs are kept, by the names of the service's bindings and
9090 /// variables: the `GIT_PACKS` R2 bucket on Cloudflare or, when PACK_STORE
9191 /// is `s3`, the bucket PACK_S3_BUCKET names on the installation's
92−/// S3-compatible store (`g1t-git-packs` on MinIO in the self-host compose
92+/// S3-compatible store (`g1t-git-packs` on RustFS in the self-host compose
9393 /// file). On either an object exists only once it is whole: a `put` makes
9494 /// it in one write, and a multipart upload is nothing anyone can read
9595 /// until it is completed.