Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Packages: the design every phase builds to | 1 | # Packages: design |
| 2 | ||
| 3 | Packages are registries a workspace publishes to and installs from, beside its code: container | |
| 4 | images, npm, Composer, Cargo and Go first, then Maven, NuGet and RubyGems. They live where the | |
| 5 | code does, take the same people and tokens, and are built and published by g1t Actions and | |
| 6 | agents without a second account anywhere. | |
| 7 | ||
| 8 | This is the design every phase builds to. Each ecosystem's own guide (apps/docs) says how to use | |
| 9 | it; this says how it works. | |
| 10 | ||
| 11 | ## Principles | |
| 12 | ||
| 13 | - **One service.** `services/packages` (Rust Worker) owns every registry: its own D1 database | |
| 14 | (`g1t-packages`), its own file store, its own contract in `crates/contracts` / | |
| 15 | `packages/contracts`. Other services talk to it over RPC and hear from it through events. | |
| 16 | - **Cloudflare in production, anything in a self-hosted install.** Files go through a | |
| 17 | `BlobStore` port. In production its adapter is R2; self-hosted it is S3-compatible storage | |
| 18 | (MinIO in the compose file) or a local directory. Upload URLs (`presign`) are part of the port: | |
| 19 | R2 and S3 sign them, the disk adapter answers with a path back through the service. | |
| 20 | Metadata is D1, which self-hosting already runs (workerd's D1 over SQLite). | |
| 21 | - **Content-addressed.** Every file is stored once by its SHA-256 (`blobs/sha256/<hex>`). A | |
| 22 | version is a list of files by digest. Pushing a layer two images share stores it once. | |
| 23 | - **The registry speaks each tool's own protocol**, unchanged. `docker`, `npm`, `composer`, | |
| Packages: Maven, NuGet and RubyGems registries for every workspace | 24 | `cargo`, `go`, `mvn` and Gradle, `dotnet` and `gem` and Bundler work with only a login and an |
| 25 | address. | |
| Packages: the design every phase builds to | 26 | - **Same access as the code.** A package can be linked to a repository and then has its |
| 27 | visibility and roles. Unlinked, it is the workspace's: members by the base permission. | |
| 28 | - **Same front door.** Everything is on `g1t.sh`. `apps/web/workers/app.ts` hands registry paths | |
| 29 | to the `PACKAGES` binding the way it hands git paths to repos. | |
| 30 | ||
| 31 | ## Model | |
| 32 | ||
| 33 | | Table | What it holds | | |
| 34 | | --- | --- | | |
| Packages: Maven, NuGet and RubyGems registries for every workspace | 35 | | `packages` | `id`, `workspace`, `ecosystem` (`container`, `npm`, `composer`, `cargo`, `go`, `maven`, `nuget`, `rubygems`), `name` (normalized per ecosystem), `repo_id` (linked repository, or null), `visibility` (`public`, `private`; linked packages follow their repository), `description`, `readme_digest`, `created_by`, `created_at`, `updated_at`, `downloads` | |
| Packages: the design every phase builds to | 36 | | `versions` | `id`, `package_id`, `version` (tag, semver or digest), `digest` (the manifest's or the archive's), `size` (sum of its files), `metadata` (JSON the ecosystem needs: npm's packument entry, a crate's index line, composer.json), `published_by`, `published_at`, `yanked`, `deprecated` | |
| 37 | | `version_files` | `version_id`, `name`, `digest`, `size`, `media_type` | | |
| 38 | | `blobs` | `digest`, `size`, `media_type`, `created_at` | | |
| 39 | | `workspace_blobs` | `workspace`, `digest`, `public` (any public package uses it): what a workspace stores, counted once each, for billing | | |
| 40 | | `uploads` | `id`, `workspace`, `package`, `multipart_id`, `parts` (JSON), `offset`, `hash_state`, `expires_at`: uploads in progress | | |
| 41 | | `tags` | `package_id`, `tag`, `version_id`: container tags and npm dist-tags | | |
| Packages: Maven, NuGet and RubyGems registries for every workspace | 42 | | `checksums` | `digest`, `md5`, `sha1`, `sha512`: a file's other checksums, worked out on upload (Maven asks for them) | |
| Packages: the design every phase builds to | 43 | |
| 44 | Deleting a version removes its rows; a daily sweep deletes blobs no version references, after a | |
| 45 | day's grace (a push in flight may reference a blob before its manifest lands). | |
| 46 | ||
| 47 | ## Access | |
| 48 | ||
| 49 | - **Who:** people (session or personal token), workspace tokens, `G1T_TOKEN` in workflows, | |
| 50 | agents' run tokens. Scopes `packages:read` and `packages:write` (and `packages:delete`), per the | |
| 51 | MCP and token scope design. | |
| 52 | - **What:** linked package: the repository's roles (Read pulls, Write publishes, Admin deletes and | |
| 53 | changes settings). Unlinked: workspace members by base permission; owners delete. | |
| 54 | - **Public packages** pull anonymously. Anonymous pulls are rate limited per address. | |
| 55 | - **Workflows** may publish the packages of their own repository, and new ones linked to it, with | |
| 56 | `G1T_TOKEN`, as code they push does. | |
| 57 | - Every publish and delete is an audit entry and an event. | |
| 58 | ||
| 59 | ## Protocols | |
| 60 | ||
| 61 | All under `g1t.sh`. A name always starts with the workspace. | |
| 62 | ||
| 63 | ### Containers (OCI Distribution 1.1) | |
| 64 | ||
| 65 | - Image names: `g1t.sh/<workspace>/<name>[:tag]`, `<name>` may contain `/`. Usually the | |
| 66 | repository's name, and then linked to it. | |
| 67 | - `GET /v2/` answers 401 with `WWW-Authenticate: Bearer realm="https://g1t.sh/v2/token", | |
| 68 | service="g1t.sh"`. `GET /v2/token` takes Basic auth (any username, a g1t token as the | |
| 69 | password) or none (anonymous, public pulls) and returns a short-lived signed token naming the | |
| 70 | scopes granted. `docker login g1t.sh -u <you> --password-stdin` stores it. | |
| 71 | - Pull: `HEAD/GET /v2/<name>/manifests/<ref>`, `HEAD/GET /v2/<name>/blobs/<digest>` (a | |
| 72 | redirect to a signed R2 URL for large blobs, so downloads never pass through the Worker), | |
| 73 | `GET /v2/<name>/tags/list`, `GET /v2/<name>/referrers/<digest>`. | |
| 74 | - Push: `POST /v2/<name>/blobs/uploads/` (`?mount=&from=` cross-repository mount within a | |
| 75 | workspace; `?digest=` monolithic), `PATCH` chunks, `PUT ?digest=` to finish, | |
| 76 | `PUT /v2/<name>/manifests/<ref>` (image manifests, indexes, OCI artifacts, `subject` for | |
| 77 | referrers). `DELETE` manifests and blobs. | |
| 78 | - **Upload size.** On Cloudflare a request body is limited by the zone's plan (Free: 100 MB). | |
| 79 | Uploads are written to R2 as multipart parts, so a layer may be any size if it arrives in | |
| 80 | chunks under the limit. `docker push` sends a layer in one request, so on Cloudflare a layer | |
| 81 | over the limit is refused with a message naming the limit and the way around it: `g1t push` | |
| g1t push: images with layers of any size, uploaded in chunks under the edge's limit | 82 | (the CLI, `crates/g1t`), which reads the image with `docker save` (OCI layout or classic, |
| 83 | gzipping uncompressed layers and writing a matching manifest) and sends every blob as OCI | |
| 84 | chunks of 90 MiB (at most 95 MiB), resuming from the upload's `Range` after a `429`/`5xx`. | |
| 85 | Self-hosted, there is no such limit. | |
| Packages: the design every phase builds to | 86 | |
| 87 | ### npm | |
| 88 | ||
| 89 | - Registry `https://g1t.sh/-/npm/`; scope = workspace: `@<workspace>/<name>`. | |
| 90 | `.npmrc`: `@acme:registry=https://g1t.sh/-/npm/` and `//g1t.sh/-/npm/:_authToken=<token>`. | |
| 91 | - `GET /@scope/name` (packument, abbreviated with `Accept: application/vnd.npm.install-v1+json`), | |
| 92 | `GET` tarballs, `PUT /@scope/name` (publish: JSON with the tarball attached), dist-tags, | |
| 93 | deprecate, unpublish (within 72 hours or with Admin), `GET /-/whoami`. | |
| 94 | - Unscoped and other scopes: optionally proxied from the public registry and kept, so one | |
| 95 | `.npmrc` line serves everything (later phase). | |
| 96 | ||
| 97 | ### Composer | |
| 98 | ||
| 99 | - Per workspace: `https://g1t.sh/-/composer/<workspace>/` with `packages.json` naming | |
| 100 | `metadata-url` `/p2/%package%.json` and `available-packages`. | |
| 101 | - Versions come from **the workspace's repositories themselves**: a repository with a | |
| 102 | `composer.json` at its root is a package (its `name` from that file); each tag is a version and | |
| 103 | each branch a `dev-` version. Dist archives are zips of the commit, made on first request and | |
| 104 | kept by commit. Pushing a tag publishes; nothing to upload. | |
| 105 | - Auth: `composer config --auth http-basic.g1t.sh <you> <token>` (`auth.json`). | |
| 106 | - A mirror of the public Packagist (metadata and dists kept, so installs survive its outages) is | |
| 107 | a later phase. | |
| 108 | ||
| 109 | ### Cargo | |
| 110 | ||
| 111 | - Sparse registry per workspace: `sparse+https://g1t.sh/-/cargo/<workspace>/`. `config.json` | |
| 112 | (`dl`, `api`, `auth-required` for private), index files at the standard prefix paths, crate | |
| 113 | downloads, `PUT /api/v1/crates/new` (publish), yank and unyank, owners. | |
| 114 | - Auth: a g1t token through `cargo login --registry g1t`. | |
| 115 | ||
| 116 | ### Go | |
| 117 | ||
| 118 | - `go get g1t.sh/<workspace>/<repo>` works from git: repository pages answer `?go-get=1` with the | |
| 119 | `go-import` meta tag. Private modules need `GOPRIVATE=g1t.sh/<workspace>` and a token in | |
| 120 | `.netrc` (as for git). | |
| 121 | - A module proxy at `https://g1t.sh/-/go/` (`@v/list`, `.info`, `.mod`, `.zip`) built from tags, | |
| 122 | for faster and repeatable private installs (later phase). | |
| 123 | ||
| Packages: Maven, NuGet and RubyGems registries for every workspace | 124 | ### Maven |
| 125 | ||
| 126 | - Per workspace: `https://g1t.sh/-/maven/<workspace>/`, the standard layout | |
| 127 | (`com/acme/web/1.0.0/web-1.0.0.jar`). A package is an artifact, named | |
| 128 | `groupId:artifactId`; a version holds every file uploaded into its | |
| 129 | directory, by file name (`version_files`), added one `PUT` at a time. | |
| 130 | - `mvn deploy` and Gradle's `publish`: Basic auth (any username, a token as | |
| 131 | the password) or `Bearer`. A release's files are written once (`409` for | |
| 132 | other content, the same content again is accepted); a SNAPSHOT's builds | |
| 133 | arrive as timestamped files beside each other. | |
| 134 | - Checksums: each file's MD5, SHA-1 and SHA-512 are worked out on upload | |
| 135 | and kept by digest (`checksums`); `.md5`, `.sha1`, `.sha256`, `.sha512` | |
| 136 | are answered from them, and uploaded ones are checked, not kept. | |
| 137 | - `maven-metadata.xml` is made on every read: per artifact (versions in | |
| 138 | Maven's order, `latest`, `release`) and per SNAPSHOT version (the newest | |
| 139 | build of each classifier and extension). Uploaded ones are accepted and | |
| 140 | let go, a plugin group's included. | |
| 141 | - The POM is the version's record: its coordinates must | |
| 142 | match its path, its description becomes the package's for the highest | |
| 143 | release, and on a new artifact its `<scm><url>` may link the repository. | |
| 144 | - The artifact's own `maven-metadata.xml`, uploaded last by Maven and Gradle, | |
| 145 | publishes (event, audit entry) each version or SNAPSHOT build whose POM | |
| 146 | arrived since, marked in its metadata so it is published once. | |
| 147 | ||
| 148 | ### NuGet | |
| 149 | ||
| 150 | - Per workspace: `https://g1t.sh/-/nuget/<workspace>/v3/index.json` naming | |
| 151 | `PackageBaseAddress/3.0.0` (flat container), `RegistrationsBaseUrl` | |
| 152 | (one inlined page), `SearchQueryService` and `PackagePublish/2.0.0`. | |
| 153 | - `dotnet nuget push`: a `PUT` of a multipart body with the `.nupkg`, | |
| 154 | `X-NuGet-ApiKey` a g1t token. The `.nuspec` (read from the zip) gives the | |
| 155 | id, version (normalized as NuGet does), description, dependency groups and | |
| 156 | README; the `.nupkg` and `.nuspec` are the version's files. | |
| 157 | - Ids are one whatever their case; a version is pushed once (`409`). | |
| 158 | - `DELETE api/v2/package/<id>/<version>` unlists (the `yanked` column), as | |
| 159 | nuget.org does; `POST` lists again. Unlisted versions stay in the flat | |
| 160 | container and registration (`listed: false`), not in search. | |
| 161 | - Restores use Basic auth from `nuget.config`, after a `401`. | |
| 162 | ||
| 163 | ### RubyGems | |
| 164 | ||
| 165 | - Per workspace: `https://g1t.sh/-/rubygems/<workspace>/`. `gem push` | |
| 166 | (`POST /api/v1/gems`, the token as the whole `Authorization` header), | |
| 167 | `gem yank` (`DELETE /api/v1/gems/yank`), downloads at | |
| 168 | `/gems/<name>-<version>[-<platform>].gem`. | |
| 169 | - The compact index Bundler reads: `versions`, `info/<gem>`, `names`, made | |
| 170 | on every read, each with the quoted MD5 of its body as its `ETag` (Bundler | |
| 171 | checks it, and `versions` names each info file's MD5). Yanked versions | |
| 172 | leave the index; their files stay. | |
| 173 | - The gem's `metadata.gz` (YAML, in the `.gem` tar) gives the name, | |
| 174 | version, platform and runtime dependencies. A version is keyed as the | |
| 175 | index writes it (`1.0.0`, `1.0.0-x86_64-linux`) and pushed once. | |
| 176 | - Bundler authenticates with Basic auth from `bundle config`. The full | |
| 177 | index (`specs.4.8.gz`, Marshal) is not served, so `gem install --source` | |
| 178 | is not supported. | |
| 179 | ||
| Packages: the design every phase builds to | 180 | ### Later |
| 181 | ||
| Packages: Maven, NuGet and RubyGems registries for every workspace | 182 | PyPI. |
| Packages: the design every phase builds to | 183 | |
| 184 | ## Billing | |
| 185 | ||
| 186 | Storage is what costs: R2 is about $0.015 per GB-month, with no charge for downloads (no egress | |
| 187 | fees) and fractions of a cent per thousand requests. Defaults, all `services/billing` variables: | |
| 188 | ||
| 189 | | | Free workspace | g1t plan | | |
| 190 | | --- | --- | --- | | |
| 191 | | Public packages | Free up to `PUBLIC_PACKAGES_FREE_BYTES` (10 GB) per workspace; pushes past it are refused | The same 10 GB free, then storage at cost plus the margin | | |
| 192 | | Private packages | `PRIVATE_PACKAGES_FREE_BYTES` (500 MB) free; pushes past it are refused | Storage at cost plus the margin from the first byte past 500 MB | | |
| 193 | | Downloads | Free; anonymous pulls rate limited | Free | | |
| 194 | ||
| 195 | - A workspace's package storage is measured daily from `workspace_blobs` (a blob counts once per | |
| 196 | workspace, and as public when any public package uses it), on the same meter run as private | |
| 197 | repository storage, and charged as `package_storage` GB-months. | |
| 198 | - Pushes check the free limits before accepting a blob, with a message naming the limit. | |
| 199 | ||
| 200 | ## Events | |
| 201 | ||
| 202 | `package.published`, `package.version_deleted`, `package.deleted`, `package.visibility_changed`. | |
| 203 | Webhooks deliver them (`package` and `registry_package` shapes); workflows can run on | |
| 204 | `registry_package`; the activity feed and search index them; the audit log records them. | |
| 205 | ||
| 206 | ## UI | |
| 207 | ||
| 208 | - Workspace **Packages** (`/<workspace>/-/packages`): every package, by ecosystem, with search. | |
| 209 | - A project's packages on its overview, and a Packages tab when it has any. | |
| 210 | - Package page: install and log-in commands for its tool, versions and tags, README, size, | |
| 211 | downloads, linked repository, who published; settings (visibility, link, delete) for Admins. | |
| 212 | - Site-wide search finds public packages. | |
| 213 | ||
| 214 | ## Build order | |
| 215 | ||
| 216 | 1. **Core and containers.** The service, file store port (R2, S3, disk), access, token exchange, | |
| 217 | OCI pull and push (chunked and multipart, mounts, deletes, referrers), the sweep, events, | |
| 218 | billing meter and limits, Packages pages, docs. Then the runner's image moves here from Docker | |
| 219 | Hub, and `g1t push` for large layers. | |
| 220 | 2. **npm.** | |
| 221 | 3. **Composer and Go** (both from the repositories themselves). | |
| 222 | 4. **Cargo.** | |
| Packages: Maven, NuGet and RubyGems registries for every workspace | 223 | 5. **Maven, NuGet, RubyGems.** |
| 224 | 6. **Mirrors** (Packagist, npm). |