flagon-io/g1t

public

docs/PACKAGES.md: what each registry has built and not, Cargo's index path and credential provider, Gradle, Maven plugin groups, NuGet symbols and RubyGems' full index

syntaqxcommitted 4h agoParent3547956Browse files

1 file+85−140/1 viewed

+85−14View BlameViewed

8	8		This is the design every phase builds to. Each ecosystem's own guide (apps/docs) says how to use
9	9		it; this says how it works.
10	10
	11	+	## Status
	12	+
	13	+	What is built, as of 2026-10-07. Each protocol section below marks the same. "Checked" says what was run
	14	+	on that day against `wrangler dev` (`services/packages/dev/`) with the tool itself, beyond unit tests.
	15	+
	16	+	\| Registry \| Built \| Not built \|
	17	+	\| --- \| --- \| --- \|
	18	+	\| Containers \| Pull, push (chunked, multipart, monolithic, cross-repository mount), deletes, referrers, token exchange, anonymous pull limits; `g1t push` for layers over the request limit. \| \|
	19	+	\| npm \| Scoped publish and install, abbreviated packuments, dist-tags, deprecate, unpublish (72 hours, or Admin), `whoami`. \| Proxying unscoped packages and other scopes from the public registry. \|
	20	+	\| Composer \| Packages from the workspace's repositories: tags and branches as versions, dist zips made and kept by commit, backfill. \| A Packagist mirror. \|
	21	+	\| Cargo \| Sparse index, `config.json` with `auth-required`, publish, yank and unyank, search; owners are not kept (`cargo owner` answers why). Checked with `cargo`, including a workflow-like publish and build with only environment variables. \| \|
	22	+	\| Go \| `go get` from git: `?go-get=1` answers with `go-import`. \| The module proxy at `/-/go/`. \|
	23	+	\| Maven \| Standard layout, releases and SNAPSHOTs, checksums (MD5, SHA-1, SHA-256, SHA-512), generated `maven-metadata.xml` per artifact, SNAPSHOT and plugin group (`mvn <prefix>:<goal>`), Gradle Module Metadata. Checked with `mvn deploy`, `mvn <prefix>:<goal>`, and Gradle `publish` and resolution (releases, SNAPSHOTs, `--refresh-dependencies`). \| \|
	24	+	\| NuGet \| v3 feed: push, flat container, registration, search, unlist and relist, symbol packages (`.snupkg`) and a symbol server, per-version download counts. Checked with `dotnet nuget push`, `dotnet restore` and `dotnet-symbol`. \| \|
	25	+	\| RubyGems \| `gem push`, `gem yank`, the compact index (Bundler), the full index (`specs.4.8.gz`, `latest_`, `prerelease_`, `quick/Marshal.4.8`). Checked with `gem push`, `gem install --source`, `gem search` and `gem specification --remote`. \| \|
	26	+	\| PyPI \| \| Everything. \|
	27	+
	28	+	Across registries: events, webhooks, the audit log, the billing meter and free limits, the
	29	+	Packages pages and a project's packages are built. Workflows triggered by `registry_package`,
	30	+	packages in site-wide search, and packages in the activity feed are not.
	31	+
11	32		## Principles
12	33
13	34		- One service. `services/packages` (Rust Worker) owns every registry: its own D1 database
⋯
62	83
63	84		### Containers (OCI Distribution 1.1)
64	85
	86	+	Built.
	87	+
65	88		- Image names: `g1t.sh/<workspace>/<name>[:tag]`, `<name>` may contain `/`. Usually the
66	89		repository's name, and then linked to it.
67	90		- `GET /v2/` answers 401 with `WWW-Authenticate: Bearer realm="https://g1t.sh/v2/token",
⋯
86	109
87	110		### npm
88	111
	112	+	Built, except the proxy of unscoped packages.
	113	+
89	114		- Registry `https://g1t.sh/-/npm/`; scope = workspace: `@<workspace>/<name>`.
90	115		`.npmrc`: `@acme:registry=https://g1t.sh/-/npm/` and `//g1t.sh/-/npm/:_authToken=<token>`.
91	116		- `GET /@scope/name` (packument, abbreviated with `Accept: application/vnd.npm.install-v1+json`),
92	117		`GET` tarballs, `PUT /@scope/name` (publish: JSON with the tarball attached), dist-tags,
93	118		deprecate, unpublish (within 72 hours or with Admin), `GET /-/whoami`.
94	119		- Unscoped and other scopes: optionally proxied from the public registry and kept, so one
95		−	`.npmrc` line serves everything (later phase).
	120	+	`.npmrc` line serves everything (later phase; not built).
96	121
97	122		### Composer
98	123
	124	+	Built, except the Packagist mirror.
	125	+
99	126		- Per workspace: `https://g1t.sh/-/composer/<workspace>/` with `packages.json` naming
100	127		`metadata-url` `/p2/%package%.json` and `available-packages`.
101	128		- Versions come from the workspace's repositories themselves: a repository with a
⋯
104	131		kept by commit. Pushing a tag publishes; nothing to upload.
105	132		- Auth: `composer config --auth http-basic.g1t.sh <you> <token>` (`auth.json`).
106	133		- A mirror of the public Packagist (metadata and dists kept, so installs survive its outages) is
107		−	a later phase.
	134	+	a later phase (not built).
108	135
109	136		### Cargo
110	137
111		−	- Sparse registry per workspace: `sparse+https://g1t.sh/-/cargo/<workspace>/`. `config.json`
	138	+	Built.
	139	+
	140	+	- Sparse registry per workspace: `sparse+https://g1t.sh/-/cargo/<workspace>/index/`. `config.json`
112	141		(`dl`, `api`, `auth-required` for private), index files at the standard prefix paths, crate
113		−	downloads, `PUT /api/v1/crates/new` (publish), yank and unyank, owners.
114		−	- Auth: a g1t token through `cargo login --registry g1t`.
	142	+	downloads, `PUT /api/v1/crates/new` (publish), yank and unyank, search. Owners are not kept:
	143	+	who publishes is decided by the repository's or workspace's roles, and `cargo owner` answers
	144	+	with an error saying so.
	145	+	- Auth: a g1t token, given to the registry's credential provider (`cargo:token`), named in
	146	+	`.cargo/config.toml` as `[registries.<workspace>]`: `cargo login --registry <workspace>`, or
	147	+	in workflows `CARGO_REGISTRIES_<WORKSPACE>_TOKEN` (with `CARGO_REGISTRIES_<WORKSPACE>_INDEX`
	148	+	and `CARGO_REGISTRIES_<WORKSPACE>_CREDENTIAL_PROVIDER=cargo:token` when no config names the
	149	+	registry). Without a provider Cargo refuses a registry with private crates.
115	150
116	151		### Go
117	152
	153	+	Built from git; the module proxy is not built.
	154	+
118	155		- `go get g1t.sh/<workspace>/<repo>` works from git: repository pages answer `?go-get=1` with the
119	156		`go-import` meta tag. Private modules need `GOPRIVATE=g1t.sh/<workspace>` and a token in
120	157		`.netrc` (as for git).
121	158		- A module proxy at `https://g1t.sh/-/go/` (`@v/list`, `.info`, `.mod`, `.zip`) built from tags,
122		−	for faster and repeatable private installs (later phase).
	159	+	for faster and repeatable private installs (later phase; not built).
123	160
124	161		### Maven
125	162
	163	+	Built, for Maven and Gradle.
	164	+
126	165		- Per workspace: `https://g1t.sh/-/maven/<workspace>/`, the standard layout
127	166		(`com/acme/web/1.0.0/web-1.0.0.jar`). A package is an artifact, named
128	167		`groupId:artifactId`; a version holds every file uploaded into its
⋯
135	174		and kept by digest (`checksums`); `.md5`, `.sha1`, `.sha256`, `.sha512`
136	175		are answered from them, and uploaded ones are checked, not kept.
137	176		- `maven-metadata.xml` is made on every read: per artifact (versions in
138		−	Maven's order, `latest`, `release`) and per SNAPSHOT version (the newest
139		−	build of each classifier and extension). Uploaded ones are accepted and
140		−	let go, a plugin group's included.
	177	+	Maven's order, `latest`, `release`), per SNAPSHOT version (the newest
	178	+	build of each classifier and extension), and per group (`<plugins>`: each
	179	+	`maven-plugin` artifact's prefix, artifactId and name, which is how
	180	+	`mvn <prefix>:<goal>` finds a plugin in its `<pluginGroups>`). The prefix
	181	+	is the `goalPrefix` of the jar's `META-INF/maven/plugin.xml`, read when
	182	+	the jar arrives, else Maven's from the artifactId. A path that is both an
	183	+	artifact's and a group's answers both. Uploaded ones are accepted and let
	184	+	go, a plugin group's included.
	185	+	- Gradle: its `.module` files are kept and served like any other file, and
	186	+	its `HEAD` requests and SHA-256 and SHA-512 checksum uploads are
	187	+	answered as Maven's are.
141	188		- The POM is the version's record: its coordinates must
142	189		match its path, its description becomes the package's for the highest
143	190		release, and on a new artifact its `<scm><url>` may link the repository.
⋯
147	194
148	195		### NuGet
149	196
	197	+	Built.
	198	+
150	199		- Per workspace: `https://g1t.sh/-/nuget/<workspace>/v3/index.json` naming
151	200		`PackageBaseAddress/3.0.0` (flat container), `RegistrationsBaseUrl`
152		−	(one inlined page), `SearchQueryService` and `PackagePublish/2.0.0`.
	201	+	(one inlined page), `SearchQueryService`, `PackagePublish/2.0.0` and
	202	+	`SymbolPackagePublish/4.9.0`.
153	203		- `dotnet nuget push`: a `PUT` of a multipart body with the `.nupkg`,
154	204		`X-NuGet-ApiKey` a g1t token. The `.nuspec` (read from the zip) gives the
155	205		id, version (normalized as NuGet does), description, dependency groups and
⋯
159	209		nuget.org does; `POST` lists again. Unlisted versions stay in the flat
160	210		container and registration (`listed: false`), not in search.
161	211		- Restores use Basic auth from `nuget.config`, after a `401`.
	212	+	- Downloads: each `.nupkg` download counts for its version (`versions.downloads`) and its
	213	+	package; the registration's catalog entries and search's versions name them.
	214	+	- Symbols: `dotnet nuget push` sends the `.snupkg` beside a `.nupkg` to
	215	+	`api/v2/symbolpackage` after it. It must be for a version already pushed, say
	216	+	`SymbolsPackage` as its package type, and hold portable PDBs. The `.snupkg` is the version's
	217	+	file `snupkg` (also in the flat container), and each PDB its file `pdb:<file>:<key>`, the key
	218	+	being the PDB id's GUID (as `Guid.ToString("N")`) and `ffffffff`. The symbol server,
	219	+	`symbols/<file>.pdb/<key>/<file>.pdb` (the Simple Symbol Query Protocol), finds it by that
	220	+	name across the workspace's packages the viewer may read (`version_files_name` index). A
	221	+	version's symbols are pushed once (`409`). The package page marks versions with symbols.
162	222
163	223		### RubyGems
164	224
	225	+	Built.
	226	+
165	227		- Per workspace: `https://g1t.sh/-/rubygems/<workspace>/`. `gem push`
166	228		(`POST /api/v1/gems`, the token as the whole `Authorization` header),
167	229		`gem yank` (`DELETE /api/v1/gems/yank`), downloads at
⋯
173	235		- The gem's `metadata.gz` (YAML, in the `.gem` tar) gives the name,
174	236		version, platform and runtime dependencies. A version is keyed as the
175	237		index writes it (`1.0.0`, `1.0.0-x86_64-linux`) and pushed once.
176		−	- Bundler authenticates with Basic auth from `bundle config`. The full
177		−	index (`specs.4.8.gz`, Marshal) is not served, so `gem install --source`
178		−	is not supported.
	238	+	- The full index `gem install --source` and `gem search` read:
	239	+	`specs.4.8.gz` (released versions), `latest_specs.4.8.gz` (the highest of
	240	+	each gem and platform) and `prerelease_specs.4.8.gz`, each a gzipped
	241	+	Ruby Marshal 4.8 array of `[name, Gem::Version, platform]`, and
	242	+	`quick/Marshal.4.8/<name>-<version>[-<platform>].gemspec.rz`, the
	243	+	deflated Marshal of a `Gem::Specification` as its `_dump` writes it. All
	244	+	are made on each read from what each version keeps (`src/marshal.rs` is
	245	+	the writer), and read by RubyGems' `SafeMarshal`.
	246	+	- Bundler authenticates with Basic auth from `bundle config`; `gem` with
	247	+	credentials in the source's address.
179	248
180	249		### Later
181	250
182		−	PyPI.
	251	+	PyPI (not built).
183	252
184	253		## Billing
185	254
⋯
222	291		4. Cargo.
223	292		5. Maven, NuGet, RubyGems.
224	293		6. Mirrors (Packagist, npm).
	294	+
	295	+	Phases 1 to 5 are built (see [Status](#status)); 6 is not.