Cargo on g1t.sh: a sparse registry per workspace, cargo publish/add/yank/search with a g1t token; npm rows show their version
- /-/cargo/<workspace>/index/ (config.json and sparse index files with ETags) and the web API: publish, yank, unyank, download, search. Names follow crates.io's rules and clash across case and -/_. - Crates link to their repository, are billed and audited like npm, and yanked versions show as such. - The Packages list shows npm's and Cargo's versions, not the tag name; container images keep their tag. - Cargo guide in the docs.
13 files+1267−230/13 viewed
| 76 | 76 | { label: 'Packages', slug: 'guides/packages' }, | |
| 77 | 77 | { label: 'Container images', slug: 'guides/containers' }, | |
| 78 | 78 | { label: 'npm', slug: 'guides/npm' }, | |
| 79 | + | { label: 'Cargo', slug: 'guides/cargo' }, | |
| 79 | 80 | { label: 'Composer', slug: 'guides/composer' }, | |
| 80 | 81 | { label: 'Go modules', slug: 'guides/go' }, | |
| 81 | 82 | { label: 'Secrets and variables', slug: 'guides/secrets-and-variables' }, |
| 1 | + | --- | |
| 2 | + | title: Cargo | |
| 3 | + | description: Publish and add a workspace's Rust crates with Cargo, from a sparse registry of its own on g1t.sh, from your machine and from workflows. | |
| 4 | + | --- | |
| 5 | + | ||
| 6 | + | Every workspace has a Cargo registry of its own: a sparse index, with the | |
| 7 | + | web API that `cargo publish`, `cargo yank` and `cargo search` use. It | |
| 8 | + | works with Cargo 1.74 or later, private crates included. | |
| 9 | + | ||
| 10 | + | ```text | |
| 11 | + | sparse+https://g1t.sh/-/cargo/<workspace>/index/ | |
| 12 | + | ``` | |
| 13 | + | ||
| 14 | + | Crates from crates.io still come from crates.io; only the crates you name | |
| 15 | + | with the workspace's registry come from g1t. | |
| 16 | + | ||
| 17 | + | ## Set up `.cargo/config.toml` | |
| 18 | + | ||
| 19 | + | Name the registry in the project's `.cargo/config.toml`, or in | |
| 20 | + | `~/.cargo/config.toml` for every project. The name you give it is the one | |
| 21 | + | you pass to `--registry`; these pages use the workspace's slug: | |
| 22 | + | ||
| 23 | + | ```toml | |
| 24 | + | [registries.acme] | |
| 25 | + | index = "sparse+https://g1t.sh/-/cargo/acme/index/" | |
| 26 | + | credential-provider = "cargo:token" | |
| 27 | + | ``` | |
| 28 | + | ||
| 29 | + | Cargo sends a token to a registry that asks for one only through a | |
| 30 | + | credential provider named for it. `cargo:token` keeps the token in | |
| 31 | + | `~/.cargo/credentials.toml`, which stays out of the project. To keep it in | |
| 32 | + | your system's keychain instead, name `cargo:wincred` (Windows), | |
| 33 | + | `cargo:macos-keychain` (macOS) or `cargo:libsecret` (Linux). | |
| 34 | + | ||
| 35 | + | Then give Cargo an [access token](https://g1t.sh/settings/tokens): | |
| 36 | + | ||
| 37 | + | ```sh | |
| 38 | + | cargo login --registry acme | |
| 39 | + | ``` | |
| 40 | + | ||
| 41 | + | Cargo asks for the token and hands it to the provider. A token with full | |
| 42 | + | access works; one with scopes needs `packages:read` to add private crates | |
| 43 | + | and `packages:write` to publish and yank them. | |
| 44 | + | ||
| 45 | + | The token can also come from the environment, as | |
| 46 | + | `CARGO_REGISTRIES_ACME_TOKEN` for a registry named `acme`, which is how | |
| 47 | + | [workflows](#in-workflows) give it. | |
| 48 | + | ||
| 49 | + | ## Publish | |
| 50 | + | ||
| 51 | + | Say in `Cargo.toml` where the crate is published and which repository it | |
| 52 | + | comes from: | |
| 53 | + | ||
| 54 | + | ```toml | |
| 55 | + | [package] | |
| 56 | + | name = "http-client" | |
| 57 | + | version = "0.3.1" | |
| 58 | + | edition = "2021" | |
| 59 | + | description = "Our HTTP client" | |
| 60 | + | license = "MIT" | |
| 61 | + | readme = "README.md" | |
| 62 | + | repository = "https://g1t.sh/acme/http-client" | |
| 63 | + | publish = ["acme"] | |
| 64 | + | ``` | |
| 65 | + | ||
| 66 | + | ```sh | |
| 67 | + | cargo publish --registry acme | |
| 68 | + | ``` | |
| 69 | + | ||
| 70 | + | The first publish makes the crate. When its `repository` is a g1t.sh | |
| 71 | + | repository of the same workspace, or a repository is named like the crate | |
| 72 | + | (a crate `http_client` is also matched to a repository `http-client`), it | |
| 73 | + | is linked to that repository and has its visibility and roles: publishing needs Write on it. | |
| 74 | + | Otherwise it is the workspace's, private, and needs the workspace's Write | |
| 75 | + | base permission. See | |
| 76 | + | [who can see and publish a package](/guides/packages/#who-can-see-and-publish-a-package). | |
| 77 | + | ||
| 78 | + | `publish = ["acme"]` keeps the crate from being published to crates.io by | |
| 79 | + | mistake, and lets `cargo publish` leave out `--registry` when it is the | |
| 80 | + | only registry named. | |
| 81 | + | ||
| 82 | + | A crate may depend on crates.io crates and on other crates of the | |
| 83 | + | workspace's registry. The crate's page on g1t.sh shows the README and | |
| 84 | + | description of its highest stable version. | |
| 85 | + | ||
| 86 | + | ## Add a crate | |
| 87 | + | ||
| 88 | + | ```sh | |
| 89 | + | cargo add http-client --registry acme | |
| 90 | + | ``` | |
| 91 | + | ||
| 92 | + | That writes the dependency with its registry into `Cargo.toml`: | |
| 93 | + | ||
| 94 | + | ```toml | |
| 95 | + | [dependencies] | |
| 96 | + | http-client = { version = "0.3.1", registry = "acme" } | |
| 97 | + | ``` | |
| 98 | + | ||
| 99 | + | `Cargo.lock` records each crate's registry and the SHA-256 of its | |
| 100 | + | `.crate` file, which g1t computes when the version is published. | |
| 101 | + | ||
| 102 | + | `cargo search --registry acme http` lists the workspace's crates you can | |
| 103 | + | see whose names match. | |
| 104 | + | ||
| 105 | + | ## Names and versions | |
| 106 | + | ||
| 107 | + | A crate's name follows the crates.io rules: ASCII letters, digits, `-` and | |
| 108 | + | `_`, starting with a letter, at most 64 characters. In a workspace, names | |
| 109 | + | that differ only in case or in `-` against `_` are one name: once | |
| 110 | + | `http-client` is published, `HTTP_Client` is refused. | |
| 111 | + | ||
| 112 | + | A version is published once. Publishing a version that is already there is | |
| 113 | + | refused, as is one that differs from it only in build metadata (`1.0.0+a` | |
| 114 | + | and `1.0.0+b`), so bump `version` first. | |
| 115 | + | ||
| 116 | + | ## Yank | |
| 117 | + | ||
| 118 | + | ```sh | |
| 119 | + | cargo yank --registry acme http-client@0.3.1 | |
| 120 | + | cargo yank --registry acme http-client@0.3.1 --undo | |
| 121 | + | ``` | |
| 122 | + | ||
| 123 | + | A yanked version stays in the registry: projects whose `Cargo.lock` | |
| 124 | + | names it still build, but Cargo no longer picks it for new lockfiles or | |
| 125 | + | `cargo update`. Yanking needs what publishing does. The crate's page marks | |
| 126 | + | yanked versions, and someone with Admin on the linked repository (an | |
| 127 | + | owner, for the workspace's own crates) can delete a version there for | |
| 128 | + | good. | |
| 129 | + | ||
| 130 | + | Crate owners are not kept: who may publish a crate is decided by its | |
| 131 | + | repository's roles, or the workspace's, so `cargo owner` answers with an | |
| 132 | + | error that says so. | |
| 133 | + | ||
| 134 | + | ## Private and public crates | |
| 135 | + | ||
| 136 | + | A crate linked to a repository has the repository's visibility; one of | |
| 137 | + | the workspace's own is private until an owner makes it public on its page. | |
| 138 | + | ||
| 139 | + | | The workspace's crates | Without a token | With a token | | |
| 140 | + | | --- | --- | --- | | |
| 141 | + | | All public | Cargo reads the index and downloads them. | The same; the token is sent to publish and yank. | | |
| 142 | + | | Some private | The registry answers `401`: Cargo needs a token for every crate in it, public ones too. | Cargo sends the token with every request, and sees the crates the token's owner may see. | | |
| 143 | + | ||
| 144 | + | Cargo first asks for the registry's `config.json` without a token. When | |
| 145 | + | the registry answers `401`, Cargo asks again with its token, and the | |
| 146 | + | answer says `"auth-required": true`, so Cargo sends the token from then on. | |
| 147 | + | A private crate you cannot see looks exactly like one that does not exist. | |
| 148 | + | ||
| 149 | + | ## In workflows | |
| 150 | + | ||
| 151 | + | A workflow's `G1T_TOKEN` is the workspace's own token for the run, and can | |
| 152 | + | add and publish the workspace's crates. Give it to Cargo for the registry: | |
| 153 | + | ||
| 154 | + | ```yaml | |
| 155 | + | jobs: | |
| 156 | + | publish: | |
| 157 | + | runs-on: ubuntu-latest | |
| 158 | + | env: | |
| 159 | + | CARGO_REGISTRIES_ACME_INDEX: sparse+https://g1t.sh/-/cargo/acme/index/ | |
| 160 | + | CARGO_REGISTRIES_ACME_CREDENTIAL_PROVIDER: cargo:token | |
| 161 | + | CARGO_REGISTRIES_ACME_TOKEN: ${{ secrets.G1T_TOKEN }} | |
| 162 | + | steps: | |
| 163 | + | - uses: actions/checkout@v4 | |
| 164 | + | - run: cargo test | |
| 165 | + | - run: cargo publish --registry acme | |
| 166 | + | ``` | |
| 167 | + | ||
| 168 | + | `CARGO_REGISTRIES_ACME_INDEX` and `CARGO_REGISTRIES_ACME_CREDENTIAL_PROVIDER` | |
| 169 | + | are needed only when the project has no `.cargo/config.toml` naming the | |
| 170 | + | registry. | |
| 171 | + | ||
| 172 | + | ## Size | |
| 173 | + | ||
| 174 | + | A publish is one request with the `.crate` file inside it, and may hold at | |
| 175 | + | most 100 MB. Without the [g1t plan](/guides/usage-and-billing/#the-g1t-plan), | |
| 176 | + | a workspace's private packages may hold 500 MB and its public ones 10 GB, | |
| 177 | + | as for [container images](/guides/containers/#storage-and-pull-limits). | |
| 178 | + | A `.crate` file is stored once, by its content. | |
| 179 | + | ||
| 180 | + | ## Errors | |
| 181 | + | ||
| 182 | + | | Error | Means | | |
| 183 | + | | --- | --- | | |
| 184 | + | | `401` | No token, or a wrong or expired one. Run `cargo login --registry acme` with a g1t access token. | | |
| 185 | + | | `authenticated registries require a credential-provider to be configured` | The workspace has private crates, and Cargo has no provider for its token. Add `credential-provider = "cargo:token"` to the registry in `.cargo/config.toml`. | | |
| 186 | + | | `403` | Signed in, but your role or your token's scopes do not allow it, or the workspace is out of free package storage. The message says which. | | |
| 187 | + | | `404` | No such crate or version, or a private one you cannot see. | | |
| 188 | + | | `400` | The publish was refused: a name that is not valid or is taken, a version already published, or metadata Cargo did not send in full. The message says which. | | |
| 189 | + | | `413` | The publish is over 100 MB. Leave large files out with `exclude` or `include` in `Cargo.toml`, and check with `cargo package --list`. | |
| 4 | 4 | --- | |
| 5 | 5 | ||
| 6 | 6 | A workspace can publish packages to g1t and install them from it, beside | |
| 7 | − | the code they are built from: container images, npm packages, Composer | |
| 8 | − | packages and Go modules, with Cargo to follow. Each registry speaks its | |
| 9 | − | tool's own protocol, so `docker`, `npm`, `composer` and `go` work with | |
| 10 | − | nothing but a login and an address. Composer packages and Go modules are | |
| 7 | + | the code they are built from: container images, npm packages, Rust | |
| 8 | + | crates, Composer packages and Go modules. Each registry speaks its | |
| 9 | + | tool's own protocol, so `docker`, `npm`, `cargo`, `composer` and `go` | |
| 10 | + | work with nothing but a login and an address. Composer packages and Go modules are | |
| 11 | 11 | read from the workspace's repositories: there is nothing to upload. | |
| 12 | 12 | ||
| 13 | 13 | | Registry | Address | Guide | | |
| 14 | 14 | | --- | --- | --- | | |
| 15 | 15 | | Container images | `g1t.sh/<workspace>/<name>` | [Container images](/guides/containers/) | | |
| 16 | 16 | | npm | `https://g1t.sh/-/npm/`, for the scope `@<workspace>` | [npm](/guides/npm/) | | |
| 17 | + | | Cargo | `sparse+https://g1t.sh/-/cargo/<workspace>/index/`, a registry per workspace | [Cargo](/guides/cargo/) | | |
| 17 | 18 | | Composer | `https://g1t.sh/-/composer/<workspace>/`, from the workspace's repositories | [Composer](/guides/composer/) | | |
| 18 | 19 | | Go | `g1t.sh/<workspace>/<repo>`, straight from git | [Go modules](/guides/go/) | | |
| 19 | 20 | ||
| 31 | 32 | repository's name (`acme/web`, `acme/web/worker` for the repository | |
| 32 | 33 | `acme/web`) links it to that repository; so does the first publish of | |
| 33 | 34 | an npm package whose `package.json` `repository` is a g1t.sh repository | |
| 34 | − | of the workspace, or which is named like one (`@acme/web`). It then has | |
| 35 | − | the repository's visibility and [roles](/guides/access-and-roles/): | |
| 35 | + | of the workspace, or which is named like one (`@acme/web`), and of a | |
| 36 | + | crate whose `Cargo.toml` `repository` is one, or which is named like | |
| 37 | + | one. It then has the repository's visibility and [roles](/guides/access-and-roles/): | |
| 36 | 38 | ||
| 37 | 39 | | | Needs | | |
| 38 | 40 | | --- | --- | | |
| 92 | 94 | ## Events and the audit log | |
| 93 | 95 | ||
| 94 | 96 | Publishing a version, deleting a version and deleting a package are | |
| 95 | − | [audit log](/guides/audit-log/) entries, and the events | |
| 97 | + | [audit log](/guides/audit-log/) entries (so are deprecating an npm version | |
| 98 | + | and yanking or unyanking a crate version), and the events | |
| 96 | 99 | `package.published`, `package.version_deleted`, `package.deleted` and | |
| 97 | 100 | `package.visibility_changed`, which [webhooks](/guides/webhooks/) can be | |
| 98 | 101 | sent: a linked package's go to its repository's webhooks and its |
| 34 | 34 | }); | |
| 35 | 35 | }); | |
| 36 | 36 | ||
| 37 | + | test("a crate is added after .cargo/config.toml names its workspace's registry, with cargo login for private ones", () => { | |
| 38 | + | const pkg = { ecosystem: "cargo" as const, address: "g1t.sh/-/cargo/acme/http-client", name: "http-client", workspace: "acme" }; | |
| 39 | + | assert.deepEqual(installCommands(pkg, "0.3.1", "ada"), { | |
| 40 | + | registry: | |
| 41 | + | "mkdir -p .cargo && printf '[registries.acme]\\nindex = \"sparse+https://g1t.sh/-/cargo/acme/index/\"\\ncredential-provider = \"cargo:token\"\\n' >> .cargo/config.toml", | |
| 42 | + | login: "cargo login --registry acme", | |
| 43 | + | install: "cargo add http-client@0.3.1 --registry acme", | |
| 44 | + | }); | |
| 45 | + | assert.equal(installCommands(pkg, null, "ada").install, "cargo add http-client --registry acme"); | |
| 46 | + | }); | |
| 47 | + | ||
| 37 | 48 | test("a container image is pulled by its address and tag", () => { | |
| 38 | 49 | const pkg = { ecosystem: "container" as const, address: "g1t.sh/acme/web", name: "web", workspace: "acme" }; | |
| 39 | 50 | assert.deepEqual(installCommands(pkg, "latest", "ada"), { |
| 40 | 40 | ready: true, | |
| 41 | 41 | }, | |
| 42 | 42 | { | |
| 43 | + | ecosystem: "cargo", | |
| 44 | + | blurb: "Rust crates in a sparse registry of the workspace's own, published with cargo publish and added with cargo add.", | |
| 45 | + | start: "cargo publish --registry <workspace>", | |
| 46 | + | guide: "/guides/cargo/", | |
| 47 | + | ready: true, | |
| 48 | + | }, | |
| 49 | + | { | |
| 43 | 50 | ecosystem: "go", | |
| 44 | 51 | blurb: "Go modules fetched from the repositories themselves with go get, private ones with a token.", | |
| 45 | 52 | start: "go get g1t.sh/<workspace>/<repo>", | |
| 46 | 53 | guide: "/guides/go/", | |
| 47 | 54 | ready: true, | |
| 48 | − | }, | |
| 49 | − | { | |
| 50 | − | ecosystem: "cargo", | |
| 51 | − | blurb: "Rust crates in a registry of the workspace's own, published with cargo publish.", | |
| 52 | − | start: "cargo publish --registry <workspace>", | |
| 53 | − | guide: "/guides/packages/", | |
| 54 | − | ready: false, | |
| 55 | 55 | }, | |
| 56 | 56 | ]; | |
| 57 | 57 | ||
| 143 | 143 | install: `composer require ${pkg.name}${version ? `:${version}` : ""}`, | |
| 144 | 144 | }; | |
| 145 | 145 | case "cargo": | |
| 146 | + | // The workspace's registry (in .cargo/config.toml, needed for any | |
| 147 | + | // install), then the token a private crate also needs, which cargo | |
| 148 | + | // login asks for. | |
| 146 | 149 | return { | |
| 150 | + | registry: `mkdir -p .cargo && printf '[registries.${pkg.workspace}]\\nindex = "sparse+https://${host}/-/cargo/${pkg.workspace}/index/"\\ncredential-provider = "cargo:token"\\n' >> .cargo/config.toml`, | |
| 147 | 151 | login: `cargo login --registry ${pkg.workspace}`, | |
| 148 | − | install: `cargo add ${pkg.name} --registry ${pkg.workspace}${version ? ` --vers ${version}` : ""}`, | |
| 152 | + | install: `cargo add ${pkg.name}${version ? `@${version}` : ""} --registry ${pkg.workspace}`, | |
| 149 | 153 | }; | |
| 150 | 154 | case "go": | |
| 151 | 155 | return { |
| 47 | 47 | assert.equal(servicePath("/-/composer"), null); | |
| 48 | 48 | }); | |
| 49 | 49 | ||
| 50 | + | test("the Cargo registries go to the packages service", () => { | |
| 51 | + | for (const path of [ | |
| 52 | + | "/-/cargo/acme/index/config.json", | |
| 53 | + | "/-/cargo/acme/index/se/rd/serde", | |
| 54 | + | "/-/cargo/acme/index/3/a/abc", | |
| 55 | + | "/-/cargo/acme/api/v1/crates/new", | |
| 56 | + | "/-/cargo/acme/api/v1/crates/serde/1.0.0/download", | |
| 57 | + | "/-/cargo/acme/api/v1/crates/serde/1.0.0/yank", | |
| 58 | + | // A crate named like a git endpoint is still Cargo's. | |
| 59 | + | "/-/cargo/acme/index/in/fo/info/refs", | |
| 60 | + | ]) { | |
| 61 | + | assert.equal(servicePath(path), "packages", path); | |
| 62 | + | } | |
| 63 | + | assert.equal(servicePath("/-/cargo"), null); | |
| 64 | + | assert.equal(servicePath("/acme/-/cargo/x"), null); | |
| 65 | + | }); | |
| 66 | + | ||
| 50 | 67 | test("git goes to repos, and everything else is the site's", () => { | |
| 51 | 68 | assert.equal(servicePath("/acme/web.git/info/refs"), "git"); | |
| 52 | 69 | assert.equal(servicePath("/acme/web/git-receive-pack"), "git"); |
| 11 | 11 | const NPM_PATH = /^\/-\/npm(?:\/|$)/; | |
| 12 | 12 | /** The Composer registries: `/-/composer/<workspace>/`, one per workspace. */ | |
| 13 | 13 | const COMPOSER_PATH = /^\/-\/composer\//; | |
| 14 | + | /** The Cargo registries: `/-/cargo/<workspace>/`, a sparse index and its web API, one per workspace. */ | |
| 15 | + | const CARGO_PATH = /^\/-\/cargo\//; | |
| 14 | 16 | ||
| 15 | 17 | export type ServicePath = "git" | "packages" | null; | |
| 16 | 18 | ||
| 17 | 19 | export function servicePath(pathname: string): ServicePath { | |
| 18 | − | if (REGISTRY_PATH.test(pathname) || NPM_PATH.test(pathname) || COMPOSER_PATH.test(pathname)) return "packages"; | |
| 20 | + | if (REGISTRY_PATH.test(pathname) || NPM_PATH.test(pathname) || COMPOSER_PATH.test(pathname) || CARGO_PATH.test(pathname)) return "packages"; | |
| 19 | 21 | if (GIT_PATH.test(pathname)) return "git"; | |
| 20 | 22 | return null; | |
| 21 | 23 | } |
| 121 | 121 | {commands.registry && <CopyLine prompt text={commands.registry} />} | |
| 122 | 122 | {pkg.visibility === "private" && <CopyLine prompt text={commands.login} />} | |
| 123 | 123 | <CopyLine prompt text={commands.install} /> | |
| 124 | − | {commands.registry && pkg.visibility === "private" && ( | |
| 124 | + | {commands.registry && pkg.visibility === "private" && pkg.ecosystem === "cargo" && ( | |
| 125 | + | <p className="text-xs text-faint"> | |
| 126 | + | <code className="font-mono">cargo login</code> asks for an{" "} | |
| 127 | + | <Link to="/settings/tokens" className="text-muted hover:text-fg"> | |
| 128 | + | access token | |
| 129 | + | </Link>{" "} | |
| 130 | + | with <code className="font-mono">packages:read</code>. | |
| 131 | + | </p> | |
| 132 | + | )} | |
| 133 | + | {commands.registry && pkg.visibility === "private" && pkg.ecosystem !== "cargo" && ( | |
| 125 | 134 | <p className="text-xs text-faint"> | |
| 126 | 135 | Put an{" "} | |
| 127 | 136 | <Link to="/settings/tokens" className="text-muted hover:text-fg"> |
| 1 | + | //! What the Cargo registry needs that does not touch the network: crate | |
| 2 | + | //! names, where a crate's index file is, the publish body cargo sends, and | |
| 3 | + | //! the index line each version is. | |
| 4 | + | //! | |
| 5 | + | //! A version keeps its index entry (without `yanked`, which is a column of | |
| 6 | + | //! its own) as its metadata, made once when it is published; the index | |
| 7 | + | //! file is those entries, one JSON line each, oldest first. | |
| 8 | + | ||
| 9 | + | use serde_json::{Map, Value, json}; | |
| 10 | + | ||
| 11 | + | /// The longest crate name crates.io allows. | |
| 12 | + | pub const MAX_NAME: usize = 64; | |
| 13 | + | ||
| 14 | + | /// Names Windows keeps for devices: a crate named so could not be checked | |
| 15 | + | /// out of a git index, and cargo refuses them too. | |
| 16 | + | const RESERVED: [&str; 22] = [ | |
| 17 | + | "con", "prn", "aux", "nul", "com1", "com2", "com3", "com4", "com5", "com6", "com7", "com8", "com9", "lpt1", "lpt2", "lpt3", | |
| 18 | + | "lpt4", "lpt5", "lpt6", "lpt7", "lpt8", "lpt9", | |
| 19 | + | ]; | |
| 20 | + | ||
| 21 | + | /// Checks crates.io's rules for a crate name: ASCII letters, digits, `-` | |
| 22 | + | /// and `_`, starting with a letter, at most 64 characters. | |
| 23 | + | pub fn valid_name(name: &str) -> Result<(), String> { | |
| 24 | + | if name.is_empty() { | |
| 25 | + | return Err("The crate has no name.".to_owned()); | |
| 26 | + | } | |
| 27 | + | if name.len() > MAX_NAME { | |
| 28 | + | return Err(format!("A crate name is at most {MAX_NAME} characters.")); | |
| 29 | + | } | |
| 30 | + | if !name.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'-' || b == b'_') { | |
| 31 | + | return Err(format!("{name} is not a valid crate name: ASCII letters, digits, `-` and `_` only.")); | |
| 32 | + | } | |
| 33 | + | if !name.as_bytes()[0].is_ascii_alphabetic() { | |
| 34 | + | return Err(format!("{name} is not a valid crate name: it must start with a letter.")); | |
| 35 | + | } | |
| 36 | + | if RESERVED.contains(&name.to_ascii_lowercase().as_str()) { | |
| 37 | + | return Err(format!("{name} is a reserved name.")); | |
| 38 | + | } | |
| 39 | + | Ok(()) | |
| 40 | + | } | |
| 41 | + | ||
| 42 | + | /// The name two crates may not share: case and `-` against `_` aside, as | |
| 43 | + | /// crates.io decides whether a name is taken. | |
| 44 | + | pub fn folded(name: &str) -> String { | |
| 45 | + | name.to_ascii_lowercase().replace('_', "-") | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | /// Where a crate's file is in a sparse index, lowercased: `1/a`, `2/ab`, | |
| 49 | + | /// `3/a/abc`, `se/rd/serde`. | |
| 50 | + | pub fn index_path(name: &str) -> String { | |
| 51 | + | let name = name.to_ascii_lowercase(); | |
| 52 | + | match name.len() { | |
| 53 | + | 1 => format!("1/{name}"), | |
| 54 | + | 2 => format!("2/{name}"), | |
| 55 | + | 3 => format!("3/{}/{name}", &name[..1]), | |
| 56 | + | _ => format!("{}/{}/{name}", &name[..2], &name[2..4]), | |
| 57 | + | } | |
| 58 | + | } | |
| 59 | + | ||
| 60 | + | /// The crate an index path names, when it is where that crate's file is. | |
| 61 | + | /// Cargo asks with the lowercased path; any case is taken. | |
| 62 | + | pub fn name_of_index_path(path: &str) -> Option<String> { | |
| 63 | + | let name = path.rsplit('/').next()?; | |
| 64 | + | valid_name(name).ok()?; | |
| 65 | + | (index_path(name) == path.to_ascii_lowercase()).then(|| name.to_owned()) | |
| 66 | + | } | |
| 67 | + | ||
| 68 | + | /// One of the registry's endpoints, under `/-/cargo/<workspace>/`. | |
| 69 | + | #[derive(Clone, Debug, PartialEq, Eq)] | |
| 70 | + | pub enum CargoRoute { | |
| 71 | + | /// `index/config.json`. | |
| 72 | + | Config, | |
| 73 | + | /// `index/<path>`: a crate's index file. | |
| 74 | + | Index { name: String }, | |
| 75 | + | /// `api/v1/crates/new`. | |
| 76 | + | Publish, | |
| 77 | + | /// `api/v1/crates?q=`: `cargo search`. | |
| 78 | + | Search, | |
| 79 | + | Yank { name: String, version: String }, | |
| 80 | + | Unyank { name: String, version: String }, | |
| 81 | + | Download { name: String, version: String }, | |
| 82 | + | Owners { name: String }, | |
| 83 | + | } | |
| 84 | + | ||
| 85 | + | /// The workspace and endpoint a path is. Names are checked; versions are not. | |
| 86 | + | pub fn route(path: &str) -> Option<(String, CargoRoute)> { | |
| 87 | + | let rest = path.strip_prefix("/-/cargo/")?; | |
| 88 | + | let (workspace, rest) = rest.split_once('/')?; | |
| 89 | + | let workspace = workspace.to_ascii_lowercase(); | |
| 90 | + | if workspace.is_empty() { | |
| 91 | + | return None; | |
| 92 | + | } | |
| 93 | + | if let Some(index) = rest.strip_prefix("index/") { | |
| 94 | + | if index == "config.json" { | |
| 95 | + | return Some((workspace, CargoRoute::Config)); | |
| 96 | + | } | |
| 97 | + | return name_of_index_path(index).map(|name| (workspace, CargoRoute::Index { name })); | |
| 98 | + | } | |
| 99 | + | let crates = rest.strip_prefix("api/v1/crates")?; | |
| 100 | + | if crates.is_empty() || crates == "/" { | |
| 101 | + | return Some((workspace, CargoRoute::Search)); | |
| 102 | + | } | |
| 103 | + | let parts: Vec<&str> = crates.strip_prefix('/')?.split('/').collect(); | |
| 104 | + | let named = |name: &str| valid_name(name).is_ok().then(|| name.to_owned()); | |
| 105 | + | let route = match parts.as_slice() { | |
| 106 | + | ["new"] => CargoRoute::Publish, | |
| 107 | + | [name, "owners"] => CargoRoute::Owners { name: named(name)? }, | |
| 108 | + | [name, version, action] if !version.is_empty() => { | |
| 109 | + | let (name, version) = (named(name)?, (*version).to_owned()); | |
| 110 | + | match *action { | |
| 111 | + | "yank" => CargoRoute::Yank { name, version }, | |
| 112 | + | "unyank" => CargoRoute::Unyank { name, version }, | |
| 113 | + | "download" => CargoRoute::Download { name, version }, | |
| 114 | + | _ => return None, | |
| 115 | + | } | |
| 116 | + | } | |
| 117 | + | _ => return None, | |
| 118 | + | }; | |
| 119 | + | Some((workspace, route)) | |
| 120 | + | } | |
| 121 | + | ||
| 122 | + | /// The two parts of `cargo publish`'s body: a little-endian `u32` length | |
| 123 | + | /// and the JSON metadata, then a `u32` length and the `.crate` file. | |
| 124 | + | pub fn parse_publish(body: &[u8]) -> Result<(Value, &[u8]), String> { | |
| 125 | + | let take = |at: usize| -> Result<(usize, usize), String> { | |
| 126 | + | let length = body.get(at..at + 4).ok_or("The publish ends early.")?; | |
| 127 | + | let length = u32::from_le_bytes([length[0], length[1], length[2], length[3]]) as usize; | |
| 128 | + | let start = at + 4; | |
| 129 | + | if body.len() < start + length { | |
| 130 | + | return Err("The publish ends early.".to_owned()); | |
| 131 | + | } | |
| 132 | + | Ok((start, start + length)) | |
| 133 | + | }; | |
| 134 | + | let (json_start, json_end) = take(0)?; | |
| 135 | + | let metadata: Value = serde_json::from_slice(&body[json_start..json_end]).map_err(|_| "The publish's metadata is not JSON.")?; | |
| 136 | + | if !metadata.is_object() { | |
| 137 | + | return Err("The publish's metadata is not a JSON object.".to_owned()); | |
| 138 | + | } | |
| 139 | + | let (crate_start, crate_end) = take(json_end)?; | |
| 140 | + | if crate_end != body.len() { | |
| 141 | + | return Err("The publish has bytes after the .crate file.".to_owned()); | |
| 142 | + | } | |
| 143 | + | Ok((metadata, &body[crate_start..crate_end])) | |
| 144 | + | } | |
| 145 | + | ||
| 146 | + | /// The version without its build metadata: `1.0.0+abc` is `1.0.0`. Two | |
| 147 | + | /// versions that differ only in it may not both be published. | |
| 148 | + | pub fn without_build(version: &str) -> &str { | |
| 149 | + | version.split_once('+').map_or(version, |(core, _)| core) | |
| 150 | + | } | |
| 151 | + | ||
| 152 | + | /// Whether a feature's list uses the syntax only newer cargo reads | |
| 153 | + | /// (`dep:name`, `name?/feature`), so it belongs in `features2`. | |
| 154 | + | fn new_syntax(values: &Value) -> bool { | |
| 155 | + | values | |
| 156 | + | .as_array() | |
| 157 | + | .is_some_and(|values| values.iter().filter_map(Value::as_str).any(|v| v.starts_with("dep:") || v.contains("?/"))) | |
| 158 | + | } | |
| 159 | + | ||
| 160 | + | /// A dependency as the index lists it, from how `cargo publish` sends it: | |
| 161 | + | /// `version_req` is `req`, and a renamed one (`explicit_name_in_toml`) is | |
| 162 | + | /// listed by its new name with `package` naming the crate. | |
| 163 | + | fn index_dependency(sent: &Value) -> Result<Value, String> { | |
| 164 | + | let name = sent["name"].as_str().ok_or("A dependency has no name.")?; | |
| 165 | + | let req = sent["version_req"].as_str().ok_or_else(|| format!("The dependency {name} has no version requirement."))?; | |
| 166 | + | let mut dep = Map::new(); | |
| 167 | + | let renamed = sent["explicit_name_in_toml"].as_str().filter(|n| !n.is_empty()); | |
| 168 | + | dep.insert("name".into(), json!(renamed.unwrap_or(name))); | |
| 169 | + | dep.insert("req".into(), json!(req)); | |
| 170 | + | dep.insert("features".into(), sent.get("features").filter(|f| f.is_array()).cloned().unwrap_or_else(|| json!([]))); | |
| 171 | + | dep.insert("optional".into(), json!(sent["optional"].as_bool().unwrap_or(false))); | |
| 172 | + | dep.insert("default_features".into(), json!(sent["default_features"].as_bool().unwrap_or(true))); | |
| 173 | + | dep.insert("target".into(), sent.get("target").filter(|t| t.is_string()).cloned().unwrap_or(Value::Null)); | |
| 174 | + | dep.insert("kind".into(), json!(sent["kind"].as_str().unwrap_or("normal"))); | |
| 175 | + | if let Some(registry) = sent["registry"].as_str() { | |
| 176 | + | dep.insert("registry".into(), json!(registry)); | |
| 177 | + | } | |
| 178 | + | if renamed.is_some() { | |
| 179 | + | dep.insert("package".into(), json!(name)); | |
| 180 | + | } | |
| 181 | + | Ok(Value::Object(dep)) | |
| 182 | + | } | |
| 183 | + | ||
| 184 | + | /// A version's index entry, from the publish's metadata and the `.crate` | |
| 185 | + | /// file's SHA-256 in hex. `yanked` is left out: it is added on each read. | |
| 186 | + | pub fn index_entry(metadata: &Value, cksum: &str) -> Result<Value, String> { | |
| 187 | + | let name = metadata["name"].as_str().ok_or("The publish names no crate.")?; | |
| 188 | + | let vers = metadata["vers"].as_str().ok_or("The publish names no version.")?; | |
| 189 | + | let deps = match &metadata["deps"] { | |
| 190 | + | Value::Null => Vec::new(), | |
| 191 | + | Value::Array(deps) => deps.iter().map(index_dependency).collect::<Result<Vec<_>, _>>()?, | |
| 192 | + | _ => return Err("The publish's dependencies are not a list.".to_owned()), | |
| 193 | + | }; | |
| 194 | + | let (mut features, mut features2) = (Map::new(), Map::new()); | |
| 195 | + | match &metadata["features"] { | |
| 196 | + | Value::Null => {} | |
| 197 | + | Value::Object(sent) => { | |
| 198 | + | for (feature, values) in sent { | |
| 199 | + | if !values.is_array() { | |
| 200 | + | return Err(format!("The feature {feature} is not a list.")); | |
| 201 | + | } | |
| 202 | + | let into = if new_syntax(values) { &mut features2 } else { &mut features }; | |
| 203 | + | into.insert(feature.clone(), values.clone()); | |
| 204 | + | } | |
| 205 | + | } | |
| 206 | + | _ => return Err("The publish's features are not an object.".to_owned()), | |
| 207 | + | } | |
| 208 | + | let mut entry = Map::new(); | |
| 209 | + | entry.insert("name".into(), json!(name)); | |
| 210 | + | entry.insert("vers".into(), json!(vers)); | |
| 211 | + | entry.insert("deps".into(), Value::Array(deps)); | |
| 212 | + | entry.insert("cksum".into(), json!(cksum)); | |
| 213 | + | entry.insert("features".into(), Value::Object(features)); | |
| 214 | + | entry.insert("links".into(), metadata.get("links").filter(|l| l.is_string()).cloned().unwrap_or(Value::Null)); | |
| 215 | + | if !features2.is_empty() { | |
| 216 | + | entry.insert("features2".into(), Value::Object(features2)); | |
| 217 | + | entry.insert("v".into(), json!(2)); | |
| 218 | + | } | |
| 219 | + | if let Some(rust_version) = metadata["rust_version"].as_str() { | |
| 220 | + | entry.insert("rust_version".into(), json!(rust_version)); | |
| 221 | + | } | |
| 222 | + | Ok(Value::Object(entry)) | |
| 223 | + | } | |
| 224 | + | ||
| 225 | + | /// One line of a crate's index file: the version's entry as kept, with | |
| 226 | + | /// whether it is yanked. | |
| 227 | + | pub fn index_line(entry: &Value, yanked: bool) -> String { | |
| 228 | + | let mut entry = match entry { | |
| 229 | + | Value::Object(map) => map.clone(), | |
| 230 | + | _ => Map::new(), | |
| 231 | + | }; | |
| 232 | + | entry.insert("yanked".into(), json!(yanked)); | |
| 233 | + | Value::Object(entry).to_string() | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | /// What `index/config.json` says: where crates are downloaded from and the | |
| 237 | + | /// web API is, under `base` (`https://g1t.sh/-/cargo/acme`), and whether | |
| 238 | + | /// cargo must send its token for every request. | |
| 239 | + | pub fn config(base: &str, auth_required: bool) -> Value { | |
| 240 | + | json!({ "dl": format!("{base}/api/v1/crates"), "api": base, "auth-required": auth_required }) | |
| 241 | + | } | |
| 242 | + | ||
| 243 | + | /// The token in cargo's `Authorization` header, which is the token alone; | |
| 244 | + | /// `Bearer <token>` is taken too. | |
| 245 | + | pub fn token(header: &str) -> Option<&str> { | |
| 246 | + | let header = header.trim(); | |
| 247 | + | let token = match header.split_once(' ') { | |
| 248 | + | Some((scheme, rest)) if scheme.eq_ignore_ascii_case("bearer") => rest.trim(), | |
| 249 | + | Some(_) => return None, | |
| 250 | + | None => header, | |
| 251 | + | }; | |
| 252 | + | (!token.is_empty()).then_some(token) | |
| 253 | + | } | |
| 254 | + | ||
| 255 | + | #[cfg(test)] | |
| 256 | + | mod tests { | |
| 257 | + | use super::*; | |
| 258 | + | ||
| 259 | + | #[test] | |
| 260 | + | fn names_follow_crates_io_rules() { | |
| 261 | + | for good in ["serde", "serde_json", "tokio-util", "a", "A1", "Inflector", &"a".repeat(64)] { | |
| 262 | + | assert!(valid_name(good).is_ok(), "{good}"); | |
| 263 | + | } | |
| 264 | + | for bad in ["", "1abc", "-abc", "_abc", "a.b", "a b", "naïve", "a/b", "nul", "COM1", &"a".repeat(65)] { | |
| 265 | + | assert!(valid_name(bad).is_err(), "{bad}"); | |
| 266 | + | } | |
| 267 | + | assert_eq!(folded("Serde_Json"), folded("serde-json"), "case and -/_ are one name"); | |
| 268 | + | assert_ne!(folded("serde"), folded("serde-json")); | |
| 269 | + | } | |
| 270 | + | ||
| 271 | + | #[test] | |
| 272 | + | fn index_paths_are_the_standard_sparse_ones_in_lowercase() { | |
| 273 | + | assert_eq!(index_path("a"), "1/a"); | |
| 274 | + | assert_eq!(index_path("ab"), "2/ab"); | |
| 275 | + | assert_eq!(index_path("abc"), "3/a/abc"); | |
| 276 | + | assert_eq!(index_path("serde"), "se/rd/serde"); | |
| 277 | + | assert_eq!(index_path("Inflector"), "in/fl/inflector"); | |
| 278 | + | assert_eq!(index_path("cargo"), "ca/rg/cargo"); | |
| 279 | + | assert_eq!(name_of_index_path("se/rd/serde").as_deref(), Some("serde")); | |
| 280 | + | assert_eq!(name_of_index_path("3/a/abc").as_deref(), Some("abc")); | |
| 281 | + | assert_eq!(name_of_index_path("in/fl/Inflector").as_deref(), Some("Inflector")); | |
| 282 | + | assert_eq!(name_of_index_path("xx/rd/serde"), None, "not where serde's file is"); | |
| 283 | + | assert_eq!(name_of_index_path("3/b/abc"), None); | |
| 284 | + | assert_eq!(name_of_index_path("1/ab"), None); | |
| 285 | + | assert_eq!(name_of_index_path("se/rd/se.de"), None); | |
| 286 | + | } | |
| 287 | + | ||
| 288 | + | #[test] | |
| 289 | + | fn every_endpoint_is_routed() { | |
| 290 | + | let at = |route: CargoRoute| Some(("acme".to_owned(), route)); | |
| 291 | + | let nv = |name: &str, version: &str| (name.to_owned(), version.to_owned()); | |
| 292 | + | assert_eq!(route("/-/cargo/acme/index/config.json"), at(CargoRoute::Config)); | |
| 293 | + | assert_eq!(route("/-/cargo/Acme/index/se/rd/serde"), at(CargoRoute::Index { name: "serde".into() })); | |
| 294 | + | assert_eq!(route("/-/cargo/acme/index/1/a"), at(CargoRoute::Index { name: "a".into() })); | |
| 295 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates/new"), at(CargoRoute::Publish)); | |
| 296 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates"), at(CargoRoute::Search)); | |
| 297 | + | let (name, version) = nv("serde", "1.0.0"); | |
| 298 | + | assert_eq!( | |
| 299 | + | route("/-/cargo/acme/api/v1/crates/serde/1.0.0/yank"), | |
| 300 | + | at(CargoRoute::Yank { name: name.clone(), version: version.clone() }) | |
| 301 | + | ); | |
| 302 | + | assert_eq!( | |
| 303 | + | route("/-/cargo/acme/api/v1/crates/serde/1.0.0/unyank"), | |
| 304 | + | at(CargoRoute::Unyank { name: name.clone(), version: version.clone() }) | |
| 305 | + | ); | |
| 306 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates/serde/1.0.0/download"), at(CargoRoute::Download { name, version })); | |
| 307 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates/serde/owners"), at(CargoRoute::Owners { name: "serde".into() })); | |
| 308 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates/serde/1.0.0/other"), None); | |
| 309 | + | assert_eq!(route("/-/cargo/acme/api/v1/crates/se.de/1.0.0/download"), None); | |
| 310 | + | assert_eq!(route("/-/cargo/acme/index/xx/yy/serde"), None); | |
| 311 | + | assert_eq!(route("/-/cargo/acme"), None); | |
| 312 | + | assert_eq!(route("/-/npm/@acme/web"), None); | |
| 313 | + | } | |
| 314 | + | ||
| 315 | + | fn body(metadata: &[u8], krate: &[u8]) -> Vec<u8> { | |
| 316 | + | let mut body = (metadata.len() as u32).to_le_bytes().to_vec(); | |
| 317 | + | body.extend_from_slice(metadata); | |
| 318 | + | body.extend_from_slice(&(krate.len() as u32).to_le_bytes()); | |
| 319 | + | body.extend_from_slice(krate); | |
| 320 | + | body | |
| 321 | + | } | |
| 322 | + | ||
| 323 | + | #[test] | |
| 324 | + | fn the_publish_body_is_two_length_prefixed_parts() { | |
| 325 | + | let sent = body(br#"{"name":"web","vers":"1.0.0"}"#, b"\x1f\x8bcrate"); | |
| 326 | + | let (metadata, krate) = parse_publish(&sent).unwrap(); | |
| 327 | + | assert_eq!(metadata["name"], "web"); | |
| 328 | + | assert_eq!(krate, b"\x1f\x8bcrate"); | |
| 329 | + | let empty = body(br#"{"name":"web"}"#, b""); | |
| 330 | + | assert_eq!(parse_publish(&empty).unwrap().1, b""); | |
| 331 | + | ||
| 332 | + | assert!(parse_publish(b"").is_err()); | |
| 333 | + | assert!(parse_publish(&[10, 0, 0, 0, b'{']).is_err(), "shorter than it says"); | |
| 334 | + | assert!(parse_publish(&body(b"not json", b"x")).is_err()); | |
| 335 | + | assert!(parse_publish(&body(b"[1]", b"x")).is_err()); | |
| 336 | + | let mut cut = sent.clone(); | |
| 337 | + | cut.pop(); | |
| 338 | + | assert!(parse_publish(&cut).is_err(), "the crate is cut short"); | |
| 339 | + | let mut long = sent.clone(); | |
| 340 | + | long.push(0); | |
| 341 | + | assert!(parse_publish(&long).is_err(), "bytes after the crate"); | |
| 342 | + | let no_crate = br#"{"name":"web"}"#; | |
| 343 | + | let mut half = (no_crate.len() as u32).to_le_bytes().to_vec(); | |
| 344 | + | half.extend_from_slice(no_crate); | |
| 345 | + | assert!(parse_publish(&half).is_err(), "no crate length"); | |
| 346 | + | } | |
| 347 | + | ||
| 348 | + | #[test] | |
| 349 | + | fn index_lines_are_cargos_shape() { | |
| 350 | + | let metadata = json!({ | |
| 351 | + | "name": "Web", | |
| 352 | + | "vers": "1.2.0", | |
| 353 | + | "deps": [ | |
| 354 | + | { "name": "serde", "version_req": "^1", "features": ["derive"], "optional": false, "default_features": true, "target": null, "kind": "normal", "registry": "https://github.com/rust-lang/crates.io-index" }, | |
| 355 | + | { "name": "core-lib", "version_req": "=0.3.0", "features": [], "optional": true, "default_features": false, "target": "cfg(unix)", "kind": "normal", "explicit_name_in_toml": "core" }, | |
| 356 | + | { "name": "tempfile", "version_req": "^3", "kind": "dev" } | |
| 357 | + | ], | |
| 358 | + | "features": { "default": ["std"], "std": [], "derive": ["dep:core", "serde?/derive"] }, | |
| 359 | + | "links": null, | |
| 360 | + | "rust_version": "1.75", | |
| 361 | + | "description": "kept elsewhere", | |
| 362 | + | }); | |
| 363 | + | let entry = index_entry(&metadata, "ab12").unwrap(); | |
| 364 | + | let line: Value = serde_json::from_str(&index_line(&entry, false)).unwrap(); | |
| 365 | + | assert_eq!(line["name"], "Web"); | |
| 366 | + | assert_eq!(line["vers"], "1.2.0"); | |
| 367 | + | assert_eq!(line["cksum"], "ab12"); | |
| 368 | + | assert_eq!(line["yanked"], false); | |
| 369 | + | assert_eq!(line["links"], Value::Null); | |
| 370 | + | assert_eq!(line["rust_version"], "1.75"); | |
| 371 | + | assert!(line.get("description").is_none(), "the index holds what resolving needs"); | |
| 372 | + | assert_eq!(line["features"], json!({ "default": ["std"], "std": [] })); | |
| 373 | + | assert_eq!(line["features2"], json!({ "derive": ["dep:core", "serde?/derive"] })); | |
| 374 | + | assert_eq!(line["v"], 2); | |
| 375 | + | ||
| 376 | + | let deps = line["deps"].as_array().unwrap(); | |
| 377 | + | assert_eq!(deps[0]["name"], "serde"); | |
| 378 | + | assert_eq!(deps[0]["req"], "^1"); | |
| 379 | + | assert_eq!(deps[0]["features"], json!(["derive"])); | |
| 380 | + | assert_eq!(deps[0]["registry"], "https://github.com/rust-lang/crates.io-index"); | |
| 381 | + | assert!(deps[0].get("package").is_none()); | |
| 382 | + | assert_eq!(deps[1]["name"], "core", "a renamed dependency by its new name"); | |
| 383 | + | assert_eq!(deps[1]["package"], "core-lib"); | |
| 384 | + | assert_eq!(deps[1]["optional"], true); | |
| 385 | + | assert_eq!(deps[1]["default_features"], false); | |
| 386 | + | assert_eq!(deps[1]["target"], "cfg(unix)"); | |
| 387 | + | assert!(deps[1].get("registry").is_none(), "this registry"); | |
| 388 | + | assert_eq!(deps[2]["kind"], "dev"); | |
| 389 | + | assert_eq!(deps[2]["features"], json!([])); | |
| 390 | + | assert_eq!(deps[2]["default_features"], true); | |
| 391 | + | ||
| 392 | + | let yanked: Value = serde_json::from_str(&index_line(&entry, true)).unwrap(); | |
| 393 | + | assert_eq!(yanked["yanked"], true); | |
| 394 | + | assert!(!index_line(&entry, true).contains('\n'), "one line"); | |
| 395 | + | } | |
| 396 | + | ||
| 397 | + | #[test] | |
| 398 | + | fn a_crate_without_new_feature_syntax_is_index_version_1() { | |
| 399 | + | let entry = index_entry(&json!({ "name": "a", "vers": "0.1.0", "deps": [], "features": { "x": ["a/b"] }, "links": "z" }), "00").unwrap(); | |
| 400 | + | assert!(entry.get("v").is_none()); | |
| 401 | + | assert!(entry.get("features2").is_none()); | |
| 402 | + | assert_eq!(entry["links"], "z"); | |
| 403 | + | assert!(index_entry(&json!({ "vers": "0.1.0" }), "00").is_err()); | |
| 404 | + | assert!(index_entry(&json!({ "name": "a", "vers": "0.1.0", "deps": [{ "name": "b" }] }), "00").is_err()); | |
| 405 | + | assert!(index_entry(&json!({ "name": "a", "vers": "0.1.0", "features": { "x": "y" } }), "00").is_err()); | |
| 406 | + | } | |
| 407 | + | ||
| 408 | + | #[test] | |
| 409 | + | fn config_names_the_download_and_api_addresses() { | |
| 410 | + | let config = config("https://g1t.sh/-/cargo/acme", true); | |
| 411 | + | assert_eq!(config["dl"], "https://g1t.sh/-/cargo/acme/api/v1/crates"); | |
| 412 | + | assert_eq!(config["api"], "https://g1t.sh/-/cargo/acme"); | |
| 413 | + | assert_eq!(config["auth-required"], true); | |
| 414 | + | } | |
| 415 | + | ||
| 416 | + | #[test] | |
| 417 | + | fn the_token_is_the_whole_header() { | |
| 418 | + | assert_eq!(token("g1t_abc"), Some("g1t_abc")); | |
| 419 | + | assert_eq!(token(" g1t_abc "), Some("g1t_abc")); | |
| 420 | + | assert_eq!(token("Bearer g1t_abc"), Some("g1t_abc")); | |
| 421 | + | assert_eq!(token("Basic YTpi"), None); | |
| 422 | + | assert_eq!(token(""), None); | |
| 423 | + | assert_eq!(without_build("1.0.0+build.1"), "1.0.0"); | |
| 424 | + | assert_eq!(without_build("1.0.0-rc.1"), "1.0.0-rc.1"); | |
| 425 | + | } | |
| 426 | + | } |
| 1 | + | //! The Cargo registry: `g1t.sh/-/cargo/<workspace>/`, a sparse registry | |
| 2 | + | //! for each workspace. `.cargo/config.toml` names it, and `cargo login` | |
| 3 | + | //! keeps a g1t token for it: | |
| 4 | + | //! | |
| 5 | + | //! ```toml | |
| 6 | + | //! [registries.acme] | |
| 7 | + | //! index = "sparse+https://g1t.sh/-/cargo/acme/index/" | |
| 8 | + | //! ``` | |
| 9 | + | //! | |
| 10 | + | //! Cargo sends the token as the whole `Authorization` header, with no | |
| 11 | + | //! scheme. The index is made from the versions on each read; a `.crate` | |
| 12 | + | //! is stored once, by its SHA-256, which is also its index `cksum`. | |
| 13 | + | ||
| 14 | + | use g1t_contracts::User; | |
| 15 | + | use g1t_contracts::audit::AuditActor; | |
| 16 | + | use g1t_contracts::events::PackageEvent; | |
| 17 | + | use g1t_contracts::new_id; | |
| 18 | + | use g1t_kit::now_ms; | |
| 19 | + | use serde_json::{Value, json}; | |
| 20 | + | use worker::{Context, Headers, Method, Request, Response, ResponseBody, Result, Url}; | |
| 21 | + | ||
| 22 | + | use crate::access::{self, Action}; | |
| 23 | + | use crate::cargo::{self, CargoRoute}; | |
| 24 | + | use crate::db::{NewFile, NewVersion, PackageRow, VersionRow}; | |
| 25 | + | use crate::digest::Digest; | |
| 26 | + | use crate::npm; | |
| 27 | + | use crate::oci::{Credentials, origin, published_by}; | |
| 28 | + | use crate::store::BlobStore; | |
| 29 | + | use crate::{Caller, Packages, TargetOf, token}; | |
| 30 | + | ||
| 31 | + | const CARGO: &str = "cargo"; | |
| 32 | + | /// The most versions an index file lists. | |
| 33 | + | const MAX_VERSIONS: u32 = 5000; | |
| 34 | + | /// The longest README kept for a crate's page. | |
| 35 | + | const MAX_README_BYTES: usize = 1024 * 1024; | |
| 36 | + | /// The most crates `cargo search` is answered with. | |
| 37 | + | const MAX_SEARCH: u32 = 100; | |
| 38 | + | const DOCS: &str = "https://docs.g1t.sh/guides/cargo/"; | |
| 39 | + | const TOKENS: &str = "https://g1t.sh/settings/tokens"; | |
| 40 | + | ||
| 41 | + | /// Cargo's error shape: `{"errors": [{"detail": "..."}]}`, which it prints. | |
| 42 | + | /// A 401 says where to get a token, which cargo shows beside its own hint. | |
| 43 | + | fn error(status: u16, message: impl Into<String>) -> Result<Response> { | |
| 44 | + | let mut response = Response::from_json(&json!({ "errors": [{ "detail": message.into() }] }))?.with_status(status); | |
| 45 | + | if status == 401 { | |
| 46 | + | response.headers_mut().set("www-authenticate", &format!("Cargo login_url=\"{TOKENS}\""))?; | |
| 47 | + | } | |
| 48 | + | Ok(response) | |
| 49 | + | } | |
| 50 | + | ||
| 51 | + | fn ok() -> Result<Response> { | |
| 52 | + | Response::from_json(&json!({ "ok": true })) | |
| 53 | + | } | |
| 54 | + | ||
| 55 | + | fn not_found() -> Result<Response> { | |
| 56 | + | error(404, "Not found: no such crate, or you cannot see it. Private crates need a token: cargo login --registry <workspace>.") | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | fn sign_in(workspace: &str) -> String { | |
| 60 | + | format!("Sign in to use this registry: cargo login --registry {workspace}, with a g1t access token from {TOKENS}") | |
| 61 | + | } | |
| 62 | + | ||
| 63 | + | impl Packages { | |
| 64 | + | /// Answers a Cargo request. | |
| 65 | + | pub async fn cargo(&self, request: Request, ctx: &Context) -> Result<Response> { | |
| 66 | + | let url = request.url()?; | |
| 67 | + | let Some((workspace, route)) = cargo::route(url.path()) else { | |
| 68 | + | return error(404, "There is nothing at this address."); | |
| 69 | + | }; | |
| 70 | + | match self.cargo_route(request, &url, &workspace, route, ctx).await { | |
| 71 | + | Ok(response) => Ok(response), | |
| 72 | + | Err(problem) => { | |
| 73 | + | worker::console_error!("packages: cargo {}: {problem}", url.path()); | |
| 74 | + | error(500, "Something went wrong on our side. Try again in a moment.") | |
| 75 | + | } | |
| 76 | + | } | |
| 77 | + | } | |
| 78 | + | ||
| 79 | + | /// Who the request is from: cargo's bare token, a `Bearer` one, or | |
| 80 | + | /// Basic credentials with a g1t token as the password. | |
| 81 | + | async fn cargo_credentials(&self, request: &Request) -> Result<Credentials> { | |
| 82 | + | let Some(header) = request.headers().get("authorization")? else { | |
| 83 | + | return Ok(Credentials::None); | |
| 84 | + | }; | |
| 85 | + | let viewer = if let Some((username, secret)) = token::basic(&header) { | |
| 86 | + | self.viewer_for(&username, &secret).await? | |
| 87 | + | } else if let Some(token) = cargo::token(&header) { | |
| 88 | + | self.viewer_for("token", token).await? | |
| 89 | + | } else { | |
| 90 | + | None | |
| 91 | + | }; | |
| 92 | + | Ok(match viewer { | |
| 93 | + | Some(user) => Credentials::Viewer(Some(user)), | |
| 94 | + | None => Credentials::Bad, | |
| 95 | + | }) | |
| 96 | + | } | |
| 97 | + | ||
| 98 | + | async fn cargo_route(&self, mut request: Request, url: &Url, workspace: &str, route: CargoRoute, ctx: &Context) -> Result<Response> { | |
| 99 | + | let method = request.method(); | |
| 100 | + | let credentials = self.cargo_credentials(&request).await?; | |
| 101 | + | if matches!(method, Method::Get | Method::Head) | |
| 102 | + | && let Some(refused) = self.limited(&request, &credentials, &format!("a token (cargo login --registry {workspace})")).await? | |
| 103 | + | { | |
| 104 | + | return Ok(refused); | |
| 105 | + | } | |
| 106 | + | let viewer = match credentials { | |
| 107 | + | Credentials::Viewer(viewer) => viewer, | |
| 108 | + | Credentials::None => None, | |
| 109 | + | Credentials::Token(_) | Credentials::Bad => { | |
| 110 | + | return error( | |
| 111 | + | 401, | |
| 112 | + | format!("The token is not right, or has expired. Make an access token at {TOKENS}, then: cargo login --registry {workspace}"), | |
| 113 | + | ); | |
| 114 | + | } | |
| 115 | + | }; | |
| 116 | + | let viewer = viewer.as_ref(); | |
| 117 | + | let read = matches!(method, Method::Get | Method::Head); | |
| 118 | + | match route { | |
| 119 | + | CargoRoute::Config if read => self.cargo_config(url, workspace, viewer).await, | |
| 120 | + | CargoRoute::Index { name } if read => self.cargo_index(&request, workspace, &name, viewer).await, | |
| 121 | + | CargoRoute::Download { name, version } if read => self.crate_download(workspace, &name, &version, viewer, method == Method::Head, ctx).await, | |
| 122 | + | CargoRoute::Search if read => self.cargo_search(url, workspace, viewer).await, | |
| 123 | + | CargoRoute::Publish if method == Method::Put => self.cargo_publish(&mut request, workspace, viewer).await, | |
| 124 | + | CargoRoute::Yank { name, version } if method == Method::Delete => self.cargo_yank(workspace, &name, &version, true, viewer).await, | |
| 125 | + | CargoRoute::Unyank { name, version } if method == Method::Put => self.cargo_yank(workspace, &name, &version, false, viewer).await, | |
| 126 | + | CargoRoute::Owners { .. } => error( | |
| 127 | + | 400, | |
| 128 | + | "Crate owners are not kept here: who may publish a crate is decided by its repository's roles, or the workspace's. See https://docs.g1t.sh/guides/packages/#who-can-see-and-publish-a-package", | |
| 129 | + | ), | |
| 130 | + | _ => error(405, "Not a method this address takes."), | |
| 131 | + | } | |
| 132 | + | } | |
| 133 | + | ||
| 134 | + | /// The crate, by its name in any case, if its workspace is not deleted. | |
| 135 | + | async fn crate_package(&self, workspace: &str, name: &str) -> Result<Option<PackageRow>> { | |
| 136 | + | Ok(self.db.package_any_case(workspace, CARGO, name).await?.filter(|p| !p.hidden())) | |
| 137 | + | } | |
| 138 | + | ||
| 139 | + | /// Whether `viewer` may `action` the crate, as the answer when not: 401 | |
| 140 | + | /// for someone not signed in who may not read it, 404 for anyone else | |
| 141 | + | /// who may not read it, and 403 with the reason for one who may. | |
| 142 | + | fn cargo_check(&self, viewer: Option<&User>, package: &PackageRow, action: Action) -> Option<Result<Response>> { | |
| 143 | + | let target = TargetOf::package(package); | |
| 144 | + | let decision = access::decide(viewer, &target.view(), action); | |
| 145 | + | if decision.allowed { | |
| 146 | + | return None; | |
| 147 | + | } | |
| 148 | + | let readable = action != Action::Pull && access::decide(viewer, &target.view(), Action::Pull).allowed; | |
| 149 | + | if !readable && viewer.is_none() { | |
| 150 | + | return Some(error(401, sign_in(&package.workspace))); | |
| 151 | + | } | |
| 152 | + | if !readable { | |
| 153 | + | return Some(not_found()); | |
| 154 | + | } | |
| 155 | + | Some(error(403, decision.reason.unwrap_or_else(|| "Not allowed.".to_owned()))) | |
| 156 | + | } | |
| 157 | + | ||
| 158 | + | /// `index/config.json`. Signed in, cargo is told to send its token with | |
| 159 | + | /// every request. Anonymous requests to a workspace that has private | |
| 160 | + | /// crates get a 401, which makes cargo ask again with its token; one | |
| 161 | + | /// with only public crates is open to anyone. | |
| 162 | + | async fn cargo_config(&self, url: &Url, workspace: &str, viewer: Option<&User>) -> Result<Response> { | |
| 163 | + | if viewer.is_none() && self.db.has_private(workspace, CARGO).await? { | |
| 164 | + | return error(401, sign_in(workspace)); | |
| 165 | + | } | |
| 166 | + | let base = format!("{}/-/cargo/{workspace}", origin(url)); | |
| 167 | + | let mut response = Response::from_json(&cargo::config(&base, viewer.is_some()))?; | |
| 168 | + | response.headers_mut().set("cache-control", "no-cache")?; | |
| 169 | + | Ok(response) | |
| 170 | + | } | |
| 171 | + | ||
| 172 | + | /// A crate's index file: one line per version, oldest first. | |
| 173 | + | async fn cargo_index(&self, request: &Request, workspace: &str, name: &str, viewer: Option<&User>) -> Result<Response> { | |
| 174 | + | let Some(package) = self.crate_package(workspace, name).await? else { | |
| 175 | + | return not_found(); | |
| 176 | + | }; | |
| 177 | + | if let Some(refusal) = self.cargo_check(viewer, &package, Action::Pull) { | |
| 178 | + | return refusal; | |
| 179 | + | } | |
| 180 | + | let mut versions = self.db.versions(&package.id, MAX_VERSIONS).await?; | |
| 181 | + | if versions.is_empty() { | |
| 182 | + | return not_found(); | |
| 183 | + | } | |
| 184 | + | versions.reverse(); | |
| 185 | + | let mut body = String::new(); | |
| 186 | + | for version in &versions { | |
| 187 | + | body.push_str(&cargo::index_line(&version.meta(), version.is_yanked())); | |
| 188 | + | body.push('\n'); | |
| 189 | + | } | |
| 190 | + | let etag = format!("\"{}\"", &Digest::of(body.as_bytes()).hex()[..32]); | |
| 191 | + | let headers = Headers::new(); | |
| 192 | + | headers.set("content-type", "text/plain; charset=utf-8")?; | |
| 193 | + | headers.set("cache-control", "no-cache")?; | |
| 194 | + | headers.set("etag", &etag)?; | |
| 195 | + | if request.headers().get("if-none-match")?.is_some_and(|sent| sent == etag) { | |
| 196 | + | return Ok(Response::empty()?.with_status(304).with_headers(headers)); | |
| 197 | + | } | |
| 198 | + | Ok(Response::ok(body)?.with_headers(headers)) | |
| 199 | + | } | |
| 200 | + | ||
| 201 | + | async fn crate_download(&self, workspace: &str, name: &str, version: &str, viewer: Option<&User>, head: bool, ctx: &Context) -> Result<Response> { | |
| 202 | + | let Some(package) = self.crate_package(workspace, name).await? else { | |
| 203 | + | return not_found(); | |
| 204 | + | }; | |
| 205 | + | if let Some(refusal) = self.cargo_check(viewer, &package, Action::Pull) { | |
| 206 | + | return refusal; | |
| 207 | + | } | |
| 208 | + | let gone = || error(404, format!("{name}@{version} is not there.")); | |
| 209 | + | let Some(row) = self.db.version_named(&package.id, version).await? else { | |
| 210 | + | return gone(); | |
| 211 | + | }; | |
| 212 | + | let Some(digest) = Digest::parse(&row.digest) else { | |
| 213 | + | return gone(); | |
| 214 | + | }; | |
| 215 | + | let Some(blob) = self.db.package_blob(&package.id, &digest).await? else { | |
| 216 | + | return gone(); | |
| 217 | + | }; | |
| 218 | + | let headers = Headers::new(); | |
| 219 | + | headers.set("content-type", "application/gzip")?; | |
| 220 | + | headers.set("content-length", &blob.size.to_string())?; | |
| 221 | + | headers.set("cache-control", "max-age=31536000")?; | |
| 222 | + | if head { | |
| 223 | + | return Ok(Response::from_body(ResponseBody::Empty)?.with_headers(headers)); | |
| 224 | + | } | |
| 225 | + | let Some(got) = self.store.get(&blob.object_key, None).await? else { | |
| 226 | + | return gone(); | |
| 227 | + | }; | |
| 228 | + | self.count_download(&package.id, ctx); | |
| 229 | + | Ok(Response::from_body(got.body)?.with_headers(headers)) | |
| 230 | + | } | |
| 231 | + | ||
| 232 | + | /// `cargo search`: the workspace's crates the viewer may see, by name. | |
| 233 | + | async fn cargo_search(&self, url: &Url, workspace: &str, viewer: Option<&User>) -> Result<Response> { | |
| 234 | + | let query = url.query_pairs().find(|(k, _)| k == "q").map(|(_, v)| v.into_owned()).unwrap_or_default(); | |
| 235 | + | let per_page = url | |
| 236 | + | .query_pairs() | |
| 237 | + | .find(|(k, _)| k == "per_page") | |
| 238 | + | .and_then(|(_, v)| v.parse::<u32>().ok()) | |
| 239 | + | .unwrap_or(10) | |
| 240 | + | .clamp(1, MAX_SEARCH); | |
| 241 | + | let rows = self.db.list(workspace, Some(CARGO), None, Some(&query), MAX_SEARCH).await?; | |
| 242 | + | let visible: Vec<_> = rows | |
| 243 | + | .iter() | |
| 244 | + | .filter(|row| access::decide(viewer, &TargetOf::package(&row.package).view(), Action::Pull).allowed) | |
| 245 | + | .collect(); | |
| 246 | + | let crates: Vec<Value> = visible | |
| 247 | + | .iter() | |
| 248 | + | .take(per_page as usize) | |
| 249 | + | .map(|row| { | |
| 250 | + | json!({ | |
| 251 | + | "name": row.package.name, | |
| 252 | + | "max_version": crate::db::latest_shown(row).unwrap_or_default(), | |
| 253 | + | "description": row.package.description, | |
| 254 | + | }) | |
| 255 | + | }) | |
| 256 | + | .collect(); | |
| 257 | + | Response::from_json(&json!({ "crates": crates, "meta": { "total": visible.len() } })) | |
| 258 | + | } | |
| 259 | + | ||
| 260 | + | /// The package publishing makes, linked to the repository the crate's | |
| 261 | + | /// `repository` names on g1t, or else the one named like it. | |
| 262 | + | async fn cargo_target(&self, workspace: &str, name: &str, metadata: &Value) -> Result<TargetOf> { | |
| 263 | + | let named = npm::repository_of(&metadata["repository"], &self.host) | |
| 264 | + | .filter(|(owner, _)| owner == workspace) | |
| 265 | + | .map(|(_, repo)| repo); | |
| 266 | + | let lower = name.to_ascii_lowercase(); | |
| 267 | + | let dashed = lower.replace('_', "-"); | |
| 268 | + | let mut repo = None; | |
| 269 | + | for candidate in named.iter().map(String::as_str).chain([lower.as_str(), dashed.as_str()]) { | |
| 270 | + | if let Some(found) = self.repo_by_name(workspace, candidate).await? { | |
| 271 | + | repo = Some(found); | |
| 272 | + | break; | |
| 273 | + | } | |
| 274 | + | } | |
| 275 | + | Ok(TargetOf { | |
| 276 | + | workspace: workspace.to_owned(), | |
| 277 | + | repo: repo.map(|r| (r.id, r.name, r.is_private)), | |
| 278 | + | public: false, | |
| 279 | + | }) | |
| 280 | + | } | |
| 281 | + | ||
| 282 | + | /// `cargo publish`: the metadata and the `.crate` in one body. | |
| 283 | + | async fn cargo_publish(&self, request: &mut Request, workspace: &str, viewer: Option<&User>) -> Result<Response> { | |
| 284 | + | let declared = request.headers().get("content-length")?.and_then(|n| n.parse::<u64>().ok()); | |
| 285 | + | let too_large = || { | |
| 286 | + | let mb = self.max_request / 1_000_000; | |
| 287 | + | error(413, format!("A publish may be at most {mb} MB, the .crate file and its metadata together. See {DOCS}#size")) | |
| 288 | + | }; | |
| 289 | + | if declared.is_some_and(|n| n > self.max_request) { | |
| 290 | + | return too_large(); | |
| 291 | + | } | |
| 292 | + | let bytes = request.bytes().await?; | |
| 293 | + | if bytes.len() as u64 > self.max_request { | |
| 294 | + | return too_large(); | |
| 295 | + | } | |
| 296 | + | let (metadata, krate) = match cargo::parse_publish(&bytes) { | |
| 297 | + | Ok(parts) => parts, | |
| 298 | + | Err(message) => return error(400, message), | |
| 299 | + | }; | |
| 300 | + | let name = metadata["name"].as_str().unwrap_or("").to_owned(); | |
| 301 | + | if let Err(message) = cargo::valid_name(&name) { | |
| 302 | + | return error(400, message); | |
| 303 | + | } | |
| 304 | + | let version = metadata["vers"].as_str().unwrap_or("").to_owned(); | |
| 305 | + | if !npm::valid_version(&version) { | |
| 306 | + | return error(400, format!("{version} is not a semver version.")); | |
| 307 | + | } | |
| 308 | + | if krate.is_empty() { | |
| 309 | + | return error(400, "The .crate file is empty."); | |
| 310 | + | } | |
| 311 | + | let digest = Digest::of(krate); | |
| 312 | + | let entry = match cargo::index_entry(&metadata, digest.hex()) { | |
| 313 | + | Ok(entry) => entry, | |
| 314 | + | Err(message) => return error(400, message), | |
| 315 | + | }; | |
| 316 | + | ||
| 317 | + | // A name is taken whatever its case, and `-` and `_` are one. | |
| 318 | + | let found = self.db.package_folded(workspace, CARGO, &cargo::folded(&name)).await?; | |
| 319 | + | if let Some(found) = &found { | |
| 320 | + | if found.hidden() { | |
| 321 | + | return error(403, format!("The workspace {workspace} is deleted; nothing can be published to it.")); | |
| 322 | + | } | |
| 323 | + | if found.name != name { | |
| 324 | + | return error(400, format!("The name {name} is taken by the crate {}. Publish it under that name.", found.name)); | |
| 325 | + | } | |
| 326 | + | } else if self.db.workspace_hidden(workspace).await? { | |
| 327 | + | return error(403, format!("The workspace {workspace} is deleted; nothing can be published to it.")); | |
| 328 | + | } | |
| 329 | + | let target = match &found { | |
| 330 | + | Some(package) => TargetOf::package(package), | |
| 331 | + | None => self.cargo_target(workspace, &name, &metadata).await?, | |
| 332 | + | }; | |
| 333 | + | let decision = access::decide(viewer, &target.view(), Action::Push); | |
| 334 | + | if !decision.allowed { | |
| 335 | + | if viewer.is_none() { | |
| 336 | + | return error(401, sign_in(workspace)); | |
| 337 | + | } | |
| 338 | + | let readable = found.is_none() || access::decide(viewer, &target.view(), Action::Pull).allowed; | |
| 339 | + | if !readable { | |
| 340 | + | return not_found(); | |
| 341 | + | } | |
| 342 | + | return error(403, decision.reason.unwrap_or_else(|| "Not allowed.".to_owned())); | |
| 343 | + | } | |
| 344 | + | let caller = Caller { actor: viewer.map(AuditActor::of) }; | |
| 345 | + | let package = match found { | |
| 346 | + | Some(package) => package, | |
| 347 | + | None => { | |
| 348 | + | self.db | |
| 349 | + | .create_package( | |
| 350 | + | &new_id("pkg", now_ms()), | |
| 351 | + | workspace, | |
| 352 | + | CARGO, | |
| 353 | + | &name, | |
| 354 | + | target.repo.as_ref().map(|(id, repo, private)| (id.as_str(), repo.as_str(), *private)), | |
| 355 | + | caller.actor.as_ref().map_or("", |actor| actor.actor_id.as_str()), | |
| 356 | + | now_ms(), | |
| 357 | + | ) | |
| 358 | + | .await? | |
| 359 | + | } | |
| 360 | + | }; | |
| 361 | + | let existing = self.db.versions(&package.id, MAX_VERSIONS).await?; | |
| 362 | + | if let Some(taken) = existing.iter().find(|v| cargo::without_build(&v.version) == cargo::without_build(&version)) { | |
| 363 | + | return error( | |
| 364 | + | 400, | |
| 365 | + | format!("{name}@{} is already published, and a version is published once. Bump the version in Cargo.toml.", taken.version), | |
| 366 | + | ); | |
| 367 | + | } | |
| 368 | + | ||
| 369 | + | let size = krate.len() as u64; | |
| 370 | + | if let Some(refusal) = self.storage_refusal(&package, &[(digest.to_string(), size)]).await? { | |
| 371 | + | return error(403, refusal); | |
| 372 | + | } | |
| 373 | + | let now = now_ms(); | |
| 374 | + | let stored = match self.db.blob(&digest).await? { | |
| 375 | + | Some(blob) => self.store.head(&blob.object_key).await?.is_some(), | |
| 376 | + | None => false, | |
| 377 | + | }; | |
| 378 | + | if !stored { | |
| 379 | + | self.store.put(&digest.object_key(), krate.to_vec()).await?; | |
| 380 | + | } | |
| 381 | + | self.db | |
| 382 | + | .keep_blob(&package.id, &digest, size, Some("application/gzip"), &digest.object_key(), now) | |
| 383 | + | .await?; | |
| 384 | + | self.db | |
| 385 | + | .publish( | |
| 386 | + | NewVersion { | |
| 387 | + | id: new_id("ver", now), | |
| 388 | + | package_id: package.id.clone(), | |
| 389 | + | version: version.clone(), | |
| 390 | + | digest: digest.to_string(), | |
| 391 | + | size, | |
| 392 | + | metadata: entry.to_string(), | |
| 393 | + | subject: None, | |
| 394 | + | published_by: published_by(&caller), | |
| 395 | + | files: vec![NewFile { | |
| 396 | + | name: "crate".to_owned(), | |
| 397 | + | digest: digest.to_string(), | |
| 398 | + | size, | |
| 399 | + | media_type: Some("application/gzip".to_owned()), | |
| 400 | + | }], | |
| 401 | + | }, | |
| 402 | + | None, | |
| 403 | + | now, | |
| 404 | + | ) | |
| 405 | + | .await?; | |
| 406 | + | // The README and description the crate's page shows: the highest | |
| 407 | + | // stable version's, so a pre-release does not replace them. | |
| 408 | + | let highest = std::iter::once(version.as_str()) | |
| 409 | + | .chain(existing.iter().map(|v| v.version.as_str())) | |
| 410 | + | .collect::<Vec<_>>() | |
| 411 | + | .join("\n"); | |
| 412 | + | if crate::db::newest_version(&highest).as_deref() == Some(version.as_str()) { | |
| 413 | + | let readme = metadata["readme"].as_str().unwrap_or("").trim(); | |
| 414 | + | let readme_digest = if readme.is_empty() || readme.len() > MAX_README_BYTES { | |
| 415 | + | None | |
| 416 | + | } else { | |
| 417 | + | let bytes = readme.as_bytes().to_vec(); | |
| 418 | + | let digest = Digest::of(&bytes); | |
| 419 | + | if self.db.blob(&digest).await?.is_none() { | |
| 420 | + | self.store.put(&digest.object_key(), bytes.clone()).await?; | |
| 421 | + | } | |
| 422 | + | self.db | |
| 423 | + | .keep_blob(&package.id, &digest, bytes.len() as u64, Some("text/markdown"), &digest.object_key(), now) | |
| 424 | + | .await?; | |
| 425 | + | Some(digest.to_string()) | |
| 426 | + | }; | |
| 427 | + | self.db.set_readme(&package.id, readme_digest.as_deref(), metadata["description"].as_str(), now).await?; | |
| 428 | + | } | |
| 429 | + | self.db.measure(&package.workspace).await?; | |
| 430 | + | let event = PackageEvent { | |
| 431 | + | version: Some(version.clone()), | |
| 432 | + | digest: Some(digest.to_string()), | |
| 433 | + | size: Some(size), | |
| 434 | + | ..self.event_of(&package) | |
| 435 | + | }; | |
| 436 | + | self.announce("package.published", &package, event, &caller).await; | |
| 437 | + | self.audit(&caller, "package.publish", &package, Some(&format!("{workspace}/{name}@{version}")), None).await; | |
| 438 | + | Response::from_json(&json!({ "warnings": { "invalid_categories": [], "invalid_badges": [], "other": [] } })) | |
| 439 | + | } | |
| 440 | + | ||
| 441 | + | /// `cargo yank` and `cargo yank --undo`: the version stays, for | |
| 442 | + | /// lockfiles that name it, but is no longer picked for new ones. | |
| 443 | + | async fn cargo_yank(&self, workspace: &str, name: &str, version: &str, yank: bool, viewer: Option<&User>) -> Result<Response> { | |
| 444 | + | let Some(package) = self.crate_package(workspace, name).await? else { | |
| 445 | + | return not_found(); | |
| 446 | + | }; | |
| 447 | + | if let Some(refusal) = self.cargo_check(viewer, &package, Action::Push) { | |
| 448 | + | return refusal; | |
| 449 | + | } | |
| 450 | + | let Some(row): Option<VersionRow> = self.db.version_named(&package.id, version).await? else { | |
| 451 | + | return error(404, format!("{name}@{version} is not there.")); | |
| 452 | + | }; | |
| 453 | + | if row.is_yanked() != yank { | |
| 454 | + | self.db.set_yanked(&row.id, yank).await?; | |
| 455 | + | self.db.touch_package(&package.id, now_ms()).await?; | |
| 456 | + | let caller = Caller { actor: viewer.map(AuditActor::of) }; | |
| 457 | + | let action = if yank { "package.yank" } else { "package.unyank" }; | |
| 458 | + | self.audit(&caller, action, &package, Some(&format!("{workspace}/{}@{version}", package.name)), None).await; | |
| 459 | + | } | |
| 460 | + | ok() | |
| 461 | + | } | |
| 462 | + | } |
| 62 | 62 | pub version_count: u32, | |
| 63 | 63 | pub bytes: u64, | |
| 64 | 64 | pub latest_tag: Option<String>, | |
| 65 | + | /// The version `latest_tag` points to: what an npm listing shows, | |
| 66 | + | /// since its tags name versions rather than being what is installed. | |
| 67 | + | #[serde(default)] | |
| 68 | + | pub latest_tag_version: Option<String>, | |
| 65 | 69 | /// Every version, newest published first, one per line: the summary | |
| 66 | 70 | /// picks the highest of them (see `newest_version`). | |
| 67 | 71 | pub latest_version: Option<String>, | |
| 91 | 95 | .or_else(|| list.first().map(|v| (*v).to_owned())) | |
| 92 | 96 | } | |
| 93 | 97 | ||
| 98 | + | /// What a listing shows as a package's latest. An image's tag is what is | |
| 99 | + | /// pulled, so it is shown as is; npm's dist-tags (`latest`) name versions, | |
| 100 | + | /// so the version the tag points to is shown, as for every other registry. | |
| 101 | + | /// Without a tag, the highest version (see `newest_version`). | |
| 102 | + | pub fn latest_shown(row: &ListedRow) -> Option<String> { | |
| 103 | + | let tagged = if row.package.ecosystem == "container" { &row.latest_tag } else { &row.latest_tag_version }; | |
| 104 | + | tagged.clone().or_else(|| row.latest_version.as_deref().and_then(newest_version)) | |
| 105 | + | } | |
| 106 | + | ||
| 94 | 107 | #[cfg(test)] | |
| 95 | 108 | mod newest_tests { | |
| 96 | − | use super::newest_version; | |
| 109 | + | use super::{ListedRow, PackageRow, latest_shown, newest_version}; | |
| 110 | + | ||
| 111 | + | fn listed(ecosystem: &str, tag: Option<&str>, tag_version: Option<&str>, versions: &str) -> ListedRow { | |
| 112 | + | ListedRow { | |
| 113 | + | package: PackageRow { | |
| 114 | + | id: "pkg_1".into(), | |
| 115 | + | workspace: "acme".into(), | |
| 116 | + | ecosystem: ecosystem.into(), | |
| 117 | + | name: "web".into(), | |
| 118 | + | repo_id: None, | |
| 119 | + | repo_name: None, | |
| 120 | + | visibility: "private".into(), | |
| 121 | + | description: None, | |
| 122 | + | created_by: "usr_1".into(), | |
| 123 | + | created_at: "2026-10-06T00:00:00.000Z".into(), | |
| 124 | + | updated_at: "2026-10-06T00:00:00.000Z".into(), | |
| 125 | + | downloads: 0, | |
| 126 | + | workspace_deleted_at: None, | |
| 127 | + | }, | |
| 128 | + | version_count: 2, | |
| 129 | + | bytes: 0, | |
| 130 | + | latest_tag: tag.map(str::to_owned), | |
| 131 | + | latest_tag_version: tag_version.map(str::to_owned), | |
| 132 | + | latest_version: Some(versions.to_owned()), | |
| 133 | + | } | |
| 134 | + | } | |
| 135 | + | ||
| 136 | + | #[test] | |
| 137 | + | fn npm_shows_the_version_its_tag_points_to_and_an_image_its_tag() { | |
| 138 | + | let npm = listed("npm", Some("latest"), Some("1.2.0"), "2.0.0-beta.1\n1.2.0\n1.0.0"); | |
| 139 | + | assert_eq!(latest_shown(&npm).as_deref(), Some("1.2.0"), "the version, not the word latest"); | |
| 140 | + | let image = listed("container", Some("latest"), Some("sha256:abc"), "sha256:abc"); | |
| 141 | + | assert_eq!(latest_shown(&image).as_deref(), Some("latest"), "an image is pulled by its tag"); | |
| 142 | + | let cargo = listed("cargo", None, None, "0.9.0\n1.1.0\n1.0.0"); | |
| 143 | + | assert_eq!(latest_shown(&cargo).as_deref(), Some("1.1.0"), "no tags: the highest version"); | |
| 144 | + | let untagged = listed("container", None, None, "sha256:abc"); | |
| 145 | + | assert_eq!(latest_shown(&untagged).as_deref(), Some("sha256:abc")); | |
| 146 | + | } | |
| 97 | 147 | ||
| 98 | 148 | #[test] | |
| 99 | 149 | fn the_latest_is_the_highest_stable_version_not_the_last_published() { | |
| 135 | 185 | /// npm's deprecation message, when the version is deprecated. | |
| 136 | 186 | #[serde(default)] | |
| 137 | 187 | pub deprecated: Option<String>, | |
| 188 | + | /// Cargo: 1 when the version is yanked. | |
| 189 | + | #[serde(default)] | |
| 190 | + | pub yanked: u32, | |
| 138 | 191 | } | |
| 139 | 192 | ||
| 140 | 193 | impl VersionRow { | |
| 194 | + | pub fn is_yanked(&self) -> bool { | |
| 195 | + | self.yanked != 0 | |
| 196 | + | } | |
| 197 | + | } | |
| 198 | + | ||
| 199 | + | impl VersionRow { | |
| 141 | 200 | pub fn meta(&self) -> serde_json::Value { | |
| 142 | 201 | serde_json::from_str(&self.metadata).unwrap_or_default() | |
| 143 | 202 | } | |
| 206 | 265 | "id, workspace, ecosystem, name, repo_id, repo_name, visibility, description, created_by, created_at, updated_at, downloads, workspace_deleted_at"; | |
| 207 | 266 | /// Workspaces that are deleted, waiting to be purged or restored. | |
| 208 | 267 | const DELETED_WORKSPACES: &str = "SELECT workspace FROM packages WHERE workspace_deleted_at IS NOT NULL"; | |
| 209 | − | const VERSION_COLUMNS: &str = "id, package_id, version, digest, size, metadata, subject, published_by, published_at, deprecated"; | |
| 268 | + | const VERSION_COLUMNS: &str = "id, package_id, version, digest, size, metadata, subject, published_by, published_at, deprecated, yanked"; | |
| 210 | 269 | ||
| 211 | 270 | pub struct Db { | |
| 212 | 271 | pub db: D1Database, | |
| 226 | 285 | .await | |
| 227 | 286 | } | |
| 228 | 287 | ||
| 288 | + | /// A package by its name in any case: Cargo's names are one name | |
| 289 | + | /// whatever their case (`Inflector` is `inflector`). | |
| 290 | + | pub async fn package_any_case(&self, workspace: &str, ecosystem: &str, name: &str) -> Result<Option<PackageRow>> { | |
| 291 | + | self.prepare( | |
| 292 | + | &format!("SELECT {PACKAGE_COLUMNS} FROM packages WHERE workspace = ? AND ecosystem = ? AND name = ? COLLATE NOCASE LIMIT 1"), | |
| 293 | + | &[text(workspace), text(ecosystem), text(name)], | |
| 294 | + | )? | |
| 295 | + | .first(None) | |
| 296 | + | .await | |
| 297 | + | } | |
| 298 | + | ||
| 299 | + | /// The package a new crate's name would clash with: one named the same | |
| 300 | + | /// apart from case and `-` against `_`, as crates.io decides. | |
| 301 | + | pub async fn package_folded(&self, workspace: &str, ecosystem: &str, folded: &str) -> Result<Option<PackageRow>> { | |
| 302 | + | self.prepare( | |
| 303 | + | &format!( | |
| 304 | + | "SELECT {PACKAGE_COLUMNS} FROM packages WHERE workspace = ? AND ecosystem = ? AND replace(lower(name), '_', '-') = ? LIMIT 1" | |
| 305 | + | ), | |
| 306 | + | &[text(workspace), text(ecosystem), text(folded)], | |
| 307 | + | )? | |
| 308 | + | .first(None) | |
| 309 | + | .await | |
| 310 | + | } | |
| 311 | + | ||
| 312 | + | /// Whether the workspace has a private package of the ecosystem. | |
| 313 | + | pub async fn has_private(&self, workspace: &str, ecosystem: &str) -> Result<bool> { | |
| 314 | + | let row: Option<serde_json::Value> = self | |
| 315 | + | .prepare( | |
| 316 | + | "SELECT 1 AS private FROM packages WHERE workspace = ? AND ecosystem = ? AND visibility = 'private' AND workspace_deleted_at IS NULL LIMIT 1", | |
| 317 | + | &[text(workspace), text(ecosystem)], | |
| 318 | + | )? | |
| 319 | + | .first(None) | |
| 320 | + | .await?; | |
| 321 | + | Ok(row.is_some()) | |
| 322 | + | } | |
| 323 | + | ||
| 324 | + | pub async fn set_yanked(&self, version_id: &str, yanked: bool) -> Result<()> { | |
| 325 | + | self.prepare("UPDATE versions SET yanked = ? WHERE id = ?", &[num(u64::from(yanked)), text(version_id)])? | |
| 326 | + | .run() | |
| 327 | + | .await?; | |
| 328 | + | Ok(()) | |
| 329 | + | } | |
| 330 | + | ||
| 229 | 331 | /// Makes a package unless one of the name is already there (a push | |
| 230 | 332 | /// beside this one may have made it), and answers with the one kept. | |
| 231 | 333 | #[allow(clippy::too_many_arguments)] | |
| 282 | 384 | (SELECT COALESCE(SUM(b.size), 0) FROM blobs b WHERE b.digest IN \ | |
| 283 | 385 | (SELECT vf.digest FROM version_files vf JOIN versions v ON v.id = vf.version_id WHERE v.package_id = p.id)) AS bytes, \ | |
| 284 | 386 | (SELECT t.tag FROM tags t WHERE t.package_id = p.id ORDER BY t.tag = 'latest' DESC, t.updated_at DESC LIMIT 1) AS latest_tag, \ | |
| 285 | − | (SELECT GROUP_CONCAT(version, char(10)) FROM (SELECT v.version FROM versions v WHERE v.package_id = p.id ORDER BY v.published_at DESC)) AS latest_version \ | |
| 387 | + | (SELECT v.version FROM tags t JOIN versions v ON v.id = t.version_id WHERE t.package_id = p.id \ | |
| 388 | + | ORDER BY t.tag = 'latest' DESC, t.updated_at DESC LIMIT 1) AS latest_tag_version, \ | |
| 389 | + | (SELECT GROUP_CONCAT(version, char(10)) FROM \ | |
| 390 | + | (SELECT v.version FROM versions v WHERE v.package_id = p.id AND v.yanked = 0 ORDER BY v.published_at DESC)) AS latest_version \ | |
| 286 | 391 | FROM packages p WHERE p.workspace = ? AND p.workspace_deleted_at IS NULL", | |
| 287 | 392 | PACKAGE_COLUMNS.split(", ").map(|c| format!("p.{c}")).collect::<Vec<_>>().join(", ") | |
| 288 | 393 | ); |
| 8 | 8 | //! `BlobStore` port (store/), metadata in D1 (db.rs). | |
| 9 | 9 | ||
| 10 | 10 | mod access; | |
| 11 | + | mod cargo; | |
| 12 | + | mod cargo_http; | |
| 11 | 13 | mod composer; | |
| 12 | 14 | mod composer_http; | |
| 13 | 15 | mod db; | |
| 348 | 350 | address: match p.ecosystem.as_str() { | |
| 349 | 351 | "npm" => format!("{}/-/npm/@{}/{}", self.host, p.workspace, p.name), | |
| 350 | 352 | "composer" => format!("{}/-/composer/{}/{}", self.host, p.workspace, p.name), | |
| 353 | + | "cargo" => format!("{}/-/cargo/{}/{}", self.host, p.workspace, p.name), | |
| 351 | 354 | _ => format!("{}/{}/{}", self.host, p.workspace, p.name), | |
| 352 | 355 | }, | |
| 353 | 356 | visibility: Visibility::parse(&p.visibility), | |
| 358 | 361 | }), | |
| 359 | 362 | description: p.description.clone(), | |
| 360 | 363 | versions: row.version_count, | |
| 361 | − | latest: row.latest_tag.clone().or_else(|| row.latest_version.as_deref().and_then(db::newest_version)), | |
| 364 | + | latest: db::latest_shown(row), | |
| 362 | 365 | size: row.bytes, | |
| 363 | 366 | downloads: p.downloads, | |
| 364 | 367 | created_at: p.created_at.clone(), | |
| 377 | 380 | version_count: 0, | |
| 378 | 381 | bytes: 0, | |
| 379 | 382 | latest_tag: None, | |
| 383 | + | latest_tag_version: None, | |
| 380 | 384 | latest_version: None, | |
| 381 | 385 | })) | |
| 382 | 386 | } | |
| 429 | 433 | subject: version.subject, | |
| 430 | 434 | published_by: version.published_by, | |
| 431 | 435 | published_at: version.published_at, | |
| 432 | − | deprecated: version.deprecated, | |
| 436 | + | // A yanked crate version reads as deprecated: still | |
| 437 | + | // there for lockfiles, no longer picked for new ones. | |
| 438 | + | deprecated: if version.yanked != 0 { | |
| 439 | + | Some("Yanked: Cargo no longer picks this version for new lockfiles.".to_owned()) | |
| 440 | + | } else { | |
| 441 | + | version.deprecated | |
| 442 | + | }, | |
| 433 | 443 | } | |
| 434 | 444 | }) | |
| 435 | 445 | .collect(); | |
| 669 | 679 | if request.path().starts_with("/-/composer/") { | |
| 670 | 680 | return packages.composer(request, &ctx).await; | |
| 671 | 681 | } | |
| 682 | + | if request.path().starts_with("/-/cargo/") { | |
| 683 | + | return packages.cargo(request, &ctx).await; | |
| 684 | + | } | |
| 672 | 685 | return packages.registry(request, &ctx).await; | |
| 673 | 686 | }; | |
| 674 | 687 | let body: serde_json::Value = request.json().await?; |
| 975 | 975 | ..self.event_of(package) | |
| 976 | 976 | }; | |
| 977 | 977 | self.announce("package.version_deleted", package, event, caller).await; | |
| 978 | − | // npm names a version by its number; an image by its digest. | |
| 978 | + | // npm and Cargo name a version by its number; an image by its digest. | |
| 979 | 979 | let path = if package.ecosystem == "npm" { | |
| 980 | 980 | format!("@{}/{}@{}", package.workspace, package.name, version.version) | |
| 981 | + | } else if package.ecosystem == "cargo" { | |
| 982 | + | format!("{}/{}@{}", package.workspace, package.name, version.version) | |
| 981 | 983 | } else { | |
| 982 | 984 | format!("{}/{}@{}", package.workspace, package.name, version.digest) | |
| 983 | 985 | }; |