Skip to content
Sign in

flagon-io/g1t

public
0

g1t/crates/runner/src/bump.rs

2,112 lines98,655 bytesCodeBlame

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1//! Makes a security or version update: raises one or more packages to a
2//! version in the lockfiles named, with the ecosystem's own tool, commits
3//! that as g1t and pushes it to a branch of its own. The push is what tells
4//! the security service to open the pull request; this opens nothing
5//! itself.
6//!
7//! A security update raises one package to a fixed version, on a branch
8//! under `g1t/security/`. A version update (`BUMP_KIND=version`) raises one
9//! package or a group of them in one commit, on the branch its dependency
10//! update file names (any but the default one), and changes manifests as
11//! its versioning strategy says.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily12//!
13//! What each lockfile is updated with (the lockfiles `g1t_scan::lockfiles`
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar14//! reads), for a security update and a version update with the `increase`
15//! strategy:
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily16//!
17//! | Lockfile | A direct dependency | Any other |
18//! | --- | --- | --- |
19//! | `package-lock.json` | `npm install --package-lock-only <pkg>@<range>` | `npm update --package-lock-only <pkg>`, then an `overrides` entry |
20//! | `pnpm-lock.yaml` | `pnpm update <pkg>@<range> --lockfile-only` | `pnpm update <pkg> --depth Infinity --lockfile-only`, then `pnpm.overrides` |
21//! | `yarn.lock` (2 and later) | `yarn up <pkg>@<range> --mode=update-lockfile` | `yarn up --recursive <pkg>`, then `resolutions` |
22//! | `yarn.lock` (1) | `yarn upgrade <pkg>@<range>` | `resolutions` |
23//! | `Cargo.lock` | `cargo update -p <pkg>@<old> --precise <version>`, else `cargo update -p <pkg>@<old>` | the same |
24//! | `go.mod`, `go.sum` | `go get <module>@v<version>`, then `go mod tidy` | the same |
25//! | `poetry.lock` | `poetry add <pkg>@^<version> --lock` | `poetry update --lock <pkg>` |
26//! | `requirements.txt` | its `==` pins rewritten | the same |
27//!
28//! A direct dependency keeps its range's style (`^`, `~` or exact). No
29//! install script runs. pnpm and yarn run through corepack, so the
30//! version a project names in `packageManager` is the one used. Poetry is
31//! installed into a virtual environment if the sandbox has none.
32//!
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar33//! A version update never adds an override or a resolution: a package
34//! nothing in `package.json` names is moved only as far as the ranges that
35//! ask for it allow. Its strategy (`BUMP_STRATEGY`) decides how a direct
36//! dependency's requirement changes:
37//!
38//! | Strategy | npm, pnpm, yarn | Cargo | Poetry |
39//! | --- | --- | --- | --- |
40//! | `increase` | the range raised, as above | `--precise`; if the requirement does not allow it, the requirement raised in `Cargo.toml` | as above |
41//! | `increase-if-necessary` | the update the range allows (`npm update`, `pnpm update`, `yarn up -R`, `yarn upgrade <pkg>`); if that is not enough, the range raised | as `increase` | as above |
42//! | `widen` | the update the range allows; if that is not enough, the range widened: `^1.2.0 \|\| ^2.0.0` | as `increase` | as above |
43//! | `lockfile-only` | the update the range allows, and `package.json` left alone | `--precise`, else as far as the requirement allows | `poetry update --lock <pkg>` |
44//!
45//! Go modules and `requirements.txt` are updated the same way whatever the
46//! strategy. A requirement raised in `Cargo.toml` keeps its operator (`^`,
47//! `~`, `=`, `>=` or none) and is found in the lockfile's directory and in
48//! the workspace members it lists by path or by a trailing `/*`: in
49//! `[dependencies]`, `[dev-dependencies]`, `[build-dependencies]`, their
50//! `[target.*]` forms and `[workspace.dependencies]`, written as
51//! `name = "1.2"`, `name = { version = "1.2", … }` on one line, or a
52//! `[dependencies.name]` table. A requirement with more than one part
53//! (`>=1, <2`) is left alone.
54//!
55//! Private registries (`BUMP_REGISTRIES`) are written where the tools read
56//! them, outside the clone, so they are never committed; credentials are
57//! never printed:
58//!
59//! - `npm-registry`: a user npmrc (`NPM_CONFIG_USERCONFIG`, read by npm and
60//! pnpm) with the registry's `_authToken` or `_auth`, `registry=` when it
61//! replaces npm's and `@scope:registry=` for each scope. Yarn 2 and later
62//! is given only a registry that replaces npm's
63//! (`YARN_NPM_REGISTRY_SERVER` and its token or identity); its scopes are
64//! not configured.
65//! - `cargo-registry`: a token (`token`, else `password`) as
66//! `CARGO_REGISTRIES_<NAME>_TOKEN` for each registry the project's
67//! `.cargo/config.toml` names with the same index; one that replaces
68//! crates.io is also given to every cargo run as source replacement
69//! (`cargo --config <file>`). A registry the project does not name and
70//! that replaces nothing is not reachable.
71//! - `python-index`: `PIP_INDEX_URL` when it replaces PyPI, otherwise
72//! `PIP_EXTRA_INDEX_URL`, with the credentials in the URL; and
73//! `POETRY_HTTP_BASIC_<NAME>_USERNAME`/`_PASSWORD` for each source in
74//! `pyproject.toml` with the same URL.
75//! - `goproxy-server`: `GOPROXY=<url>,https://proxy.golang.org,direct`
76//! (`<url>,direct` when it replaces the public proxy) and a netrc entry
77//! (`NETRC`) for its host. The checksum database is not changed, so a
78//! private module it does not know still fails to verify.
79//!
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily80//! Afterwards every lockfile is read again, and the update counts only if
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar81//! none of them resolves a package below its version any more. When the
82//! tool cannot get there (another package holds it back, or the strategy
83//! does not allow the change it needs), the job fails saying it needs code
84//! changes, with the tool's last lines.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily85//!
86//! Configuration:
87//!
88//! - `GIT_REMOTE`, `GIT_BRANCH_BASE`: the repository and its default branch.
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar89//! - `GIT_BRANCH`: the branch to push: under `g1t/security/` for a security
90//! update; for a version update, any name git takes but the default
91//! branch.
92//! - `BUMP_KIND`: `security` (the default) or `version`.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily93//! - `BUMP_ECOSYSTEM` (OSV's name: `npm`, `crates.io`, `Go`, `PyPI`),
94//! `BUMP_PACKAGE`, `BUMP_VERSION`: what to raise, to what.
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar95//! - `BUMP_PACKAGES`: several packages raised in one commit, as a JSON
96//! array of `{"package", "version"}`; `BUMP_PACKAGE` and `BUMP_VERSION`
97//! when absent or empty.
98//! - `BUMP_STRATEGY`: a version update's versioning strategy, `increase`
99//! by default. A security update always updates as the first table says.
100//! - `BUMP_FORCE`: `1` to replace the branch if it is there already.
101//! - `BUMP_REGISTRIES`: private registries, as a JSON array of
102//! `{"type", "url", "username", "password", "token", "replacesBase",
103//! "scopes"}`.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily104//! - `BUMP_LOCKFILES`: the lockfiles' paths from the root, one per line or
105//! as a JSON array.
106//! - `COMMIT_MESSAGE`: the commit's message.
107//! - `G1T_USER`, `G1T_TOKEN`: to clone and push; passed per command, never
108//! written to the clone's config or remote.
109//!
110//! It prints one line of JSON on stdout saying what happened, and exits 0
111//! once the branch is pushed; otherwise non-zero, with why on stderr:
112//! `NEEDS_CHANGES_EXIT` when the update needs code changes,
113//! `UNSUPPORTED_EXIT` for a lockfile or tool it cannot update.
114
115use std::collections::BTreeSet;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar116use std::path::{Path, PathBuf};
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily117use std::process::Command;
118
119use anyhow::{Context, Result, anyhow, bail};
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar120use base64::Engine;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily121use g1t_scan::lockfiles::{Ecosystem, Lockfile};
122use g1t_scan::version;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar123use serde::{Deserialize, Serialize};
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily124use serde_json::{Value, json};
125
126use crate::{WORKDIR, auth_option, env, git};
127
g1t is one name: its agent's work, commits and comments show as @g1t, and nobody can claim g1t or g1t-agent128use crate::{AUTHOR_EMAIL, AUTHOR_NAME};
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily129/// Every security update's branch starts with this:
130/// `g1t_contracts::security::UPDATE_BRANCH_PREFIX`.
131const BRANCH_PREFIX: &str = "g1t/security/";
132
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar133/// Where the private registries' configuration is written: outside the
134/// clone, so it is never committed.
135const REGISTRY_DIR: &str = "/tmp/g1t-registries";
136
137/// The most packages one update raises.
138const MAX_PACKAGES: usize = 50;
139
140/// Why a version update with the `lockfile-only` strategy stopped short.
141const LOCKFILE_ONLY: &str = "needs a manifest change, which lockfile-only does not make";
142
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily143/// The exit code when the update needs code changes, not just a lockfile.
144pub const NEEDS_CHANGES_EXIT: i32 = 3;
145/// The exit code for a lockfile or ecosystem this cannot update.
146pub const UNSUPPORTED_EXIT: i32 = 4;
147
148/// Why a bump stopped, when that is not just an error.
149#[derive(Debug)]
150enum Stop {
151 /// The tool could not raise it: something else holds it back.
152 NeedsChanges(String),
153 /// Not something this can update.
154 Unsupported(String),
155}
156
157impl std::fmt::Display for Stop {
158 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
159 match self {
160 Stop::NeedsChanges(why) => write!(f, "needs code changes: {why}"),
161 Stop::Unsupported(why) => write!(f, "unsupported: {why}"),
162 }
163 }
164}
165
166impl std::error::Error for Stop {}
167
168fn needs_changes(why: impl Into<String>) -> anyhow::Error {
169 anyhow!(Stop::NeedsChanges(why.into()))
170}
171
172fn unsupported(why: impl Into<String>) -> anyhow::Error {
173 anyhow!(Stop::Unsupported(why.into()))
174}
175
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar176/// A security update or a version update.
177#[derive(Clone, Copy, Debug, PartialEq, Eq)]
178enum Kind {
179 Security,
180 Version,
181}
182
183impl Kind {
184 fn parse(text: &str) -> Result<Kind> {
185 match text.trim() {
186 "" | "security" => Ok(Kind::Security),
187 "version" => Ok(Kind::Version),
188 other => bail!("g1t cannot make a {other} update"),
189 }
190 }
191
192 fn name(self) -> &'static str {
193 match self {
194 Kind::Security => "security",
195 Kind::Version => "version",
196 }
197 }
198}
199
200/// How a version update changes a direct dependency's requirement.
201#[derive(Clone, Copy, Debug, PartialEq, Eq)]
202enum Strategy {
203 /// Raise the requirement to the version.
204 Increase,
205 /// Only when what it allows is not enough.
206 IncreaseIfNecessary,
207 /// Allow the version as well as what it allowed.
208 Widen,
209 /// Never: only the lockfile changes.
210 LockfileOnly,
211}
212
213impl Strategy {
214 fn parse(text: &str) -> Result<Strategy> {
215 Ok(match text.trim() {
216 "" | "increase" => Strategy::Increase,
217 "increase-if-necessary" => Strategy::IncreaseIfNecessary,
218 "widen" => Strategy::Widen,
219 "lockfile-only" => Strategy::LockfileOnly,
220 other => bail!("{other} is not a versioning strategy"),
221 })
222 }
223}
224
225/// A package and the version to raise it to.
226#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
227struct Target {
228 package: String,
229 /// As the ecosystem's tools write it (Go's with `v`).
230 version: String,
231}
232
233/// A private registry the tools may read: a `BUMP_REGISTRIES` entry.
234#[derive(Clone, Default, PartialEq, Eq, Deserialize)]
235#[serde(rename_all = "camelCase")]
236struct Registry {
237 /// `npm-registry`, `cargo-registry`, `python-index` or `goproxy-server`.
238 #[serde(rename = "type")]
239 kind: String,
240 url: String,
241 #[serde(default)]
242 username: Option<String>,
243 #[serde(default)]
244 password: Option<String>,
245 #[serde(default)]
246 token: Option<String>,
247 /// Used in place of the ecosystem's public registry.
248 #[serde(default, alias = "replaces_base")]
249 replaces_base: bool,
250 /// npm scopes it serves: `@acme`.
251 #[serde(default)]
252 scopes: Vec<String>,
253}
254
255/// Never shows the credentials.
256impl std::fmt::Debug for Registry {
257 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
258 f.debug_struct("Registry")
259 .field("kind", &self.kind)
260 .field("url", &self.url)
261 .field("replaces_base", &self.replaces_base)
262 .field("scopes", &self.scopes)
263 .finish_non_exhaustive()
264 }
265}
266
267impl Registry {
268 /// The token, else the password: what a registry that takes one
269 /// secret is given.
270 fn secret(&self) -> Option<&str> {
271 self.token.as_deref().or(self.password.as_deref()).filter(|secret| !secret.is_empty())
272 }
273}
274
275/// What to raise, to what, where, and how.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily276#[derive(Debug)]
277struct Bump {
278 ecosystem: Ecosystem,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar279 kind: Kind,
280 packages: Vec<Target>,
281 /// A version update's strategy; none for a security update, which
282 /// always updates as it did before strategies.
283 strategy: Option<Strategy>,
284 /// Replace the branch if it is there already.
285 force: bool,
286 registries: Vec<Registry>,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily287 jobs: Vec<Job>,
288}
289
290/// One directory's lockfiles of one kind, updated by one tool run.
291#[derive(Debug, PartialEq, Eq)]
292struct Job {
293 /// From the repository's root; empty for the root.
294 dir: String,
295 lockfile: Lockfile,
296 /// The lockfiles' paths from the root, each read again afterwards.
297 paths: Vec<String>,
298}
299
300impl Job {
301 fn file(&self, name: &str) -> String {
302 if self.dir.is_empty() { name.to_owned() } else { format!("{}/{name}", self.dir) }
303 }
304
305 /// What the tool may change: the lockfiles and their manifest. Only
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar306 /// these, and the `Cargo.toml` files a version update raises a
307 /// requirement in, are committed, whatever else a tool leaves behind.
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily308 fn touched(&self) -> Vec<String> {
309 let mut files: Vec<String> = self.paths.clone();
310 let manifests: &[&str] = match self.lockfile {
311 Lockfile::PackageLock | Lockfile::PnpmLock | Lockfile::YarnLock => &["package.json"],
312 Lockfile::GoMod | Lockfile::GoSum => &["go.mod", "go.sum"],
313 Lockfile::PoetryLock => &["pyproject.toml"],
314 Lockfile::CargoLock | Lockfile::Requirements => &[],
315 };
316 files.extend(manifests.iter().map(|name| self.file(name)));
317 files.sort();
318 files.dedup();
319 files
320 }
321}
322
323/// `BUMP_LOCKFILES`: a JSON array, or one path per line.
324fn lockfile_list(text: &str) -> Result<Vec<String>> {
325 let text = text.trim();
326 let paths: Vec<String> = if text.starts_with('[') {
327 serde_json::from_str(text).context("BUMP_LOCKFILES is not a JSON array of paths")?
328 } else {
329 text.lines().map(str::to_owned).collect()
330 };
331 Ok(paths.into_iter().map(|path| path.trim().trim_start_matches("./").to_owned()).filter(|path| !path.is_empty()).collect())
332}
333
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar334/// The packages to raise: `BUMP_PACKAGES`, or `BUMP_PACKAGE` and
335/// `BUMP_VERSION` when it is absent or empty. Each is checked as a tool
336/// argument, its version written as the tools take it, and a package named
337/// twice is raised once.
338fn targets(ecosystem: Ecosystem, listed: Option<&str>, package: Option<&str>, version: Option<&str>) -> Result<Vec<Target>> {
339 let listed: Vec<Target> = match listed.map(str::trim).filter(|text| !text.is_empty()) {
340 Some(text) => serde_json::from_str(text).context("BUMP_PACKAGES is not a JSON array of packages and versions")?,
341 None => Vec::new(),
342 };
343 let listed = if listed.is_empty() {
344 vec![Target {
345 package: package.ok_or_else(|| anyhow!("BUMP_PACKAGE is not set"))?.to_owned(),
346 version: version.ok_or_else(|| anyhow!("BUMP_VERSION is not set"))?.to_owned(),
347 }]
348 } else {
349 listed
350 };
351 if listed.len() > MAX_PACKAGES {
352 bail!("an update raises at most {MAX_PACKAGES} packages");
353 }
354 let mut out: Vec<Target> = Vec::new();
355 for target in listed {
356 let target = Target { package: target.package.trim().to_owned(), version: tool_version(ecosystem, &target.version) };
357 safe_argument("package", &target.package)?;
358 safe_argument("version", &target.version)?;
359 if !out.iter().any(|seen| ecosystem.normalize(&seen.package) == ecosystem.normalize(&target.package)) {
360 out.push(target);
361 }
362 }
363 Ok(out)
364}
365
366/// Whether git takes `branch` as a branch's name, short of a full ref.
367/// Mirrors `isBranchName` in services/runner bump.ts.
368fn branch_name_ok(branch: &str) -> bool {
369 let bad = |c: char| c.is_whitespace() || c.is_control() || "~^:?*[\\".contains(c);
370 !branch.trim().is_empty()
371 && branch.len() <= 200
372 && !branch.chars().any(bad)
373 && !branch.contains("..")
374 && !branch.contains("@{")
375 && !branch.starts_with('-')
376 && !branch.starts_with('/')
377 && !branch.starts_with("refs/")
378 && !branch.ends_with('/')
379 && !branch.ends_with(".lock")
380}
381
382/// Whether this kind of update may push to `branch`: a security update's
383/// is under `g1t/security/`; a version update's is any name git takes but
384/// `base`, the default branch.
385fn check_branch(kind: Kind, branch: &str, base: &str) -> Result<()> {
386 match kind {
387 Kind::Security => {
388 if !branch.starts_with(BRANCH_PREFIX) || branch.contains("..") || branch.chars().any(char::is_whitespace) {
389 bail!("{branch} is not a security update's branch");
390 }
391 }
392 Kind::Version => {
393 if !branch_name_ok(branch) {
394 bail!("{branch:?} is not a branch name git takes");
395 }
396 if branch == base.trim() {
397 bail!("a version update cannot push to {base}, the default branch");
398 }
399 }
400 }
401 Ok(())
402}
403
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily404/// The lockfiles grouped into what one tool run updates: `go.mod` and
405/// `go.sum` in one directory are one module. Each must be a lockfile of
406/// `ecosystem`, inside the repository.
407fn jobs(ecosystem: Ecosystem, paths: &[String]) -> Result<Vec<Job>> {
408 let mut jobs: Vec<Job> = Vec::new();
409 for path in paths {
410 if path.starts_with('/') || path.contains('\\') || path.split('/').any(|part| part == ".." || part.is_empty()) {
411 bail!("{path} is not a path inside the repository");
412 }
413 let lockfile = Lockfile::for_path(path).ok_or_else(|| unsupported(format!("{path} is not a lockfile g1t can update")))?;
414 if lockfile.ecosystem() != ecosystem {
415 bail!("{path} is not a {} lockfile", ecosystem.osv());
416 }
417 let dir = path.rsplit_once('/').map(|(dir, _)| dir.to_owned()).unwrap_or_default();
418 let lockfile = if lockfile == Lockfile::GoSum { Lockfile::GoMod } else { lockfile };
419 match jobs.iter_mut().find(|job| job.dir == dir && job.lockfile == lockfile) {
420 Some(job) => {
421 if !job.paths.contains(path) {
422 job.paths.push(path.clone());
423 }
424 }
425 None => jobs.push(Job { dir, lockfile, paths: vec![path.clone()] }),
426 }
427 }
428 if jobs.is_empty() {
429 bail!("BUMP_LOCKFILES names no lockfile");
430 }
431 Ok(jobs)
432}
433
434/// A version as the ecosystem's tools take it: Go's with a leading `v`,
435/// everyone else's without.
436fn tool_version(ecosystem: Ecosystem, version: &str) -> String {
437 let version = version.trim();
438 let bare = match version.strip_prefix(['v', 'V']) {
439 Some(rest) if rest.starts_with(|c: char| c.is_ascii_digit()) => rest,
440 _ => version,
441 };
442 match ecosystem {
443 Ecosystem::Go => format!("v{bare}"),
444 _ => bare.to_owned(),
445 }
446}
447
448/// A package name or version is passed to tools as one argument: it must
449/// not read as an option, and holds only what names and versions do.
450fn safe_argument(what: &str, text: &str) -> Result<()> {
451 let allowed = |c: char| c.is_ascii_alphanumeric() || "@/._-+~".contains(c);
452 if text.is_empty() || text.starts_with('-') || !text.chars().all(allowed) || text.len() > 214 {
453 bail!("{what} {text:?} is not a package name or version g1t can pass to a tool");
454 }
455 Ok(())
456}
457
458/// The range to ask for a direct dependency now at `current`: the same
459/// style (`^1.2.3`, `~1.2.3`, exact), and `^` for any other.
460fn raised_range(current: &str, version: &str) -> String {
461 let current = current.trim();
462 if current.starts_with('~') {
463 format!("~{version}")
464 } else if current.starts_with(|c: char| c.is_ascii_digit()) || current.starts_with('=') {
465 version.to_owned()
466 } else {
467 format!("^{version}")
468 }
469}
470
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar471/// The range `widen` asks for: what `current` allowed, or the raised range.
472fn widened_range(current: &str, version: &str) -> String {
473 format!("{} || {}", current.trim(), raised_range(current, version))
474}
475
476/// How a direct JavaScript dependency is raised: first the update its
477/// range allows, or not, then the range to ask for if still below, if any.
478#[derive(Debug, PartialEq, Eq)]
479struct DirectPlan {
480 in_range_first: bool,
481 range: Option<String>,
482}
483
484fn direct_plan(strategy: Option<Strategy>, current: &str, version: &str) -> DirectPlan {
485 match strategy {
486 None | Some(Strategy::Increase) => DirectPlan { in_range_first: false, range: Some(raised_range(current, version)) },
487 Some(Strategy::IncreaseIfNecessary) => DirectPlan { in_range_first: true, range: Some(raised_range(current, version)) },
488 Some(Strategy::Widen) => DirectPlan { in_range_first: true, range: Some(widened_range(current, version)) },
489 Some(Strategy::LockfileOnly) => DirectPlan { in_range_first: true, range: None },
490 }
491}
492
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily493/// The range `package.json` asks for `package` with, if it is a direct
494/// dependency from the registry (not a workspace, link, alias or URL).
495fn direct_range(manifest: &Value, package: &str) -> Option<String> {
496 ["dependencies", "devDependencies", "optionalDependencies"].iter().find_map(|section| {
497 let range = manifest.get(section)?.get(package)?.as_str()?;
498 let registry = !range.contains(':') && !range.contains('/');
499 registry.then(|| range.to_owned())
500 })
501}
502
503/// Which JavaScript package manager wrote a lockfile.
504#[derive(Clone, Copy, Debug, PartialEq, Eq)]
505enum Node {
506 Npm,
507 Pnpm,
508 /// Yarn 1.
509 YarnClassic,
510 /// Yarn 2 and later, whose lockfile has `__metadata`.
511 YarnBerry,
512}
513
514impl Node {
515 fn of(lockfile: Lockfile, text: &str) -> Option<Node> {
516 Some(match lockfile {
517 Lockfile::PackageLock => Node::Npm,
518 Lockfile::PnpmLock => Node::Pnpm,
519 Lockfile::YarnLock if text.lines().any(|line| line.starts_with("__metadata:")) => Node::YarnBerry,
520 Lockfile::YarnLock => Node::YarnClassic,
521 _ => return None,
522 })
523 }
524
525 /// The command, through corepack for pnpm and yarn, so the version the
526 /// project's `packageManager` names is the one that runs.
527 fn command(self, args: &[&str]) -> Vec<String> {
528 let mut command: Vec<&str> = match self {
529 Node::Npm => vec!["npm"],
530 Node::Pnpm => vec!["corepack", "pnpm"],
531 Node::YarnClassic | Node::YarnBerry => vec!["corepack", "yarn"],
532 };
533 command.extend(args);
534 command.into_iter().map(str::to_owned).collect()
535 }
536
537 /// Raises a direct dependency to `range`.
538 fn direct(self, package: &str, range: &str) -> Vec<String> {
539 let spec = format!("{package}@{range}");
540 match self {
541 Node::Npm => self.command(&["install", "--package-lock-only", "--ignore-scripts", "--no-audit", "--no-fund", &spec]),
542 Node::Pnpm => self.command(&["update", &spec, "--lockfile-only", "--ignore-scripts"]),
543 Node::YarnBerry => self.command(&["up", &spec, "--mode=update-lockfile"]),
544 Node::YarnClassic => self.command(&["upgrade", &spec, "--ignore-scripts", "--non-interactive"]),
545 }
546 }
547
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar548 /// Moves a direct dependency as far as its range allows, leaving the
549 /// range in `package.json` as it is.
550 fn in_range(self, package: &str) -> Vec<String> {
551 match self {
552 Node::Npm => self.command(&["update", "--package-lock-only", "--no-save", "--ignore-scripts", "--no-audit", "--no-fund", package]),
553 Node::Pnpm => self.command(&["update", package, "--lockfile-only", "--no-save", "--ignore-scripts"]),
554 Node::YarnBerry => self.command(&["up", "--recursive", package, "--mode=update-lockfile"]),
555 Node::YarnClassic => self.command(&["upgrade", package, "--ignore-scripts", "--non-interactive"]),
556 }
557 }
558
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily559 /// Moves a package something else depends on as far as the ranges
560 /// that ask for it allow. Yarn 1 has no such command.
561 fn transitive(self, package: &str) -> Option<Vec<String>> {
562 Some(match self {
563 Node::Npm => self.command(&["update", "--package-lock-only", "--ignore-scripts", "--no-audit", "--no-fund", package]),
564 Node::Pnpm => self.command(&["update", package, "--depth", "Infinity", "--lockfile-only", "--ignore-scripts"]),
565 Node::YarnBerry => self.command(&["up", "--recursive", package, "--mode=update-lockfile"]),
566 Node::YarnClassic => return None,
567 })
568 }
569
570 /// Where `package.json` forces a version on everything that asks for
571 /// a package.
572 fn override_path(self) -> &'static [&'static str] {
573 match self {
574 Node::Npm => &["overrides"],
575 Node::Pnpm => &["pnpm", "overrides"],
576 Node::YarnClassic | Node::YarnBerry => &["resolutions"],
577 }
578 }
579
580 /// Writes the lockfile again from `package.json`.
581 fn relock(self) -> Vec<String> {
582 match self {
583 Node::Npm => self.command(&["install", "--package-lock-only", "--ignore-scripts", "--no-audit", "--no-fund"]),
584 Node::Pnpm => self.command(&["install", "--lockfile-only", "--ignore-scripts"]),
585 Node::YarnBerry => self.command(&["install", "--mode=update-lockfile"]),
586 Node::YarnClassic => self.command(&["install", "--ignore-scripts", "--non-interactive"]),
587 }
588 }
589}
590
591/// Adds `package: version` to the overrides at `path` in `package.json`,
592/// creating the objects on the way. Returns false when it is already there.
593fn add_override(manifest: &mut Value, path: &[&str], package: &str, version: &str) -> Result<bool> {
594 let mut at = manifest;
595 for key in path {
596 let object = at.as_object_mut().ok_or_else(|| anyhow!("package.json is not an object"))?;
597 at = object.entry(key.to_string()).or_insert_with(|| json!({}));
598 }
599 let object = at.as_object_mut().ok_or_else(|| anyhow!("package.json's {} is not an object", path.join(".")))?;
600 if object.get(package).and_then(Value::as_str) == Some(version) {
601 return Ok(false);
602 }
603 object.insert(package.to_owned(), Value::String(version.to_owned()));
604 Ok(true)
605}
606
607/// What Cargo runs for each locked version of `package` below `version`:
608/// straight to it, or, when that is past what a dependent's requirement
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar609/// allows, as far as the requirement does. `config` is a configuration
610/// file every cargo run is given (a private registry's source replacement).
611fn cargo_commands(package: &str, old: &str, version: &str, config: Option<&str>) -> [Vec<String>; 2] {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily612 let spec = format!("{package}@{old}");
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar613 let cargo = || -> Vec<String> {
614 let mut command = vec!["cargo".to_owned()];
615 if let Some(config) = config {
616 command.extend(["--config".to_owned(), config.to_owned()]);
617 }
618 command
619 };
620 let mut precise = cargo();
621 precise.extend(["update", "-p", &spec, "--precise", version].map(str::to_owned));
622 let mut compatible = cargo();
623 compatible.extend(["update", "-p", &spec].map(str::to_owned));
624 [precise, compatible]
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily625}
626
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar627/// A Cargo requirement raised to `version`, keeping its operator (`^`, `~`,
628/// `=`, `>=` or none), or `None` when it is not a single requirement below
629/// `version`.
630fn raised_requirement(requirement: &str, version: &str) -> Option<String> {
631 let requirement = requirement.trim();
632 let (operator, rest) = [">=", "^", "~", "="]
633 .iter()
634 .find_map(|operator| requirement.strip_prefix(operator).map(|rest| (*operator, rest)))
635 .unwrap_or(("", requirement));
636 let rest = rest.trim();
637 let plain = !rest.is_empty() && rest.starts_with(|c: char| c.is_ascii_digit()) && rest.chars().all(|c| c.is_ascii_alphanumeric() || ".-+".contains(c));
638 (plain && version::compare(rest, version).is_lt()).then(|| format!("{operator}{version}"))
639}
640
641/// The dotted keys of a TOML table header (`[target.'cfg(unix)'.dependencies]`),
642/// unquoted; `None` for an array of tables or a line that is not a header.
643fn header_keys(line: &str) -> Option<Vec<String>> {
644 let code = line.trim_start();
645 if code.starts_with("[[") || !code.starts_with('[') {
646 return None;
647 }
648 let mut keys = Vec::new();
649 let mut key = String::new();
650 let mut quote: Option<char> = None;
651 for c in code[1..].chars() {
652 match (quote, c) {
653 (Some(q), c) if c == q => quote = None,
654 (Some(_), c) => key.push(c),
655 (None, '"' | '\'') => quote = Some(c),
656 (None, '.') => keys.push(std::mem::take(&mut key).trim().to_owned()),
657 (None, ']') => {
658 keys.push(key.trim().to_owned());
659 return Some(keys);
660 }
661 (None, c) => key.push(c),
662 }
663 }
664 None
665}
666
667/// What a table header is to dependencies: `Some(None)` for a table of
668/// them (`[dependencies]`, `[workspace.dependencies]`,
669/// `[target.<cfg>.dev-dependencies]` …), `Some(Some(name))` for one
670/// dependency's own table (`[dependencies.serde]`), `None` for anything else.
671fn dependency_table(keys: &[String]) -> Option<Option<String>> {
672 let tables = ["dependencies", "dev-dependencies", "build-dependencies"];
673 let rest = match keys {
674 [first, rest @ ..] if tables.contains(&first.as_str()) => rest,
675 [workspace, dependencies, rest @ ..] if workspace == "workspace" && dependencies == "dependencies" => rest,
676 [target, _, table, rest @ ..] if target == "target" && tables.contains(&table.as_str()) => rest,
677 _ => return None,
678 };
679 match rest {
680 [] => Some(None),
681 [name] => Some(Some(name.clone())),
682 _ => None,
683 }
684}
685
686/// Where the value of `key` starts in a one-line TOML `line`: just after
687/// its `=`, if `key` is a bare key there (first, or after `{` or `,`).
688fn value_after(line: &str, key: &str) -> Option<usize> {
689 let mut from = 0;
690 while let Some(at) = line[from..].find(key).map(|at| from + at) {
691 let before = line[..at].trim_end();
692 let after = line[at + key.len()..].trim_start();
693 let bare_key = before.is_empty() || before.ends_with('{') || before.ends_with(',');
694 if bare_key && after.starts_with('=') {
695 let equals = line.len() - after.len();
696 return Some(equals + 1);
697 }
698 from = at + key.len();
699 }
700 None
701}
702
703/// `line` with the first quoted `old` from `from` on replaced by `new`.
704fn replace_quoted(line: &str, from: usize, old: &str, new: &str) -> Option<String> {
705 ['"', '\''].iter().find_map(|quote| {
706 let quoted = format!("{quote}{old}{quote}");
707 let at = from + line[from..].find(&quoted)?;
708 Some(format!("{}{quote}{new}{quote}{}", &line[..at], &line[at + quoted.len()..]))
709 })
710}
711
712/// One `key = value` line of a dependency table, rewritten when it asks
713/// for `package` with a requirement below `version`.
714fn rewrite_dependency_line(line: &str, package: &str, version: &str) -> Option<String> {
715 let table: toml::Table = toml::from_str(line).ok()?;
716 let (key, value) = table.iter().next()?;
717 let (name, requirement, anchor) = match value {
718 toml::Value::String(requirement) => (key.as_str(), requirement.as_str(), line.find('=')? + 1),
719 toml::Value::Table(inline) => {
720 let name = inline.get("package").and_then(toml::Value::as_str).unwrap_or(key);
721 let requirement = inline.get("version").and_then(toml::Value::as_str)?;
722 (name, requirement, value_after(line, "version")?)
723 }
724 _ => return None,
725 };
726 if name != package {
727 return None;
728 }
729 replace_quoted(line, anchor, requirement, &raised_requirement(requirement, version)?)
730}
731
732/// A `Cargo.toml` with every requirement on `package` below `version`
733/// raised to it, keeping its operator, and nothing else changed. Returns
734/// the text and how many requirements moved.
735fn rewrite_cargo_requirements(text: &str, package: &str, version: &str) -> (String, usize) {
736 // Each section: its header's meaning to dependencies, and its lines.
737 let mut sections: Vec<(Option<Option<String>>, Vec<String>)> = vec![(None, Vec::new())];
738 for line in text.split_inclusive('\n') {
739 if let Some(keys) = header_keys(line) {
740 sections.push((dependency_table(&keys), Vec::new()));
741 } else if line.trim_start().starts_with("[[") {
742 sections.push((None, Vec::new()));
743 }
744 sections.last_mut().expect("a section").1.push(line.to_owned());
745 }
746 let mut moved = 0;
747 let mut out = String::with_capacity(text.len() + 8);
748 for (table, mut lines) in sections {
749 match table {
750 Some(None) => {
751 for line in lines.iter_mut().skip(1) {
752 if let Some(rewritten) = rewrite_dependency_line(line, package, version) {
753 *line = rewritten;
754 moved += 1;
755 }
756 }
757 }
758 Some(Some(key)) => {
759 let field = |name: &str| {
760 lines.iter().enumerate().skip(1).find_map(|(at, line)| {
761 let table: toml::Table = toml::from_str(line).ok()?;
762 Some((at, table.get(name)?.as_str()?.to_owned()))
763 })
764 };
765 let name = field("package").map_or(key, |(_, name)| name);
766 if let (true, Some((at, requirement))) = (name == package, field("version")) {
767 let rewritten = raised_requirement(&requirement, version).and_then(|raised| {
768 let line = &lines[at];
769 replace_quoted(line, line.find('=')? + 1, &requirement, &raised)
770 });
771 if let Some(rewritten) = rewritten {
772 lines[at] = rewritten;
773 moved += 1;
774 }
775 }
776 }
777 None => {}
778 }
779 lines.iter().for_each(|line| out.push_str(line));
780 }
781 (out, moved)
782}
783
784/// The workspace members a root `Cargo.toml` lists, as written: paths, and
785/// paths ending in `/*`. Other globs and paths outside it are left out.
786fn workspace_members(root: &toml::Value) -> Vec<String> {
787 let Some(members) = root.get("workspace").and_then(|workspace| workspace.get("members")).and_then(toml::Value::as_array) else {
788 return Vec::new();
789 };
790 members
791 .iter()
792 .filter_map(toml::Value::as_str)
793 .map(|member| member.trim().trim_start_matches("./").trim_end_matches('/').to_owned())
794 .filter(|member| {
795 let globbed = member.strip_suffix("/*").unwrap_or(member);
796 !member.is_empty() && !globbed.contains(['*', '?', '[']) && !member.starts_with('/') && !member.split('/').any(|part| part == "..")
797 })
798 .collect()
799}
800
801/// The `Cargo.toml` files of a Cargo job, from the repository's root: its
802/// directory's and its workspace members'.
803fn cargo_manifests(workdir: &Path, job: &Job) -> Vec<String> {
804 let root = job.file("Cargo.toml");
805 let mut found = vec![root.clone()];
806 let Ok(text) = std::fs::read_to_string(workdir.join(&root)) else {
807 return found;
808 };
809 let Ok(parsed) = toml::from_str::<toml::Value>(&text) else {
810 return found;
811 };
812 for member in workspace_members(&parsed) {
813 let dirs: Vec<String> = match member.strip_suffix("/*") {
814 Some(parent) => {
815 let parent = job.file(parent);
816 let mut names: Vec<String> = std::fs::read_dir(workdir.join(&parent))
817 .map(|entries| {
818 entries
819 .filter_map(|entry| entry.ok())
820 .filter(|entry| entry.path().is_dir())
821 .filter_map(|entry| entry.file_name().into_string().ok())
822 .map(|name| format!("{parent}/{name}"))
823 .collect()
824 })
825 .unwrap_or_default();
826 names.sort();
827 names
828 }
829 None => vec![job.file(&member)],
830 };
831 for dir in dirs {
832 let manifest = format!("{dir}/Cargo.toml");
833 if workdir.join(&manifest).is_file() && !found.contains(&manifest) {
834 found.push(manifest);
835 }
836 }
837 }
838 found
839}
840
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily841fn go_commands(module: &str, version: &str) -> [Vec<String>; 2] {
842 [
843 vec!["go".into(), "get".into(), format!("{module}@{version}")],
844 ["go", "mod", "tidy"].map(str::to_owned).to_vec(),
845 ]
846}
847
848/// Which dependency group of `pyproject.toml` names `package`: `Some(None)`
849/// for the main one, `Some(Some(group))` for another, `None` when it is not
850/// a direct dependency.
851fn poetry_group(pyproject: &toml::Value, package: &str) -> Option<Option<String>> {
852 let wanted = Ecosystem::PyPI.normalize(package);
853 let names = |table: Option<&toml::Value>| -> bool {
854 table
855 .and_then(toml::Value::as_table)
856 .is_some_and(|table| table.keys().any(|key| Ecosystem::PyPI.normalize(key) == wanted))
857 };
858 let poetry = pyproject.get("tool").and_then(|tool| tool.get("poetry"));
859 if names(poetry.and_then(|poetry| poetry.get("dependencies"))) {
860 return Some(None);
861 }
862 let pep621 = pyproject
863 .get("project")
864 .and_then(|project| project.get("dependencies"))
865 .and_then(toml::Value::as_array)
866 .is_some_and(|list| {
867 list.iter().filter_map(toml::Value::as_str).any(|requirement| {
868 let name: String = requirement.chars().take_while(|c| c.is_ascii_alphanumeric() || "-_.".contains(*c)).collect();
869 Ecosystem::PyPI.normalize(&name) == wanted
870 })
871 });
872 if pep621 {
873 return Some(None);
874 }
875 if names(poetry.and_then(|poetry| poetry.get("dev-dependencies"))) {
876 return Some(Some("dev".to_owned()));
877 }
878 let groups = poetry.and_then(|poetry| poetry.get("group")).and_then(toml::Value::as_table)?;
879 groups
880 .iter()
881 .find(|(_, group)| names(group.get("dependencies")))
882 .map(|(name, _)| Some(name.clone()))
883}
884
885fn poetry_commands(poetry: &[String], package: &str, version: &str, group: Option<Option<String>>) -> Vec<String> {
886 let mut command = poetry.to_vec();
887 match group {
888 Some(group) => {
889 command.extend(["add".to_owned(), format!("{package}@^{version}"), "--lock".to_owned()]);
890 if let Some(group) = group {
891 command.extend(["--group".to_owned(), group]);
892 }
893 }
894 None => command.extend(["update".to_owned(), "--lock".to_owned(), package.to_owned()]),
895 }
896 command
897}
898
899/// `requirements.txt` with every `==` (or `===`) pin of `package` below
900/// `version` raised to it, and nothing else changed. Returns the text and
901/// how many pins moved.
902fn rewrite_pins(text: &str, package: &str, version: &str) -> (String, usize) {
903 let wanted = Ecosystem::PyPI.normalize(package);
904 let mut moved = 0;
905 let mut out = String::with_capacity(text.len() + 8);
906 for line in text.split_inclusive('\n') {
907 let rewritten = (|| {
908 let code = line.split('#').next().unwrap_or_default();
909 let trimmed = code.trim_start();
910 if trimmed.starts_with('-') || code.contains("://") {
911 return None;
912 }
913 let operator = code.find("===").map(|at| (at, 3)).or_else(|| code.find("==").map(|at| (at, 2)))?;
914 let name = code[..operator.0].split('[').next().unwrap_or_default().trim();
915 if Ecosystem::PyPI.normalize(name) != wanted {
916 return None;
917 }
918 let start = operator.0 + operator.1;
919 let rest = &code[start..];
920 let leading = rest.len() - rest.trim_start().len();
921 let from = start + leading;
922 let end = code[from..]
923 .find(|c: char| c.is_whitespace() || ",;\\".contains(c))
924 .map_or(code.len(), |at| from + at);
925 let old = &line[from..end];
926 if old.is_empty() || old.contains('*') || version::compare(old, version).is_ge() {
927 return None;
928 }
929 Some(format!("{}{version}{}", &line[..from], &line[end..]))
930 })();
931 match rewritten {
932 Some(line) => {
933 moved += 1;
934 out.push_str(&line);
935 }
936 None => out.push_str(line),
937 }
938 }
939 (out, moved)
940}
941
942/// The versions of `package` a lockfile still resolves below `version`.
943fn below(lockfile: Lockfile, text: &str, ecosystem: Ecosystem, package: &str, version: &str) -> Vec<String> {
944 let name = ecosystem.normalize(package);
945 let found: BTreeSet<String> = lockfile
946 .parse(text)
947 .into_iter()
948 .filter(|found| found.name == name && version::compare(&found.version, version).is_lt())
949 .map(|found| found.version)
950 .collect();
951 found.into_iter().collect()
952}
953
954/// The last lines of what a tool said, for the reason it failed.
955fn tail(text: &str, lines: usize) -> String {
956 let all: Vec<&str> = text.lines().filter(|line| !line.trim().is_empty()).collect();
957 all[all.len().saturating_sub(lines)..].join("\n")
958}
959
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar960/// The registries a project names itself, so their credentials can be
961/// given under its names: Cargo's `[registries.<name>] index` and Poetry's
962/// `[[tool.poetry.source]]`, each as (name, URL).
963#[derive(Debug, Default, PartialEq, Eq)]
964struct Named {
965 cargo: Vec<(String, String)>,
966 poetry: Vec<(String, String)>,
967}
968
969/// `[registries.<name>] index = "…"` in a `.cargo/config.toml`.
970fn cargo_registries(config: &str) -> Vec<(String, String)> {
971 let Ok(config) = toml::from_str::<toml::Value>(config) else {
972 return Vec::new();
973 };
974 let Some(registries) = config.get("registries").and_then(toml::Value::as_table) else {
975 return Vec::new();
976 };
977 registries
978 .iter()
979 .filter_map(|(name, registry)| Some((name.clone(), registry.get("index")?.as_str()?.to_owned())))
980 .collect()
981}
982
983/// `[[tool.poetry.source]]` names and URLs in a `pyproject.toml`.
984fn poetry_sources(pyproject: &str) -> Vec<(String, String)> {
985 let Ok(pyproject) = toml::from_str::<toml::Value>(pyproject) else {
986 return Vec::new();
987 };
988 let Some(sources) = pyproject.get("tool").and_then(|tool| tool.get("poetry")).and_then(|poetry| poetry.get("source")).and_then(toml::Value::as_array) else {
989 return Vec::new();
990 };
991 sources
992 .iter()
993 .filter_map(|source| Some((source.get("name")?.as_str()?.to_owned(), source.get("url")?.as_str()?.to_owned())))
994 .collect()
995}
996
997/// A registry URL as compared with another: no index protocol, scheme or
998/// host case, or trailing slashes.
999fn same_registry(a: &str, b: &str) -> bool {
1000 let plain = |url: &str| -> String {
1001 let url = url.trim();
1002 let url = url.strip_prefix("sparse+").or_else(|| url.strip_prefix("registry+")).unwrap_or(url);
1003 url.trim_end_matches('/').trim_end_matches(".git").to_ascii_lowercase()
1004 };
1005 plain(a) == plain(b)
1006}
1007
1008/// A name as an environment variable's part: upper case, with `-` and `.`
1009/// as `_`.
1010fn env_part(name: &str) -> String {
1011 name.to_ascii_uppercase().replace(['-', '.'], "_")
1012}
1013
1014/// Percent-encodes a URL's user or password.
1015fn percent_encode(text: &str) -> String {
1016 text.bytes()
1017 .map(|byte| match byte {
1018 b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' => (byte as char).to_string(),
1019 _ => format!("%{byte:02X}"),
1020 })
1021 .collect()
1022}
1023
1024/// `https://host/path` as npm keys its credentials: `//host/path/`.
1025fn npm_nerf_dart(url: &str) -> String {
1026 let rest = url.trim().strip_prefix("https://").unwrap_or(url);
1027 let rest = rest.trim_end_matches('/');
1028 format!("//{rest}/")
1029}
1030
1031/// A registry URL's host, for a netrc entry.
1032fn url_host(url: &str) -> &str {
1033 let rest = url.trim().strip_prefix("https://").unwrap_or(url);
1034 let host = rest.split(['/', '?', '#']).next().unwrap_or(rest);
1035 host.rsplit('@').next().unwrap_or(host).split(':').next().unwrap_or(host)
1036}
1037
1038/// Checks one registry before anything is written: a kind this can
1039/// configure, an `https://` URL and text that cannot break out of a
1040/// configuration line.
1041fn check_registry(registry: &Registry) -> Result<()> {
1042 let kinds = ["npm-registry", "cargo-registry", "python-index", "goproxy-server"];
1043 if !kinds.contains(&registry.kind.as_str()) {
1044 bail!("g1t cannot read a {} registry", registry.kind);
1045 }
1046 let url = registry.url.trim();
1047 if !url.starts_with("https://") || url_host(url).is_empty() || url.chars().any(|c| c.is_whitespace() || c.is_control() || c == ',') {
1048 bail!("a private registry's URL must start with https://, without spaces or commas");
1049 }
1050 let secrets = [&registry.username, &registry.password, &registry.token];
1051 if secrets.iter().filter_map(|secret| secret.as_deref()).any(|secret| secret.chars().any(|c| c.is_control())) {
1052 bail!("a private registry's credentials must not hold control characters ({url})");
1053 }
1054 if registry.kind == "goproxy-server" && secrets.iter().filter_map(|secret| secret.as_deref()).any(|secret| secret.chars().any(char::is_whitespace)) {
1055 bail!("a Go proxy's credentials must not hold spaces ({url})");
1056 }
1057 if let Some(scope) = registry.scopes.iter().find(|scope| !scope.starts_with('@') || scope.len() < 2 || !scope[1..].chars().all(|c| c.is_ascii_alphanumeric() || "._-".contains(c))) {
1058 bail!("{scope:?} is not an npm scope");
1059 }
1060 Ok(())
1061}
1062
1063/// What the tools are given to read private registries: variables for
1064/// every run, files written outside the clone, and a configuration file
1065/// every cargo run is given.
1066#[derive(Default)]
1067struct Tools {
1068 env: Vec<(String, String)>,
1069 files: Vec<(PathBuf, String)>,
1070 cargo_config: Option<String>,
1071}
1072
1073impl Tools {
1074 fn set(&mut self, name: impl Into<String>, value: impl Into<String>) {
1075 self.env.push((name.into(), value.into()));
1076 }
1077
1078 fn get(&self, name: &str) -> Option<&str> {
1079 self.env.iter().find(|(key, _)| key == name).map(|(_, value)| value.as_str())
1080 }
1081
1082 /// Writes the files, readable by their owner only.
1083 fn write(&self) -> Result<()> {
1084 for (path, text) in &self.files {
1085 if let Some(parent) = path.parent() {
1086 std::fs::create_dir_all(parent).with_context(|| format!("could not create {}", parent.display()))?;
1087 }
1088 std::fs::write(path, text).with_context(|| format!("could not write {}", path.display()))?;
1089 #[cfg(unix)]
1090 {
1091 use std::os::unix::fs::PermissionsExt;
1092 std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600))?;
1093 }
1094 }
1095 Ok(())
1096 }
1097}
1098
1099/// The configuration that lets the tools read `registries`, with files
1100/// under `dir`. `named` are the registries the project names itself.
1101fn registry_tools(registries: &[Registry], named: &Named, dir: &Path) -> Result<Tools> {
1102 let mut tools = Tools::default();
1103 let mut npmrc = String::new();
1104 let mut extra_indexes: Vec<String> = Vec::new();
1105 let mut proxies: Vec<String> = Vec::new();
1106 let mut proxy_replaced = false;
1107 let mut netrc = String::new();
1108 let mut cargo_config: Option<toml::Table> = None;
1109 for registry in registries {
1110 check_registry(registry)?;
1111 let url = registry.url.trim();
1112 match registry.kind.as_str() {
1113 "npm-registry" => {
1114 let nerf = npm_nerf_dart(url);
1115 let auth = match (registry.token.as_deref(), registry.username.as_deref(), registry.password.as_deref()) {
1116 (Some(token), _, _) if !token.is_empty() => Some(("_authToken", token.to_owned(), "YARN_NPM_AUTH_TOKEN", token.to_owned())),
1117 (_, Some(user), Some(password)) => {
1118 let ident = format!("{user}:{password}");
1119 let encoded = base64::engine::general_purpose::STANDARD.encode(&ident);
1120 Some(("_auth", encoded, "YARN_NPM_AUTH_IDENT", ident))
1121 }
1122 _ => None,
1123 };
1124 if let Some((key, value, _, _)) = &auth {
1125 npmrc.push_str(&format!("{nerf}:{key}={value}\n"));
1126 }
1127 if registry.replaces_base {
1128 npmrc.push_str(&format!("registry={url}\n"));
1129 if tools.get("YARN_NPM_REGISTRY_SERVER").is_none() {
1130 tools.set("YARN_NPM_REGISTRY_SERVER", url);
1131 if let Some((_, _, name, value)) = auth {
1132 tools.set(name, value);
1133 tools.set("YARN_NPM_ALWAYS_AUTH", "true");
1134 }
1135 }
1136 }
1137 for scope in &registry.scopes {
1138 npmrc.push_str(&format!("{scope}:registry={url}\n"));
1139 }
1140 }
1141 "cargo-registry" => {
1142 let secret = registry.secret();
1143 for (name, _) in named.cargo.iter().filter(|(_, index)| same_registry(index, url)) {
1144 if let Some(secret) = secret {
1145 tools.set(format!("CARGO_REGISTRIES_{}_TOKEN", env_part(name)), secret);
1146 }
1147 }
1148 if registry.replaces_base && cargo_config.is_none() {
1149 let index = if url.ends_with(".git") { url.to_owned() } else { format!("sparse+{}/", url.trim_end_matches('/')) };
1150 let table = |pairs: &[(&str, toml::Value)]| -> toml::Value {
1151 toml::Value::Table(pairs.iter().map(|(key, value)| (key.to_string(), value.clone())).collect())
1152 };
1153 let text = |text: &str| toml::Value::String(text.to_owned());
1154 let mut config = toml::Table::new();
1155 config.insert(
1156 "source".into(),
1157 table(&[
1158 ("crates-io", table(&[("replace-with", text("g1t-private"))])),
1159 ("g1t-private", table(&[("registry", text(&index))])),
1160 ]),
1161 );
1162 config.insert("registries".into(), table(&[("g1t-private", table(&[("index", text(&index))]))]));
1163 cargo_config = Some(config);
1164 if let Some(secret) = secret {
1165 tools.set("CARGO_REGISTRIES_G1T_PRIVATE_TOKEN", secret);
1166 }
1167 }
1168 }
1169 "python-index" => {
1170 let user = registry.username.as_deref().filter(|user| !user.is_empty());
1171 let password = registry.password.as_deref().or(registry.token.as_deref()).filter(|password| !password.is_empty());
1172 let with_auth = match (user, password) {
1173 (user, Some(password)) => {
1174 let rest = url.strip_prefix("https://").unwrap_or(url);
1175 format!("https://{}:{}@{rest}", percent_encode(user.unwrap_or("__token__")), percent_encode(password))
1176 }
1177 (Some(user), None) => format!("https://{}@{}", percent_encode(user), url.strip_prefix("https://").unwrap_or(url)),
1178 (None, None) => url.to_owned(),
1179 };
1180 if registry.replaces_base && tools.get("PIP_INDEX_URL").is_none() {
1181 tools.set("PIP_INDEX_URL", with_auth);
1182 } else {
1183 extra_indexes.push(with_auth);
1184 }
1185 for (name, _) in named.poetry.iter().filter(|(_, source)| same_registry(source, url)) {
1186 if let Some(password) = password {
1187 tools.set(format!("POETRY_HTTP_BASIC_{}_USERNAME", env_part(name)), user.unwrap_or("__token__"));
1188 tools.set(format!("POETRY_HTTP_BASIC_{}_PASSWORD", env_part(name)), password);
1189 }
1190 }
1191 }
1192 "goproxy-server" => {
1193 proxies.push(url.to_owned());
1194 proxy_replaced |= registry.replaces_base;
1195 if let Some(secret) = registry.secret() {
1196 let login = registry.username.as_deref().filter(|user| !user.is_empty()).unwrap_or("g1t");
1197 netrc.push_str(&format!("machine {}\nlogin {login}\npassword {secret}\n", url_host(url)));
1198 }
1199 }
1200 _ => unreachable!("checked above"),
1201 }
1202 }
1203 if !npmrc.is_empty() {
1204 let path = dir.join("npmrc");
1205 tools.set("NPM_CONFIG_USERCONFIG", path.display().to_string());
1206 tools.files.push((path, npmrc));
1207 }
1208 if !extra_indexes.is_empty() {
1209 tools.set("PIP_EXTRA_INDEX_URL", extra_indexes.join(" "));
1210 }
1211 if !proxies.is_empty() {
1212 let tail = if proxy_replaced { "direct" } else { "https://proxy.golang.org,direct" };
1213 tools.set("GOPROXY", format!("{},{tail}", proxies.join(",")));
1214 }
1215 if !netrc.is_empty() {
1216 let path = dir.join("netrc");
1217 tools.set("NETRC", path.display().to_string());
1218 tools.files.push((path, netrc));
1219 }
1220 if let Some(config) = cargo_config {
1221 let path = dir.join("cargo.toml");
1222 tools.cargo_config = Some(path.display().to_string());
1223 tools.files.push((path, toml::to_string(&config)?));
1224 }
1225 Ok(tools)
1226}
1227
1228/// The registries the clone names itself: Cargo's in `.cargo/config.toml`
1229/// at the root and in each job's directory, Poetry's in each
1230/// `pyproject.toml` updated.
1231fn named_registries(workdir: &Path, jobs: &[Job]) -> Named {
1232 let mut named = Named::default();
1233 let mut dirs: Vec<&str> = vec![""];
1234 dirs.extend(jobs.iter().map(|job| job.dir.as_str()));
1235 dirs.dedup();
1236 for dir in dirs {
1237 for name in [".cargo/config.toml", ".cargo/config"] {
1238 if let Ok(text) = std::fs::read_to_string(workdir.join(dir).join(name)) {
1239 named.cargo.extend(cargo_registries(&text));
1240 }
1241 }
1242 }
1243 for job in jobs.iter().filter(|job| job.lockfile == Lockfile::PoetryLock) {
1244 if let Ok(text) = std::fs::read_to_string(workdir.join(job.file("pyproject.toml"))) {
1245 named.poetry.extend(poetry_sources(&text));
1246 }
1247 }
1248 named
1249}
1250
1251/// Runs a tool in `dir` with `extra` variables and returns what it said,
1252/// failing with the last lines of its output. A tool that is not installed
1253/// is unsupported. The variables are never printed.
1254fn run(dir: &Path, command: &[String], extra: &[(String, String)]) -> Result<String> {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1255 let (program, args) = command.split_first().ok_or_else(|| anyhow!("no command"))?;
1256 eprintln!("g1t-runner: {} (in {})", command.join(" "), dir.display());
1257 let output = Command::new(program)
1258 .current_dir(dir)
1259 .args(args)
1260 // Lockfiles only: nothing installs, prompts, audits or runs scripts.
1261 .env("CI", "true")
1262 .env("npm_config_audit", "false")
1263 .env("npm_config_fund", "false")
1264 .env("npm_config_update_notifier", "false")
1265 .env("npm_config_ignore_scripts", "true")
1266 .env("COREPACK_ENABLE_DOWNLOAD_PROMPT", "0")
1267 .env("YARN_ENABLE_IMMUTABLE_INSTALLS", "false")
1268 .env("YARN_ENABLE_SCRIPTS", "false")
1269 .env("YARN_ENABLE_TELEMETRY", "0")
1270 .env("POETRY_NO_INTERACTION", "1")
1271 .env("POETRY_VIRTUALENVS_CREATE", "false")
1272 .env("GOFLAGS", "-mod=mod")
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1273 .envs(extra.iter().map(|(name, value)| (name, value)))
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1274 .output()
1275 .map_err(|error| {
1276 if error.kind() == std::io::ErrorKind::NotFound {
1277 unsupported(format!("{program} is not installed in this sandbox"))
1278 } else {
1279 anyhow!("could not run {program}: {error}")
1280 }
1281 })?;
1282 let said = format!("{}\n{}", String::from_utf8_lossy(&output.stdout), String::from_utf8_lossy(&output.stderr));
1283 if !output.status.success() {
1284 bail!("{} failed:\n{}", command.join(" "), tail(&said, 20));
1285 }
1286 Ok(said)
1287}
1288
1289fn read(path: &Path) -> Result<String> {
1290 std::fs::read_to_string(path).with_context(|| format!("could not read {}", path.display()))
1291}
1292
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1293/// Poetry, installed into a virtual environment from PyPI (or the index
1294/// that replaces it) when the sandbox has none.
1295fn poetry(extra: &[(String, String)]) -> Result<Vec<String>> {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1296 if Command::new("poetry").arg("--version").output().is_ok_and(|output| output.status.success()) {
1297 return Ok(vec!["poetry".to_owned()]);
1298 }
1299 let venv = "/tmp/g1t-poetry";
1300 let here = Path::new("/");
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1301 run(here, &["python3", "-m", "venv", venv].map(str::to_owned), extra)?;
1302 run(here, &[format!("{venv}/bin/pip"), "install".into(), "--quiet".into(), "poetry".into()], extra)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1303 Ok(vec![format!("{venv}/bin/poetry")])
1304}
1305
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1306/// What one update of one job did: what the tools said when they failed,
1307/// and the files other than the job's own it changed.
1308#[derive(Default)]
1309struct Outcome {
1310 said: Vec<String>,
1311 files: Vec<String>,
1312}
1313
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1314impl Bump {
1315 fn from_env() -> Result<Bump> {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1316 let optional = |name: &str| std::env::var(name).ok();
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1317 let ecosystem_name = env("BUMP_ECOSYSTEM")?;
1318 let ecosystem = Ecosystem::parse(ecosystem_name.trim())
1319 .ok_or_else(|| unsupported(format!("g1t cannot update {ecosystem_name} dependencies")))?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1320 let kind = Kind::parse(&optional("BUMP_KIND").unwrap_or_default())?;
1321 let packages = targets(
1322 ecosystem,
1323 optional("BUMP_PACKAGES").as_deref(),
1324 optional("BUMP_PACKAGE").as_deref(),
1325 optional("BUMP_VERSION").as_deref(),
1326 )?;
1327 let strategy = Strategy::parse(&optional("BUMP_STRATEGY").unwrap_or_default())?;
1328 let force = matches!(optional("BUMP_FORCE").as_deref().map(str::trim), Some("1" | "true"));
1329 let registries: Vec<Registry> = match optional("BUMP_REGISTRIES").as_deref().map(str::trim).filter(|text| !text.is_empty()) {
1330 Some(text) => serde_json::from_str(text).context("BUMP_REGISTRIES is not a JSON array of registries")?,
1331 None => Vec::new(),
1332 };
1333 registries.iter().try_for_each(check_registry)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1334 let jobs = jobs(ecosystem, &lockfile_list(&env("BUMP_LOCKFILES")?)?)?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1335 Ok(Bump {
1336 ecosystem,
1337 kind,
1338 packages,
1339 strategy: (kind == Kind::Version).then_some(strategy),
1340 force,
1341 registries,
1342 jobs,
1343 })
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1344 }
1345
1346 /// The versions every lockfile of `job` still resolves below the target.
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1347 fn still_below(&self, workdir: &Path, job: &Job, target: &Target) -> Result<Vec<String>> {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1348 let mut found = BTreeSet::new();
1349 for path in &job.paths {
1350 let lockfile = Lockfile::for_path(path).unwrap_or(job.lockfile);
1351 let file = workdir.join(path);
1352 if !file.exists() {
1353 bail!("{path} is not in the repository's default branch");
1354 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1355 found.extend(below(lockfile, &read(&file)?, self.ecosystem, &target.package, &target.version));
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1356 }
1357 Ok(found.into_iter().collect())
1358 }
1359
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1360 /// Runs the tool for one job and one package. Errors from the tool are
1361 /// kept for the reason, should the lockfile still be behind afterwards.
1362 fn update(&self, workdir: &Path, job: &Job, target: &Target, tools: &Tools) -> Result<Outcome> {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1363 let dir = workdir.join(&job.dir);
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1364 let (package, version) = (target.package.as_str(), target.version.as_str());
1365 let mut outcome = Outcome::default();
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1366 let attempt = |command: Vec<String>, said: &mut Vec<String>| -> Result<bool> {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1367 match run(&dir, &command, &tools.env) {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1368 Ok(_) => Ok(true),
1369 Err(error) if error.downcast_ref::<Stop>().is_some() => Err(error),
1370 Err(error) => {
1371 said.push(format!("{error:#}"));
1372 Ok(false)
1373 }
1374 }
1375 };
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1376 let said = &mut outcome.said;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1377 match job.lockfile {
1378 Lockfile::PackageLock | Lockfile::PnpmLock | Lockfile::YarnLock => {
1379 let lock = read(&workdir.join(&job.paths[0]))?;
1380 let node = Node::of(job.lockfile, &lock).ok_or_else(|| anyhow!("not a JavaScript lockfile"))?;
1381 let manifest_path = dir.join("package.json");
1382 let mut manifest: Value = serde_json::from_str(&read(&manifest_path)?).context("package.json is not JSON")?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1383 match direct_range(&manifest, package) {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1384 Some(range) => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1385 let plan = direct_plan(self.strategy, &range, version);
1386 if plan.in_range_first {
1387 attempt(node.in_range(package), said)?;
1388 }
1389 if !plan.in_range_first || !self.still_below(workdir, job, target)?.is_empty() {
1390 match plan.range {
1391 Some(range) => {
1392 attempt(node.direct(package, &range), said)?;
1393 }
1394 None => said.push(format!("{package} {LOCKFILE_ONLY}")),
1395 }
1396 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1397 }
1398 None => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1399 if let Some(command) = node.transitive(package) {
1400 attempt(command, said)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1401 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1402 // Held back by what asks for it: a security update
1403 // forces the version; a version update never does.
1404 if self.kind == Kind::Security && !self.still_below(workdir, job, target)?.is_empty() {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1405 manifest = serde_json::from_str(&read(&manifest_path)?).context("package.json is not JSON")?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1406 if add_override(&mut manifest, node.override_path(), package, version)? {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1407 let indent = if read(&manifest_path)?.contains("\n \"") { " " } else { " " };
1408 write_json(&manifest_path, &manifest, indent)?;
1409 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1410 attempt(node.relock(), said)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1411 }
1412 }
1413 }
1414 }
1415 Lockfile::CargoLock => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1416 let config = tools.cargo_config.as_deref();
1417 let raise = matches!(self.strategy, Some(Strategy::Increase | Strategy::IncreaseIfNecessary | Strategy::Widen));
1418 for old in self.still_below(workdir, job, target)? {
1419 let [precise, compatible] = cargo_commands(package, &old, version, config);
1420 if attempt(precise.clone(), said)? {
1421 continue;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1422 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1423 // The requirement does not allow it: raise it, then try again.
1424 if raise {
1425 let mut raised = 0;
1426 for manifest in cargo_manifests(workdir, job) {
1427 let file = workdir.join(&manifest);
1428 let (text, moved) = rewrite_cargo_requirements(&read(&file)?, package, version);
1429 if moved > 0 {
1430 std::fs::write(&file, text).with_context(|| format!("could not write {manifest}"))?;
1431 outcome.files.push(manifest);
1432 raised += moved;
1433 }
1434 }
1435 if raised > 0 && attempt(precise, said)? {
1436 continue;
1437 }
1438 }
1439 attempt(compatible, said)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1440 }
1441 }
1442 Lockfile::GoMod | Lockfile::GoSum => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1443 for command in go_commands(package, version) {
1444 if !attempt(command, said)? {
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1445 break;
1446 }
1447 }
1448 }
1449 Lockfile::PoetryLock => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1450 let group = if self.strategy == Some(Strategy::LockfileOnly) {
1451 None
1452 } else {
1453 let pyproject: toml::Value = toml::from_str(&read(&dir.join("pyproject.toml"))?).context("pyproject.toml is not TOML")?;
1454 poetry_group(&pyproject, package)
1455 };
1456 attempt(poetry_commands(&poetry(&tools.env)?, package, version, group), said)?;
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1457 }
1458 Lockfile::Requirements => {
1459 for path in &job.paths {
1460 let file = workdir.join(path);
1461 let text = read(&file)?;
1462 if text.contains("--hash") {
1463 return Err(unsupported(format!("{path} pins hashes, which need its compiler to update")));
1464 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1465 let (rewritten, moved) = rewrite_pins(&text, package, version);
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1466 if moved > 0 {
1467 std::fs::write(&file, rewritten).with_context(|| format!("could not write {path}"))?;
1468 }
1469 }
1470 }
1471 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1472 Ok(outcome)
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1473 }
1474}
1475
1476/// Writes JSON as package managers do: indented, with a final newline.
1477fn write_json(path: &Path, value: &Value, indent: &str) -> Result<()> {
1478 let mut out = Vec::new();
1479 let formatter = serde_json::ser::PrettyFormatter::with_indent(indent.as_bytes());
1480 let mut serializer = serde_json::Serializer::with_formatter(&mut out, formatter);
1481 serde::Serialize::serialize(value, &mut serializer)?;
1482 out.push(b'\n');
1483 std::fs::write(path, out).with_context(|| format!("could not write {}", path.display()))
1484}
1485
1486/// What was pushed.
1487struct Pushed {
1488 commit: String,
1489 /// The branch was there already, with the same files.
1490 existed: bool,
1491}
1492
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1493/// The packages as a reason names them: `lodash 4.17.21, vitest 1.6.0`.
1494fn package_list(packages: &[Target]) -> String {
1495 packages.iter().map(|target| format!("{} {}", target.package, target.version)).collect::<Vec<_>>().join(", ")
1496}
1497
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1498fn bump() -> Result<(Bump, String, Pushed)> {
1499 let bump = Bump::from_env()?;
1500 let remote = env("GIT_REMOTE")?;
1501 let base = env("GIT_BRANCH_BASE")?;
1502 let branch = env("GIT_BRANCH")?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1503 check_branch(bump.kind, &branch, &base)?;
1504 let message = env("COMMIT_MESSAGE").unwrap_or_else(|_| match bump.packages.as_slice() {
1505 [only] => format!("Update {} to {}", only.package, only.version),
1506 [first, rest @ ..] => format!("Update {} and {} more", first.package, rest.len()),
1507 [] => "Update dependencies".to_owned(),
1508 });
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1509 let auth = auth_option(&env("G1T_USER")?, &env("G1T_TOKEN")?);
1510 let workdir = Path::new(WORKDIR);
1511
1512 std::fs::create_dir_all("/work")?;
1513 crate::clone::clone(Path::new("/work"), &auth, &["--branch", &base], &remote, WORKDIR)
1514 .with_context(|| format!("could not clone {base}"))?;
1515 git(workdir, &["checkout", "--quiet", "-b", &branch])?;
1516 git(workdir, &["config", "user.name", AUTHOR_NAME])?;
1517 git(workdir, &["config", "user.email", AUTHOR_EMAIL])?;
1518
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1519 let tools = registry_tools(&bump.registries, &named_registries(workdir, &bump.jobs), Path::new(REGISTRY_DIR))?;
1520 tools.write()?;
1521
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1522 let mut behind = Vec::new();
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1523 let mut changed: Vec<String> = Vec::new();
1524 for target in &bump.packages {
1525 for job in &bump.jobs {
1526 if bump.still_below(workdir, job, target)?.is_empty() {
1527 continue; // Already at the version or later here.
1528 }
1529 let outcome = bump.update(workdir, job, target, &tools)?;
1530 changed.extend(outcome.files);
1531 let left = bump.still_below(workdir, job, target)?;
1532 if !left.is_empty() {
1533 let why = outcome.said.last().map(|said| format!("\n{said}")).unwrap_or_default();
1534 behind.push(format!(
1535 "{} still resolves {} {} (wanted {} or later){why}",
1536 job.paths.join(", "),
1537 target.package,
1538 left.join(", "),
1539 target.version
1540 ));
1541 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1542 }
1543 }
1544 if !behind.is_empty() {
1545 return Err(needs_changes(behind.join("\n\n")));
1546 }
1547
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1548 let mut touched: Vec<String> = bump.jobs.iter().flat_map(Job::touched).chain(changed).collect();
1549 touched.sort();
1550 touched.dedup();
1551 touched.retain(|path| workdir.join(path).exists());
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1552 let mut add = vec!["add", "--"];
1553 add.extend(touched.iter().map(String::as_str));
1554 git(workdir, &add)?;
1555 if git(workdir, &["diff", "--cached", "--name-only"])?.is_empty() {
1556 bail!(
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1557 "nothing to change: {} already resolves {} or later",
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1558 bump.jobs.iter().flat_map(|job| job.paths.iter().map(String::as_str)).collect::<Vec<_>>().join(", "),
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1559 package_list(&bump.packages)
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1560 );
1561 }
1562 git(workdir, &["commit", "--quiet", "--message", &message])?;
1563 let commit = git(workdir, &["rev-parse", "HEAD"])?;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1564 let refspec = format!("{}HEAD:refs/heads/{branch}", if bump.force { "+" } else { "" });
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1565 let pushed = git(workdir, &["-c", &auth, "push", "--quiet", "origin", &refspec]);
1566 if let Err(error) = pushed {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1567 if bump.force {
1568 return Err(error.context("could not push the update"));
1569 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1570 // Pushed before (a retried job): the same files there is success.
1571 let theirs = git(workdir, &["-c", &auth, "ls-remote", "origin", &format!("refs/heads/{branch}")]).unwrap_or_default();
1572 if theirs.is_empty() {
1573 return Err(error.context("could not push the update"));
1574 }
1575 crate::clone::fetch(workdir, &auth, "origin", &format!("refs/heads/{branch}")).context("could not read the branch already pushed")?;
1576 let same = git(workdir, &["rev-parse", "FETCH_HEAD^{tree}"])? == git(workdir, &["rev-parse", "HEAD^{tree}"])?;
1577 if !same {
1578 return Err(error.context(format!("{branch} already exists with other changes")));
1579 }
1580 let commit = git(workdir, &["rev-parse", "FETCH_HEAD"])?;
1581 return Ok((bump, branch, Pushed { commit, existed: true }));
1582 }
1583 Ok((bump, branch, Pushed { commit, existed: false }))
1584}
1585
1586pub fn main() -> i32 {
1587 match bump() {
1588 Ok((bump, branch, pushed)) => {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1589 let first = &bump.packages[0];
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1590 println!(
1591 "{}",
1592 json!({
1593 "bump": if pushed.existed { "exists" } else { "pushed" },
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1594 "kind": bump.kind.name(),
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1595 "branch": branch,
1596 "commit": pushed.commit,
1597 "ecosystem": bump.ecosystem.osv(),
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1598 "package": first.package,
1599 "version": first.version,
1600 "packages": bump.packages,
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1601 "lockfiles": bump.jobs.iter().flat_map(|job| job.paths.clone()).collect::<Vec<_>>(),
1602 })
1603 );
1604 0
1605 }
1606 Err(error) => {
1607 let (reason, code) = match error.downcast_ref::<Stop>() {
1608 Some(Stop::NeedsChanges(_)) => ("needs_code_changes", NEEDS_CHANGES_EXIT),
1609 Some(Stop::Unsupported(_)) => ("unsupported", UNSUPPORTED_EXIT),
1610 None => ("failed", 1),
1611 };
1612 println!("{}", json!({ "bump": "failed", "reason": reason, "message": format!("{error:#}") }));
1613 eprintln!("g1t-runner: {error:#}");
1614 code
1615 }
1616 }
1617}
1618
1619#[cfg(test)]
1620mod tests {
1621 use super::*;
1622
1623 fn strings(items: &[&str]) -> Vec<String> {
1624 items.iter().map(|item| item.to_string()).collect()
1625 }
1626
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1627 fn target(package: &str, version: &str) -> Target {
1628 Target { package: package.into(), version: version.into() }
1629 }
1630
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1631 #[test]
1632 fn lockfiles_come_as_lines_or_json() {
1633 assert_eq!(lockfile_list("Cargo.lock\n web/package-lock.json \n\n").unwrap(), ["Cargo.lock", "web/package-lock.json"]);
1634 assert_eq!(lockfile_list(r#"["./go.mod","go.sum"]"#).unwrap(), ["go.mod", "go.sum"]);
1635 assert!(lockfile_list("[not json").is_err());
1636 }
1637
1638 #[test]
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1639 fn packages_come_as_a_list_or_one() {
1640 // A security update's one package, as before.
1641 assert_eq!(targets(Ecosystem::Npm, None, Some(" lodash "), Some("v4.17.21")).unwrap(), [target("lodash", "4.17.21")]);
1642 assert_eq!(targets(Ecosystem::Npm, Some("[]"), Some("lodash"), Some("4.17.21")).unwrap(), [target("lodash", "4.17.21")]);
1643 assert!(targets(Ecosystem::Npm, None, None, Some("1.0.0")).is_err());
1644 // A group, each version as the tool takes it, a package once.
1645 let listed = r#"[{"package":"golang.org/x/net","version":"0.23.0"},{"package":"golang.org/x/text","version":"v0.14.0"},{"package":"golang.org/x/net","version":"0.24.0"}]"#;
1646 assert_eq!(
1647 targets(Ecosystem::Go, Some(listed), Some("ignored"), Some("1")).unwrap(),
1648 [target("golang.org/x/net", "v0.23.0"), target("golang.org/x/text", "v0.14.0")]
1649 );
1650 assert!(targets(Ecosystem::Npm, Some(r#"[{"package":"--registry=evil","version":"1"}]"#), None, None).is_err());
1651 assert!(targets(Ecosystem::Npm, Some(r#"[{"package":"a","version":"1;rm"}]"#), None, None).is_err());
1652 assert!(targets(Ecosystem::Npm, Some("{"), None, None).is_err());
1653 let many = format!("[{}]", (0..51).map(|i| format!(r#"{{"package":"p{i}","version":"1.0.0"}}"#)).collect::<Vec<_>>().join(","));
1654 assert!(targets(Ecosystem::Npm, Some(&many), None, None).is_err());
1655 }
1656
1657 #[test]
1658 fn kinds_and_strategies_by_name() {
1659 assert_eq!(Kind::parse("").unwrap(), Kind::Security);
1660 assert_eq!(Kind::parse("security").unwrap(), Kind::Security);
1661 assert_eq!(Kind::parse("version").unwrap(), Kind::Version);
1662 assert!(Kind::parse("major").is_err());
1663 assert_eq!(Strategy::parse("").unwrap(), Strategy::Increase);
1664 assert_eq!(Strategy::parse("increase-if-necessary").unwrap(), Strategy::IncreaseIfNecessary);
1665 assert_eq!(Strategy::parse("widen").unwrap(), Strategy::Widen);
1666 assert_eq!(Strategy::parse("lockfile-only").unwrap(), Strategy::LockfileOnly);
1667 assert!(Strategy::parse("auto").is_err());
1668 }
1669
1670 #[test]
1671 fn branches_by_kind() {
1672 assert!(check_branch(Kind::Security, "g1t/security/lodash-4.17.21", "main").is_ok());
1673 for branch in ["main", "deps/lodash", "g1t/security/a..b", "g1t/security/a b"] {
1674 assert!(check_branch(Kind::Security, branch, "main").is_err(), "{branch}");
1675 }
1676 for branch in ["deps/npm/lodash", "dependabot/cargo/serde-1.0.200", "g1t/security/x", "develop"] {
1677 assert!(check_branch(Kind::Version, branch, "main").is_ok(), "{branch}");
1678 }
1679 assert!(check_branch(Kind::Version, "main", "main").is_err());
1680 let long = "x".repeat(201);
1681 for branch in ["", " ", "a b", "a..b", "a~1", "a^", "a:b", "a?", "a*", "a[b", "a\\b", "a@{1}", "-x", "/x", "refs/heads/x", "x/", "x.lock", "a\u{7}", long.as_str()] {
1682 assert!(check_branch(Kind::Version, branch, "main").is_err(), "{branch:?}");
1683 }
1684 }
1685
1686 #[test]
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1687 fn lockfiles_group_by_directory_and_tool() {
1688 let found = jobs(Ecosystem::Go, &strings(&["go.mod", "go.sum", "tools/go.sum"])).unwrap();
1689 assert_eq!(
1690 found,
1691 [
1692 Job { dir: String::new(), lockfile: Lockfile::GoMod, paths: strings(&["go.mod", "go.sum"]) },
1693 Job { dir: "tools".into(), lockfile: Lockfile::GoMod, paths: strings(&["tools/go.sum"]) },
1694 ]
1695 );
1696 assert_eq!(found[0].touched(), ["go.mod", "go.sum"]);
1697 let web = jobs(Ecosystem::Npm, &strings(&["web/package-lock.json"])).unwrap();
1698 assert_eq!(web[0].touched(), ["web/package-lock.json", "web/package.json"]);
1699 }
1700
1701 #[test]
1702 fn lockfiles_must_be_the_ecosystems_and_inside_the_repository() {
1703 assert!(jobs(Ecosystem::Npm, &strings(&["Cargo.lock"])).is_err());
1704 assert!(jobs(Ecosystem::Npm, &strings(&["../package-lock.json"])).is_err());
1705 assert!(jobs(Ecosystem::Npm, &strings(&["/etc/package-lock.json"])).is_err());
1706 assert!(jobs(Ecosystem::Npm, &[]).is_err());
1707 let error = jobs(Ecosystem::PyPI, &strings(&["uv.lock"])).unwrap_err();
1708 assert!(matches!(error.downcast_ref::<Stop>(), Some(Stop::Unsupported(_))));
1709 }
1710
1711 #[test]
1712 fn versions_as_each_tool_takes_them() {
1713 assert_eq!(tool_version(Ecosystem::Go, "0.17.0"), "v0.17.0");
1714 assert_eq!(tool_version(Ecosystem::Go, "v0.17.0"), "v0.17.0");
1715 assert_eq!(tool_version(Ecosystem::Npm, "v4.17.21"), "4.17.21");
1716 assert_eq!(tool_version(Ecosystem::Cargo, " 1.6.1 "), "1.6.1");
1717 assert_eq!(tool_version(Ecosystem::PyPI, "2.31.0"), "2.31.0");
1718 }
1719
1720 #[test]
1721 fn names_and_versions_cannot_be_options() {
1722 assert!(safe_argument("package", "@babel/core").is_ok());
1723 assert!(safe_argument("package", "golang.org/x/net").is_ok());
1724 assert!(safe_argument("version", "1.2.3-rc.1+build").is_ok());
1725 assert!(safe_argument("package", "--registry=evil").is_err());
1726 assert!(safe_argument("package", "a b").is_err());
1727 assert!(safe_argument("version", "1.0;rm").is_err());
1728 assert!(safe_argument("version", "").is_err());
1729 }
1730
1731 #[test]
1732 fn a_direct_dependency_keeps_its_range_style() {
1733 assert_eq!(raised_range("^4.17.0", "4.17.21"), "^4.17.21");
1734 assert_eq!(raised_range("~1.2.0", "1.2.5"), "~1.2.5");
1735 assert_eq!(raised_range("1.2.0", "1.2.5"), "1.2.5");
1736 assert_eq!(raised_range("=1.2.0", "1.2.5"), "1.2.5");
1737 assert_eq!(raised_range(">=1 <2", "1.2.5"), "^1.2.5");
1738 assert_eq!(raised_range("*", "1.2.5"), "^1.2.5");
1739 }
1740
1741 #[test]
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1742 fn widen_keeps_what_was_allowed() {
1743 assert_eq!(widened_range("^1.2.0", "2.0.0"), "^1.2.0 || ^2.0.0");
1744 assert_eq!(widened_range(" ~1.2.0 ", "1.3.0"), "~1.2.0 || ~1.3.0");
1745 assert_eq!(widened_range("1.2.0", "2.0.0"), "1.2.0 || 2.0.0");
1746 assert_eq!(widened_range("^1.0.0 || ^2.0.0", "3.1.0"), "^1.0.0 || ^2.0.0 || ^3.1.0");
1747 }
1748
1749 #[test]
1750 fn strategies_plan_a_direct_javascript_dependency() {
1751 let raised = |in_range_first, range: &str| DirectPlan { in_range_first, range: Some(range.to_owned()) };
1752 // A security update, and increase: the range raised, at once.
1753 assert_eq!(direct_plan(None, "^1.2.0", "2.0.0"), raised(false, "^2.0.0"));
1754 assert_eq!(direct_plan(Some(Strategy::Increase), "~1.2.0", "1.3.0"), raised(false, "~1.3.0"));
1755 // The others first take what the range allows.
1756 assert_eq!(direct_plan(Some(Strategy::IncreaseIfNecessary), "^1.2.0", "2.0.0"), raised(true, "^2.0.0"));
1757 assert_eq!(direct_plan(Some(Strategy::Widen), "^1.2.0", "2.0.0"), raised(true, "^1.2.0 || ^2.0.0"));
1758 assert_eq!(direct_plan(Some(Strategy::LockfileOnly), "^1.2.0", "2.0.0"), DirectPlan { in_range_first: true, range: None });
1759 }
1760
1761 #[test]
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1762 fn direct_dependencies_from_the_registry_only() {
1763 let manifest = json!({
1764 "dependencies": { "lodash": "^4.17.0", "local": "file:../local", "shared": "workspace:*" },
1765 "devDependencies": { "vitest": "~1.0.0", "fork": "github:me/fork" },
1766 });
1767 assert_eq!(direct_range(&manifest, "lodash").as_deref(), Some("^4.17.0"));
1768 assert_eq!(direct_range(&manifest, "vitest").as_deref(), Some("~1.0.0"));
1769 assert_eq!(direct_range(&manifest, "local"), None);
1770 assert_eq!(direct_range(&manifest, "shared"), None);
1771 assert_eq!(direct_range(&manifest, "fork"), None);
1772 assert_eq!(direct_range(&manifest, "minimist"), None);
1773 }
1774
1775 #[test]
1776 fn javascript_commands_by_lockfile() {
1777 let npm = Node::of(Lockfile::PackageLock, "{}").unwrap();
1778 assert_eq!(
1779 npm.direct("lodash", "^4.17.21"),
1780 strings(&["npm", "install", "--package-lock-only", "--ignore-scripts", "--no-audit", "--no-fund", "lodash@^4.17.21"])
1781 );
1782 assert_eq!(
1783 npm.transitive("minimist").unwrap(),
1784 strings(&["npm", "update", "--package-lock-only", "--ignore-scripts", "--no-audit", "--no-fund", "minimist"])
1785 );
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1786 assert_eq!(
1787 npm.in_range("lodash"),
1788 strings(&["npm", "update", "--package-lock-only", "--no-save", "--ignore-scripts", "--no-audit", "--no-fund", "lodash"])
1789 );
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1790 assert_eq!(npm.override_path(), ["overrides"]);
1791
1792 let pnpm = Node::of(Lockfile::PnpmLock, "lockfileVersion: '9.0'").unwrap();
1793 assert_eq!(pnpm.direct("lodash", "^4.17.21"), strings(&["corepack", "pnpm", "update", "lodash@^4.17.21", "--lockfile-only", "--ignore-scripts"]));
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1794 assert_eq!(pnpm.in_range("lodash"), strings(&["corepack", "pnpm", "update", "lodash", "--lockfile-only", "--no-save", "--ignore-scripts"]));
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1795 assert_eq!(pnpm.override_path(), ["pnpm", "overrides"]);
1796
1797 let berry = Node::of(Lockfile::YarnLock, "__metadata:\n version: 6\n").unwrap();
1798 assert_eq!(berry, Node::YarnBerry);
1799 assert_eq!(berry.direct("lodash", "^4.17.21"), strings(&["corepack", "yarn", "up", "lodash@^4.17.21", "--mode=update-lockfile"]));
1800 assert_eq!(berry.transitive("minimist").unwrap(), strings(&["corepack", "yarn", "up", "--recursive", "minimist", "--mode=update-lockfile"]));
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1801 assert_eq!(berry.in_range("lodash"), strings(&["corepack", "yarn", "up", "--recursive", "lodash", "--mode=update-lockfile"]));
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1802
1803 let classic = Node::of(Lockfile::YarnLock, "# yarn lockfile v1\n").unwrap();
1804 assert_eq!(classic, Node::YarnClassic);
1805 assert_eq!(classic.transitive("minimist"), None);
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1806 assert_eq!(classic.in_range("lodash"), strings(&["corepack", "yarn", "upgrade", "lodash", "--ignore-scripts", "--non-interactive"]));
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1807 assert_eq!(classic.override_path(), ["resolutions"]);
1808 assert_eq!(Node::of(Lockfile::CargoLock, ""), None);
1809 }
1810
1811 #[test]
1812 fn overrides_are_added_once() {
1813 let mut manifest = json!({ "name": "app" });
1814 assert!(add_override(&mut manifest, &["pnpm", "overrides"], "minimist", "1.2.6").unwrap());
1815 assert_eq!(manifest["pnpm"]["overrides"]["minimist"], "1.2.6");
1816 assert!(!add_override(&mut manifest, &["pnpm", "overrides"], "minimist", "1.2.6").unwrap());
1817 assert!(add_override(&mut manifest, &["pnpm", "overrides"], "minimist", "1.2.8").unwrap());
1818 assert_eq!(manifest["pnpm"]["overrides"]["minimist"], "1.2.8");
1819 }
1820
1821 #[test]
1822 fn cargo_and_go_commands() {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1823 let [precise, compatible] = cargo_commands("time", "0.1.43", "0.1.45", None);
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1824 assert_eq!(precise, strings(&["cargo", "update", "-p", "time@0.1.43", "--precise", "0.1.45"]));
1825 assert_eq!(compatible, strings(&["cargo", "update", "-p", "time@0.1.43"]));
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1826 let [precise, _] = cargo_commands("time", "0.1.43", "0.1.45", Some("/tmp/g1t-registries/cargo.toml"));
1827 assert_eq!(precise, strings(&["cargo", "--config", "/tmp/g1t-registries/cargo.toml", "update", "-p", "time@0.1.43", "--precise", "0.1.45"]));
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1828 let [get, tidy] = go_commands("golang.org/x/net", "v0.23.0");
1829 assert_eq!(get, strings(&["go", "get", "golang.org/x/net@v0.23.0"]));
1830 assert_eq!(tidy, strings(&["go", "mod", "tidy"]));
1831 }
1832
1833 #[test]
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1834 fn cargo_requirements_keep_their_operator() {
1835 assert_eq!(raised_requirement("1.2", "2.0.3").as_deref(), Some("2.0.3"));
1836 assert_eq!(raised_requirement("^1.2.0", "2.0.3").as_deref(), Some("^2.0.3"));
1837 assert_eq!(raised_requirement("~0.4", "0.5.1").as_deref(), Some("~0.5.1"));
1838 assert_eq!(raised_requirement("=1.0.0", "1.1.0").as_deref(), Some("=1.1.0"));
1839 assert_eq!(raised_requirement(">= 1.0", "2.0.0").as_deref(), Some(">=2.0.0"));
1840 // Not a single requirement below the version: left alone.
1841 assert_eq!(raised_requirement(">=1, <2", "2.0.0"), None);
1842 assert_eq!(raised_requirement("1.*", "2.0.0"), None);
1843 assert_eq!(raised_requirement("*", "2.0.0"), None);
1844 assert_eq!(raised_requirement("2.1", "2.0.0"), None);
1845 }
1846
1847 #[test]
1848 fn cargo_toml_requirements_are_raised_in_every_form() {
1849 let text = r#"[package]
1850name = "app"
1851version = "0.1.0"
1852
1853[dependencies]
1854serde = "1.0" # data
1855tokio = { version = "~1.20", features = ["full"] }
1856"serde_json" = { version = "=1.0.100" }
1857other = { version = "0.1", package = "serde" }
1858shared = { workspace = true }
1859time = "0.1"
1860
1861[dev-dependencies.serde]
1862features = ["derive"]
1863version = "^1.0.150"
1864
1865[target.'cfg(unix)'.build-dependencies]
1866serde = { path = "../serde", version = ">=1.0" }
1867
1868[workspace.dependencies]
1869serde = '1'
1870
1871[[bin]]
1872name = "serde"
1873version = "0.1"
1874
1875[features]
1876serde = []
1877"#;
1878 let (out, moved) = rewrite_cargo_requirements(text, "serde", "1.0.200");
1879 assert_eq!(moved, 5);
1880 assert!(out.contains("serde = \"1.0.200\" # data\n"));
1881 assert!(out.contains("other = { version = \"1.0.200\", package = \"serde\" }\n"));
1882 assert!(out.contains("[dev-dependencies.serde]\nfeatures = [\"derive\"]\nversion = \"^1.0.200\"\n"));
1883 assert!(out.contains("serde = { path = \"../serde\", version = \">=1.0.200\" }\n"));
1884 assert!(out.contains("serde = '1.0.200'\n"));
1885 // The package's own version, a binary and a feature are not dependencies.
1886 assert!(out.contains("[package]\nname = \"app\"\nversion = \"0.1.0\"\n"));
1887 assert!(out.contains("[[bin]]\nname = \"serde\"\nversion = \"0.1\"\n"));
1888 assert!(out.contains("time = \"0.1\"\n"));
1889
1890 let (out, moved) = rewrite_cargo_requirements(text, "tokio", "1.37.0");
1891 assert_eq!(moved, 1);
1892 assert!(out.contains("tokio = { version = \"~1.37.0\", features = [\"full\"] }\n"));
1893 let (out, moved) = rewrite_cargo_requirements(text, "serde_json", "1.0.117");
1894 assert_eq!(moved, 1);
1895 assert!(out.contains("\"serde_json\" = { version = \"=1.0.117\" }\n"));
1896 // Already allowed, or not named: nothing changes.
1897 let (same, moved) = rewrite_cargo_requirements(text, "serde", "0.0.5");
1898 assert_eq!((same.as_str(), moved), (text, 0));
1899 assert_eq!(rewrite_cargo_requirements(text, "rand", "0.9.0").1, 0);
1900 // A renamed table section names its package.
1901 let renamed = "[dependencies.json]\npackage = \"serde_json\"\nversion = \"1.0.0\"\n";
1902 assert_eq!(rewrite_cargo_requirements(renamed, "serde_json", "1.0.117").0, "[dependencies.json]\npackage = \"serde_json\"\nversion = \"1.0.117\"\n");
1903 assert_eq!(rewrite_cargo_requirements(renamed, "json", "1.0.117").1, 0);
1904 }
1905
1906 #[test]
1907 fn workspace_members_by_path_or_trailing_star() {
1908 let root: toml::Value = toml::from_str("[workspace]\nmembers = [\"crates/*\", \"./tools/xtask/\", \"apps/**\", \"../outside\", \"a?b\"]\n").unwrap();
1909 assert_eq!(workspace_members(&root), ["crates/*", "tools/xtask"]);
1910 assert!(workspace_members(&toml::from_str::<toml::Value>("[package]\nname = \"x\"\n").unwrap()).is_empty());
1911 assert_eq!(header_keys("[target.'cfg(target_os = \"linux\")'.dependencies]"), Some(strings(&["target", "cfg(target_os = \"linux\")", "dependencies"])));
1912 assert_eq!(header_keys("[[bin]]"), None);
1913 assert_eq!(dependency_table(&strings(&["workspace", "dependencies"])), Some(None));
1914 assert_eq!(dependency_table(&strings(&["dependencies", "serde"])), Some(Some("serde".into())));
1915 assert_eq!(dependency_table(&strings(&["package"])), None);
1916 }
1917
1918 #[test]
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily1919 fn poetry_adds_a_direct_dependency_and_updates_any_other() {
1920 let pyproject: toml::Value = toml::from_str(
1921 "[tool.poetry.dependencies]\npython = \"^3.11\"\nRequests = \"^2.0\"\n\n[tool.poetry.group.test.dependencies]\npytest = \"^7\"\n",
1922 )
1923 .unwrap();
1924 assert_eq!(poetry_group(&pyproject, "requests"), Some(None));
1925 assert_eq!(poetry_group(&pyproject, "pytest"), Some(Some("test".to_owned())));
1926 assert_eq!(poetry_group(&pyproject, "urllib3"), None);
1927 let pep621: toml::Value = toml::from_str("[project]\ndependencies = [\"jinja2>=3.0\", \"Flask_Cors\"]\n").unwrap();
1928 assert_eq!(poetry_group(&pep621, "Jinja2"), Some(None));
1929 assert_eq!(poetry_group(&pep621, "flask-cors"), Some(None));
1930
1931 let poetry = strings(&["poetry"]);
1932 assert_eq!(poetry_commands(&poetry, "requests", "2.31.0", Some(None)), strings(&["poetry", "add", "requests@^2.31.0", "--lock"]));
1933 assert_eq!(
1934 poetry_commands(&poetry, "pytest", "7.4.0", Some(Some("test".into()))),
1935 strings(&["poetry", "add", "pytest@^7.4.0", "--lock", "--group", "test"])
1936 );
1937 assert_eq!(poetry_commands(&poetry, "urllib3", "2.0.7", None), strings(&["poetry", "update", "--lock", "urllib3"]));
1938 }
1939
1940 #[test]
1941 fn requirements_pins_are_rewritten_in_place() {
1942 let text = "# pinned\nrequests==2.25.0 # http\nDjango[argon2]===3.2.0 ; python_version >= \"3.8\"\nurllib3==1.26.18\nrequests_toolbelt==0.9.1\n-r base.txt\nflask>=2.0\n";
1943 let (out, moved) = rewrite_pins(text, "requests", "2.31.0");
1944 assert_eq!(moved, 1);
1945 assert!(out.contains("requests==2.31.0 # http\n"));
1946 assert!(out.contains("requests_toolbelt==0.9.1\n"));
1947 let (out, moved) = rewrite_pins(&out, "django", "3.2.25");
1948 assert_eq!(moved, 1);
1949 assert!(out.contains("Django[argon2]===3.2.25 ; python_version >= \"3.8\"\n"));
1950 // Already at or past the version: left alone.
1951 let (same, moved) = rewrite_pins(text, "urllib3", "1.26.18");
1952 assert_eq!((same.as_str(), moved), (text, 0));
1953 // A line without a final newline keeps it that way.
1954 assert_eq!(rewrite_pins("Requests==2.0", "requests", "2.31.0").0, "Requests==2.31.0");
1955 assert_eq!(rewrite_pins("requests == 2.0,<3\n", "requests", "2.31.0").0, "requests == 2.31.0,<3\n");
1956 }
1957
1958 #[test]
1959 fn lockfiles_are_read_again_for_what_is_still_below() {
1960 let lock = r#"{"lockfileVersion":3,"packages":{"":{},"node_modules/lodash":{"version":"4.17.21"},"node_modules/a/node_modules/lodash":{"version":"4.17.4"}}}"#;
1961 assert_eq!(below(Lockfile::PackageLock, lock, Ecosystem::Npm, "lodash", "4.17.21"), ["4.17.4"]);
1962 let sum = "golang.org/x/net v0.17.0 h1:x=\ngolang.org/x/net v0.23.0 h1:y=\n";
1963 assert!(below(Lockfile::GoSum, sum, Ecosystem::Go, "golang.org/x/net", "v0.23.0").is_empty());
1964 assert_eq!(below(Lockfile::Requirements, "Requests==2.0\n", Ecosystem::PyPI, "requests", "2.31.0"), ["2.0"]);
1965 }
1966
1967 #[test]
1968 fn a_failure_says_its_last_lines() {
1969 assert_eq!(tail("a\n\nb\nc\n", 2), "b\nc");
1970 assert_eq!(tail("only", 5), "only");
1971 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1972
1973 fn registry(kind: &str, url: &str) -> Registry {
1974 Registry { kind: kind.into(), url: url.into(), ..Registry::default() }
1975 }
1976
1977 fn env_of(tools: &Tools) -> Vec<(&str, &str)> {
1978 tools.env.iter().map(|(name, value)| (name.as_str(), value.as_str())).collect()
1979 }
1980
1981 #[test]
1982 fn registries_arrive_as_the_contract_writes_them() {
1983 let text = r#"[{"type":"npm-registry","url":"https://npm.acme.dev","token":"t0k","replacesBase":true,"scopes":["@acme"]}]"#;
1984 let registries: Vec<Registry> = serde_json::from_str(text).unwrap();
1985 assert_eq!(registries[0].kind, "npm-registry");
1986 assert!(registries[0].replaces_base);
1987 assert_eq!(registries[0].scopes, ["@acme"]);
1988 // Its debug form never holds the token.
1989 assert!(!format!("{:?}", registries[0]).contains("t0k"));
1990 }
1991
1992 #[test]
1993 fn npm_registries_become_a_user_npmrc() {
1994 let dir = Path::new("/tmp/g1t-registries");
1995 let mut replacing = registry("npm-registry", "https://npm.acme.dev/");
1996 replacing.token = Some("t0k".into());
1997 replacing.replaces_base = true;
1998 let mut scoped = registry("npm-registry", "https://pkgs.acme.dev/npm/");
1999 scoped.username = Some("ada".into());
2000 scoped.password = Some("p:w".into());
2001 scoped.scopes = strings(&["@acme", "@tools"]);
2002 let tools = registry_tools(&[replacing, scoped], &Named::default(), dir).unwrap();
2003 assert_eq!(tools.files.len(), 1);
2004 assert_eq!(tools.files[0].0, dir.join("npmrc"));
2005 assert_eq!(
2006 tools.files[0].1,
2007 "//npm.acme.dev/:_authToken=t0k\nregistry=https://npm.acme.dev/\n//pkgs.acme.dev/npm/:_auth=YWRhOnA6dw==\n@acme:registry=https://pkgs.acme.dev/npm/\n@tools:registry=https://pkgs.acme.dev/npm/\n"
2008 );
2009 assert_eq!(
2010 env_of(&tools),
2011 [
2012 ("YARN_NPM_REGISTRY_SERVER", "https://npm.acme.dev/"),
2013 ("YARN_NPM_AUTH_TOKEN", "t0k"),
2014 ("YARN_NPM_ALWAYS_AUTH", "true"),
2015 ("NPM_CONFIG_USERCONFIG", dir.join("npmrc").display().to_string().as_str()),
2016 ]
2017 );
2018 assert!(registry_tools(&[], &Named::default(), dir).unwrap().env.is_empty());
2019 }
2020
2021 #[test]
2022 fn cargo_registries_by_the_projects_names_or_as_crates_io() {
2023 let dir = Path::new("/tmp/g1t-registries");
2024 let config = "[registries.acme]\nindex = \"sparse+https://cargo.acme.dev/index/\"\n[registries.other]\nindex = \"https://elsewhere.dev/git\"\n";
2025 let named = Named { cargo: cargo_registries(config), poetry: Vec::new() };
2026 assert_eq!(named.cargo.len(), 2);
2027 let mut acme = registry("cargo-registry", "https://cargo.acme.dev/index");
2028 acme.token = Some("ct".into());
2029 let tools = registry_tools(std::slice::from_ref(&acme), &named, dir).unwrap();
2030 assert_eq!(env_of(&tools), [("CARGO_REGISTRIES_ACME_TOKEN", "ct")]);
2031 assert!(tools.cargo_config.is_none());
2032
2033 acme.replaces_base = true;
2034 let tools = registry_tools(&[acme], &Named::default(), dir).unwrap();
2035 assert_eq!(env_of(&tools), [("CARGO_REGISTRIES_G1T_PRIVATE_TOKEN", "ct")]);
2036 assert_eq!(tools.cargo_config.as_deref(), Some(dir.join("cargo.toml").display().to_string().as_str()));
2037 let written: toml::Value = toml::from_str(&tools.files[0].1).unwrap();
2038 assert_eq!(written["source"]["crates-io"]["replace-with"].as_str(), Some("g1t-private"));
2039 assert_eq!(written["source"]["g1t-private"]["registry"].as_str(), Some("sparse+https://cargo.acme.dev/index/"));
2040 assert_eq!(written["registries"]["g1t-private"]["index"].as_str(), Some("sparse+https://cargo.acme.dev/index/"));
2041 }
2042
2043 #[test]
2044 fn python_indexes_carry_their_credentials_in_the_url() {
2045 let dir = Path::new("/tmp/g1t-registries");
2046 let mut base = registry("python-index", "https://pypi.acme.dev/simple/");
2047 base.username = Some("ada@acme".into());
2048 base.password = Some("p w/1".into());
2049 base.replaces_base = true;
2050 let mut extra = registry("python-index", "https://extra.acme.dev/simple");
2051 extra.token = Some("tok".into());
2052 let named = Named { cargo: Vec::new(), poetry: poetry_sources("[[tool.poetry.source]]\nname = \"acme-extra\"\nurl = \"https://extra.acme.dev/simple/\"\n") };
2053 let tools = registry_tools(&[base, extra, registry("python-index", "https://open.acme.dev/simple")], &named, dir).unwrap();
2054 assert_eq!(
2055 env_of(&tools),
2056 [
2057 ("PIP_INDEX_URL", "https://ada%40acme:p%20w%2F1@pypi.acme.dev/simple/"),
2058 ("POETRY_HTTP_BASIC_ACME_EXTRA_USERNAME", "__token__"),
2059 ("POETRY_HTTP_BASIC_ACME_EXTRA_PASSWORD", "tok"),
2060 ("PIP_EXTRA_INDEX_URL", "https://__token__:tok@extra.acme.dev/simple https://open.acme.dev/simple"),
2061 ]
2062 );
2063 assert!(tools.files.is_empty());
2064 }
2065
2066 #[test]
2067 fn go_proxies_come_first_with_a_netrc() {
2068 let dir = Path::new("/tmp/g1t-registries");
2069 let mut proxy = registry("goproxy-server", "https://goproxy.acme.dev/mod");
2070 proxy.username = Some("ada".into());
2071 proxy.password = Some("pw".into());
2072 let tools = registry_tools(std::slice::from_ref(&proxy), &Named::default(), dir).unwrap();
2073 assert_eq!(
2074 env_of(&tools),
2075 [("GOPROXY", "https://goproxy.acme.dev/mod,https://proxy.golang.org,direct"), ("NETRC", dir.join("netrc").display().to_string().as_str())]
2076 );
2077 assert_eq!(tools.files, [(dir.join("netrc"), "machine goproxy.acme.dev\nlogin ada\npassword pw\n".to_owned())]);
2078 proxy.replaces_base = true;
2079 let tools = registry_tools(&[proxy], &Named::default(), dir).unwrap();
2080 assert_eq!(tools.get("GOPROXY"), Some("https://goproxy.acme.dev/mod,direct"));
2081 }
2082
2083 #[test]
2084 fn registries_are_checked_before_anything_is_written() {
2085 let dir = Path::new("/tmp/g1t-registries");
2086 let check = |registry: Registry| registry_tools(&[registry], &Named::default(), dir).map(|_| ());
2087 assert!(check(registry("maven-repository", "https://m.acme.dev")).is_err());
2088 assert!(check(registry("npm-registry", "http://npm.acme.dev")).is_err());
2089 assert!(check(registry("npm-registry", "https://")).is_err());
2090 assert!(check(registry("goproxy-server", "https://a.dev,https://evil.dev")).is_err());
2091 let mut newline = registry("npm-registry", "https://npm.acme.dev");
2092 newline.token = Some("t\nregistry=https://evil.dev".into());
2093 assert!(check(newline).is_err());
2094 let mut spaced = registry("goproxy-server", "https://goproxy.acme.dev");
2095 spaced.password = Some("a b".into());
2096 assert!(check(spaced).is_err());
2097 let mut scope = registry("npm-registry", "https://npm.acme.dev");
2098 scope.scopes = strings(&["acme"]);
2099 assert!(check(scope).is_err());
2100 }
2101
2102 #[test]
2103 fn registry_urls_compare_without_protocol_or_slash() {
2104 assert!(same_registry("sparse+https://cargo.acme.dev/index/", "https://Cargo.acme.dev/index"));
2105 assert!(same_registry("https://git.acme.dev/index.git", "https://git.acme.dev/index"));
2106 assert!(!same_registry("https://a.dev/x", "https://a.dev/y"));
2107 assert_eq!(npm_nerf_dart("https://npm.acme.dev"), "//npm.acme.dev/");
2108 assert_eq!(url_host("https://goproxy.acme.dev:8443/mod"), "goproxy.acme.dev");
2109 assert_eq!(env_part("acme-extra.v2"), "ACME_EXTRA_V2");
2110 assert_eq!(percent_encode("a@b:c/d e"), "a%40b%3Ac%2Fd%20e");
2111 }
Git storage hardened, pages in tens of milliseconds, honest security alerts, and costs reconciled daily2112}