Skip to content
1,663 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1//! Version updates: pull requests that keep dependencies current, as the
2//! dependency update file (`dependabot.yml`, version 2) asks.
3//!
4//! 1. Reading the file (`deps::read_version_updates`, on every dependency
5//! scan) schedules each entry g1t can act on: a new entry runs at once,
6//! then on its `schedule`. A sweep every few minutes runs those due.
7//! 2. A run reads the entry's manifests and lockfiles on the default
8//! branch, asks each dependency's registry for its versions, and picks
9//! each one's target with `allow`, `ignore`, people's `@g1t ignore`
10//! comments, `cooldown` and `versioning-strategy` (`planning`). Updates
11//! are gathered by `groups`, and up to `open-pull-requests-limit` pull
12//! requests are asked of the runner's `bump` sandbox, the same one
13//! security updates use.
14//! 3. The sandbox's push opens the pull request as g1t (`User::system`),
15//! titled and described as `pull_text` says, assigned and with reviews
16//! asked as the file says. An older one for the same dependency or group
17//! is closed as superseded.
18//! 4. The sweep watches open ones: a failing required check closes it and
19//! puts g1t on an issue to make the code changes, as security updates
20//! do; one asked to merge (`@g1t merge`) merges once its checks pass;
21//! one in conflict is made again from its base (`rebase-strategy`).
22//! 5. `@g1t` comments on these pull requests are commands
23//! (`g1t_contracts::updates::update_command`).
24
25use std::collections::{BTreeMap, BTreeSet};
26
27use g1t_contracts::access::{Capability, CollaboratorPermissionArgs, PermissionInfo};
28use g1t_contracts::actions::{ResolveSettingsArgs, ResolvedSettings};
29use g1t_contracts::events::Event;
30use g1t_contracts::repos::{BlobArgs, BlobView, BlobText, FileList, ListFilesArgs, RawFile, RawFileArgs, ReadBlobsArgs, RepoPath};
31use g1t_contracts::security::{BumpArgs, UpdateState};
32use g1t_contracts::time::{parse_rfc3339, rfc3339};
33use g1t_contracts::updates::{
34 BumpPackage, BumpRegistry, CheckUpdatesArgs, DEPENDABOT_CHECK, DEPENDABOT_PATHS, IgnoreCondition, UpdateCommand, UpdatedDependency,
35 VersionUpdatesState, update_command,
36};
37use g1t_contracts::work::{
Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in.38 Mergeable, OpenPullArgs, Pull, PullActionArgs, PullDetail, PullStatus, RequiredState, Runtime, SetCommitStatusArgs,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar39 UpdatePullArgs, ViewArgs,
40};
41use g1t_contracts::{FailureCode, Outcome, User};
42use g1t_kit::now_ms;
43use g1t_scan::lockfiles::Lockfile;
44use g1t_scan::version;
45use serde::Deserialize;
Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in.46use serde_json::Value;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar47use worker::{Fetch, Headers, Method, Request, RequestInit, Result};
48
49use crate::Security;
50use crate::config::{self, Config, Entry, Registry, glob};
51use crate::manifests::{self, Declared, DependencyType};
52use crate::planning::{self, Candidate, Planned, PullPlan, Skip};
53use crate::pull_text;
54use crate::ranges::{self, Bare};
55use crate::registries::{self, Package, Source};
56use crate::store::RepoRow;
57use crate::update_store::{NewPull, PullRow};
58use crate::updates;
59
60/// Entries run per sweep.
61const DUE_PER_SWEEP: u32 = 3;
62/// Open update pull requests looked at per sweep.
63const WATCHED_PER_SWEEP: u32 = 10;
64/// Dependencies one run asks registries about, at most.
65const MAX_PACKAGES: usize = 200;
66/// Registry requests in flight at once.
67const FETCH_AT_ONCE: usize = 8;
68/// Versions whose publish time a Go module's run asks the proxy for.
69const GO_TIMES: usize = 3;
70/// A sandbox that has not pushed after this long is taken to have failed.
71const STALLED_MS: u64 = 45 * 60 * 1000;
72const MAX_BLOB_BYTES: u32 = 5_000_000;
73const SKIPPED_DIRECTORIES: [&str; 6] = ["node_modules", ".git", "target", "dist", "build", ".venv"];
74
75/// OSV's name for a `package-ecosystem`, as lockfiles and the bump sandbox name it.
76pub fn osv_ecosystem(ecosystem: &str) -> Option<&'static str> {
77 Some(match ecosystem {
78 "npm" => "npm",
79 "cargo" => "crates.io",
80 "gomod" => "Go",
81 "pip" => "PyPI",
82 _ => return None,
83 })
84}
85
86/// The `package-ecosystem` for one of OSV's names.
87pub fn package_ecosystem(osv: &str) -> Option<&'static str> {
88 Some(match osv {
89 "npm" => "npm",
90 "crates.io" => "cargo",
91 "Go" => "gomod",
92 "PyPI" => "pip",
93 _ => return None,
94 })
95}
96
97/// A package name as its ecosystem compares names.
Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches98pub(crate) fn normalize(ecosystem: &str, name: &str) -> String {
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar99 if ecosystem == "pip" { manifests::python_name(name) } else { name.to_owned() }
100}
101
102fn bare(ecosystem: &str) -> Bare {
103 if ecosystem == "cargo" { Bare::Caret } else { Bare::Exact }
104}
105
106const BASE64: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
107
108pub fn base64_encode(bytes: &[u8]) -> String {
109 let mut out = String::with_capacity(bytes.len().div_ceil(3) * 4);
110 for chunk in bytes.chunks(3) {
111 let n = (u32::from(chunk[0]) << 16) | (u32::from(*chunk.get(1).unwrap_or(&0)) << 8) | u32::from(*chunk.get(2).unwrap_or(&0));
112 for (index, shift) in [18, 12, 6, 0].into_iter().enumerate() {
113 if index <= chunk.len() {
114 out.push(BASE64[(n >> shift & 63) as usize] as char);
115 } else {
116 out.push('=');
117 }
118 }
119 }
120 out
121}
122
123pub fn base64_decode(text: &str) -> Option<Vec<u8>> {
124 let mut out = Vec::with_capacity(text.len() / 4 * 3);
125 let mut buffer = 0u32;
126 let mut bits = 0;
127 for c in text.bytes().filter(|c| !c.is_ascii_whitespace() && *c != b'=') {
128 let value = BASE64.iter().position(|known| *known == c)? as u32;
129 buffer = (buffer << 6) | value;
130 bits += 6;
131 if bits >= 8 {
132 bits -= 8;
133 out.push((buffer >> bits & 0xff) as u8);
134 }
135 }
136 Some(out)
137}
138
139/// `${{secrets.NAME}}` in `text` with each secret's value; the names of
140/// those that are not set.
141pub fn fill_secrets(text: &str, secrets: &serde_json::Map<String, Value>) -> (String, Vec<String>) {
142 let mut out = String::new();
143 let mut missing = Vec::new();
144 let mut rest = text;
145 while let Some(start) = rest.find("${{") {
146 out.push_str(&rest[..start]);
147 let after = &rest[start + 3..];
148 let Some(end) = after.find("}}") else {
149 out.push_str(&rest[start..]);
150 return (out, missing);
151 };
152 let inner = after[..end].trim();
153 match inner.strip_prefix("secrets.").map(str::trim) {
154 Some(name) => match secrets.get(name).and_then(Value::as_str) {
155 Some(value) => out.push_str(value),
156 None => missing.push(name.to_owned()),
157 },
158 None => out.push_str(&rest[start..start + 3 + end + 2]),
159 }
160 rest = &after[end + 2..];
161 }
162 out.push_str(rest);
163 (out, missing)
164}
165
166/// Whether a pull request's required checks passed (`Some(true)`), one
167/// failed (`Some(false)`), or they are still to come. Without required
168/// checks, what its workflows reported on its head.
169pub fn checks_verdict(detail: &PullDetail) -> Option<bool> {
170 if !detail.required_checks.is_empty() {
171 if detail.required_checks.iter().any(|check| check.state == RequiredState::Failure) {
172 return Some(false);
173 }
174 return detail.required_checks.iter().all(|check| check.state == RequiredState::Success).then_some(true);
175 }
176 if detail.statuses.iter().any(|status| matches!(status.state.as_str(), "failure" | "error")) {
177 return Some(false);
178 }
179 detail.statuses.iter().all(|status| status.state == "success").then_some(true)
180}
181
182/// The directories of `entry` that hold one of its manifests: each
183/// `directory`, and what each `directories` glob matches, without those
184/// `exclude-paths` covers. From the root, starting with `/`.
185pub fn entry_directories(entry: &Entry, files: &[String]) -> Vec<String> {
186 let names = manifests::manifest_names(&entry.ecosystem);
187 let mut found = BTreeSet::new();
188 for file in files {
189 let (directory, name) = file.rsplit_once('/').map_or(("", file.as_str()), |(directory, name)| (directory, name));
190 if !names.contains(&name) {
191 continue;
192 }
193 let directory = format!("/{directory}");
194 let directory = if directory == "/" { directory } else { directory.trim_end_matches('/').to_owned() };
195 for wanted in &entry.directories {
196 let hit = if wanted.contains(['*', '?']) { glob(wanted, &directory) } else { *wanted == directory };
197 if !hit {
198 continue;
199 }
200 let relative = file.strip_prefix(wanted.trim_start_matches('/')).unwrap_or(file).trim_start_matches('/');
201 let excluded = entry.exclude_paths.iter().any(|pattern| {
202 let pattern = pattern.trim_start_matches("./").trim_start_matches('/');
203 glob(pattern, relative) || relative.starts_with(&format!("{}/", pattern.trim_end_matches('/')))
204 });
205 if !excluded {
206 found.insert(directory.clone());
207 }
208 }
209 }
210 found.into_iter().collect()
211}
212
213/// The lockfiles that resolve `directory`'s dependencies in `ecosystem`:
214/// its own, or the nearest above it (a workspace's).
215pub fn lockfiles_for(ecosystem: &str, directory: &str, files: &[String]) -> Vec<String> {
216 let Some(osv) = osv_ecosystem(ecosystem) else { return Vec::new() };
217 let mut at = directory.trim_matches('/').to_owned();
218 loop {
219 let found: Vec<String> = files
220 .iter()
221 .filter(|file| {
222 let (dir, _) = file.rsplit_once('/').unwrap_or(("", file));
223 dir == at && Lockfile::for_path(file).is_some_and(|lockfile| lockfile.ecosystem().osv() == osv)
224 })
225 .cloned()
226 .collect();
227 if !found.is_empty() || at.is_empty() {
228 return found;
229 }
230 at = at.rsplit_once('/').map_or(String::new(), |(parent, _)| parent.to_owned());
231 }
232}
233
234/// What a run reads in one directory.
235pub struct Directory {
236 pub path: String,
237 pub declared: Vec<Declared>,
238 /// Each package the lockfiles resolve, by name, with every version.
239 pub locked: BTreeMap<String, Vec<String>>,
240 pub lockfiles: Vec<String>,
241}
242
243/// The dependencies a run looks at in one directory, with the version
244/// each is at now. A declared dependency with no version to be found is
245/// left out.
246pub fn directory_candidates(entry: &Entry, directory: &Directory) -> Vec<(String, DependencyType, String, Option<String>)> {
247 let ecosystem = entry.ecosystem.as_str();
248 let mut out = Vec::new();
249 let mut named = BTreeSet::new();
250 for declared in &directory.declared {
251 let name = normalize(ecosystem, &declared.name);
252 named.insert(name.clone());
253 let locked = directory.locked.get(&name).cloned().unwrap_or_default();
254 let fitting: Vec<&String> = locked
255 .iter()
256 .filter(|version| declared.requirement.as_deref().is_none_or(|req| ranges::satisfies(req, version, bare(ecosystem)) != Some(false)))
257 .collect();
258 let current = fitting
259 .into_iter()
260 .max_by(|a, b| version::compare(a, b))
261 .cloned()
262 .or_else(|| locked.iter().max_by(|a, b| version::compare(a, b)).cloned())
263 .or_else(|| match (ecosystem, declared.requirement.as_deref()) {
264 ("gomod", Some(version)) => Some(version.to_owned()),
265 ("pip", Some(requirement)) => requirement.strip_prefix("==").filter(|v| !v.contains([',', '*'])).map(str::to_owned),
266 _ => None,
267 });
268 if let Some(current) = current {
269 out.push((declared.name.clone(), declared.kind, current, declared.requirement.clone()));
270 }
271 }
272 let wants_indirect = entry.allow.iter().any(|rule| matches!(rule.dependency_type.as_deref(), Some("indirect" | "all")));
273 if wants_indirect && matches!(ecosystem, "cargo" | "gomod" | "pip") {
274 for (name, versions) in &directory.locked {
275 if named.contains(name) {
276 continue;
277 }
278 if let Some(current) = versions.iter().max_by(|a, b| version::compare(a, b)) {
279 out.push((name.clone(), DependencyType::Indirect, current.clone(), None));
280 }
281 }
282 }
283 out
284}
285
286/// What the run decided for the open-pull-requests limit: the plans to
287/// ask for now (replacing an older one when `Some`), and how many waited.
288pub fn admit<'a>(plans: &'a [PullPlan], existing: &'a [PullRow], limit: u32) -> (Vec<(&'a PullPlan, Option<&'a PullRow>)>, usize) {
289 let in_progress = |row: &&PullRow| matches!(row.state(), UpdateState::Requested | UpdateState::Open);
290 let mut open = existing.iter().filter(in_progress).count();
291 let mut admitted = Vec::new();
292 let mut held = 0;
293 let mut fresh = Vec::new();
294 for plan in plans {
295 let subject = plan.subject();
296 let signature = plan.signature();
297 let mine: Vec<&PullRow> = existing.iter().filter(|row| row.subject == subject).collect();
298 // Already open for these versions, or closed by a person for them.
299 if mine.iter().any(|row| row.signature == signature && (in_progress(row) || matches!(row.state(), UpdateState::Closed | UpdateState::NeedsCode))) {
300 continue;
301 }
302 match mine.into_iter().find(in_progress) {
303 Some(older) => admitted.push((plan, Some(older))),
304 None => fresh.push(plan),
305 }
306 }
307 for plan in fresh {
308 if open < limit as usize {
309 open += 1;
310 admitted.push((plan, None));
311 } else {
312 held += 1;
313 }
314 }
315 (admitted, held)
316}
317
318/// What the dependency update file says, as a run uses it.
319pub struct Loaded {
320 pub config: Config,
321 pub file: String,
322 pub default_branch: String,
323}
324
325#[derive(Deserialize)]
326struct CommentAuthor {
327 username: String,
328}
329
330impl Security {
331 fn repo_path(repo: &RepoRow) -> RepoPath {
332 RepoPath { namespace: repo.namespace.clone(), name: repo.name.clone() }
333 }
334
335 /// The dependency update file on the default branch, if it is there
336 /// and has no problems.
337 pub(crate) async fn load_config(&self, repo: &RepoRow) -> Result<Option<Loaded>> {
338 let path = Self::repo_path(repo);
339 let mut found = Vec::new();
340 let mut default_branch = None;
341 for file in DEPENDABOT_PATHS {
342 let read: Outcome<BlobView> = g1t_kit::call(
343 &self.repos,
344 "blob",
345 &BlobArgs { path: path.clone(), viewer: Some(User::system(&repo.namespace)), git_ref: String::new(), file_path: file.to_owned() },
346 )
347 .await?;
348 if let Outcome::Ok(blob) = read {
349 default_branch = Some(blob.repo.default_branch.clone());
350 found.push((file.to_owned(), blob.text));
351 }
352 }
353 let Some(((file, Some(text)), _)) = updates::choose(found) else { return Ok(None) };
354 let read = config::read(&text);
355 if !read.problems.is_empty() {
356 return Ok(None);
357 }
358 Ok(Some(Loaded { config: read.config, file, default_branch: default_branch.unwrap_or_default() }))
359 }
360
361 /// Schedules each entry g1t can act on, after the file is read: a new
362 /// one runs now, the rest when their schedule next says since they last
363 /// ran. A file with problems runs nothing.
364 pub(crate) async fn schedule_entries(&self, repo: &RepoRow, config: Option<&Config>, default_branch: Option<&str>) -> Result<()> {
365 let known: BTreeMap<String, Option<String>> =
366 self.store.runs(&repo.repo_id).await?.into_iter().map(|run| (run.entry, run.last_checked_at)).collect();
367 let now = now_ms();
368 let mut rows = Vec::new();
369 for entry in config.map(|config| config.updates.as_slice()).unwrap_or_default() {
370 if !runnable(entry, default_branch) {
371 continue;
372 }
373 let Some(schedule) = &entry.schedule else { continue };
374 let next = match known.get(&entry.id()) {
375 Some(Some(last)) => schedule.next_run(parse_rfc3339(last).unwrap_or(now), &updates::seed(&repo.repo_id, entry)),
376 _ => Some(now),
377 };
378 rows.push((entry.id(), next.map(rfc3339)));
379 }
380 self.store.set_runs(&repo.repo_id, &rows).await
381 }
382
383 /// The sweep's part for version updates: due entries, open update
384 /// pull requests, and sandboxes that never pushed.
385 pub async fn version_update_sweep(&self) -> Result<()> {
386 for run in self.store.due_runs(&rfc3339(now_ms()), DUE_PER_SWEEP).await? {
387 let Some(repo) = self.store.repo(&run.repo_id).await? else { continue };
388 if let Err(error) = self.run_claimed(&repo, &run.entry).await {
389 worker::console_error!("security: version updates for {} {} failed: {error}", run.repo_id, run.entry);
390 }
391 }
392 for row in self.store.open_update_pulls(WATCHED_PER_SWEEP).await? {
393 if let Err(error) = self.watch_update_pull(&row).await {
394 worker::console_error!("security: update pull request {} not looked at: {error}", row.id);
395 }
396 }
397 let before = rfc3339(now_ms().saturating_sub(STALLED_MS));
398 for row in self.store.stalled_update_pulls(&before, 10).await? {
399 let error = "The update could not be made in a sandbox: its packages may need code changes to move, or a registry could not be reached.";
400 self.store.set_update_pull(&row.id, UpdateState::Failed, row.pull(), None, Some(error)).await?;
401 }
Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once.402 // Left from when g1t opened an issue for an update whose checks
403 // failed: the issue closes, and the next run opens the update again
404 // as a pull request that stays open with its failing checks.
405 for row in self.store.needs_code_update_pulls(10).await? {
406 if let (Some(repo), Some(number)) = (self.store.repo(&row.repo_id).await?, row.issue()) {
407 let why = "g1t no longer opens issues for updates whose checks fail. The entry's next run opens this update again \
408 as a pull request, which stays open with its checks for you to decide on.";
409 self.close_issue_with(&repo, number, why.to_owned()).await?;
410 }
411 let error = "Raising the versions failed this branch's required checks.";
412 self.store.set_update_pull(&row.id, UpdateState::Failed, row.pull(), None, Some(error)).await?;
413 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar414 Ok(())
415 }
416
417 /// Runs one entry if no one else is, and schedules its next run.
418 async fn run_claimed(&self, repo: &RepoRow, entry_id: &str) -> Result<()> {
419 if !self.store.claim_run(&repo.repo_id, entry_id).await? {
420 return Ok(());
421 }
422 let loaded = self.load_config(repo).await;
423 let (result, error, next) = match loaded {
424 Ok(Some(loaded)) => match loaded.config.updates.iter().find(|entry| entry.id() == entry_id && runnable(entry, Some(&loaded.default_branch))) {
425 Some(entry) => {
426 let next = entry
427 .schedule
428 .as_ref()
429 .and_then(|schedule| schedule.next_run(now_ms(), &updates::seed(&repo.repo_id, entry)))
430 .map(rfc3339);
431 match self.run_entry(repo, &loaded, entry).await {
432 Ok(Ok(summary)) => (Some(summary), None, next),
433 Ok(Err(problem)) => (None, Some(problem), next),
434 Err(error) => (None, Some(format!("The check could not finish: {error}")), next),
435 }
436 }
437 None => (None, None, None),
438 },
439 Ok(None) => (None, Some("The dependency update file is missing or has problems.".to_owned()), None),
440 Err(error) => (None, Some(format!("The dependency update file could not be read: {error}")), None),
441 };
442 self.store.finish_run(&repo.repo_id, entry_id, next.as_deref(), result.as_deref(), error.as_deref()).await
443 }
444
445 /// `check_updates`: one entry, now.
446 pub async fn check_updates(&self, a: CheckUpdatesArgs) -> Result<Outcome<VersionUpdatesState>> {
447 let repo = match self.member_repo(&a.repo, &Some(a.actor.clone()), Capability::Push).await? {
448 Outcome::Ok(repo) => repo,
449 Outcome::Fail(failure) => return Ok(Outcome::Fail(failure)),
450 };
451 if !a.actor.verified {
452 return Ok(Outcome::fail(FailureCode::Forbidden, "Confirm your email address first."));
453 }
454 let state = repo.version_updates();
455 let Some(entry) = state.updates.iter().find(|entry| entry.id == a.entry) else {
456 return Ok(Outcome::fail(FailureCode::NotFound, "The dependency update file has no such entry."));
457 };
458 if !entry.supported {
459 return Ok(Outcome::fail(FailureCode::Invalid, format!("g1t does not open version updates for {} yet.", entry.ecosystem)));
460 }
461 if !self.store.claim_run(&repo.repo_id, &a.entry).await? {
462 return Ok(Outcome::fail(FailureCode::Conflict, "This entry is being checked now."));
463 }
464 // Claimed: run_claimed would claim again, so run it here.
465 let loaded = self.load_config(&repo).await?;
466 let (result, error) = match loaded.as_ref().and_then(|loaded| loaded.config.updates.iter().find(|entry| entry.id() == a.entry).map(|entry| (loaded, entry))) {
467 Some((loaded, entry)) => match self.run_entry(&repo, loaded, entry).await {
468 Ok(Ok(summary)) => (Some(summary), None),
469 Ok(Err(problem)) => (None, Some(problem)),
470 Err(error) => (None, Some(format!("The check could not finish: {error}"))),
471 },
472 None => (None, Some("The dependency update file is missing or has problems.".to_owned())),
473 };
474 let next = self
475 .store
476 .runs(&repo.repo_id)
477 .await?
478 .into_iter()
479 .find(|run| run.entry == a.entry)
480 .and_then(|run| run.next_run_at);
481 self.store.finish_run(&repo.repo_id, &a.entry, next.as_deref(), result.as_deref(), error.as_deref()).await?;
482 Ok(Outcome::Ok(self.version_updates_view(&repo).await?))
483 }
484
485 /// The registries an entry may use, ready for the sandbox and for
486 /// asking about versions, and what kept any from being used.
487 async fn entry_registries(&self, repo: &RepoRow, loaded: &Loaded, entry: &Entry) -> Result<(Vec<(Registry, BumpRegistry, Source)>, Vec<String>)> {
488 let kind_for = |ecosystem: &str| match ecosystem {
489 "npm" => "npm-registry",
490 "cargo" => "cargo-registry",
491 "pip" => "python-index",
492 "gomod" => "goproxy-server",
493 _ => "",
494 };
495 let wanted: Vec<&Registry> = entry
496 .registries
497 .iter()
498 .filter_map(|name| loaded.config.registries.iter().find(|registry| &registry.name == name))
499 .filter(|registry| registry.kind == kind_for(&entry.ecosystem))
500 .collect();
501 if wanted.is_empty() {
502 return Ok((Vec::new(), Vec::new()));
503 }
504 let settings: ResolvedSettings = g1t_kit::call(
505 &self.actions,
506 "resolve_settings",
507 &ResolveSettingsArgs {
508 repo_id: repo.repo_id.clone(),
509 repo: Self::repo_path(repo),
510 project_id: None,
511 project_slug: None,
512 consumer: "workflows".to_owned(),
513 environment: None,
514 trusted: true,
515 },
516 )
517 .await
518 .unwrap_or_default();
519 let mut ready = Vec::new();
520 let mut notes = Vec::new();
521 for registry in wanted {
522 if registry.oidc {
523 notes.push(format!("Registry {} signs in with OIDC, which g1t cannot do, so it was not used.", registry.name));
524 continue;
525 }
526 let mut missing = Vec::new();
527 let mut fill = |text: &Option<String>| {
528 text.as_deref().map(|text| {
529 let (filled, absent) = fill_secrets(text, &settings.secrets);
530 missing.extend(absent);
531 filled
532 })
533 };
534 let username = fill(&registry.username);
535 let password = fill(&registry.password);
536 let token = fill(&registry.token).or_else(|| fill(&registry.key));
537 if !missing.is_empty() {
538 notes.push(format!(
539 "Registry {} names secrets that are not set for this repository's workflows: {}.",
540 registry.name,
541 missing.join(", ")
542 ));
543 continue;
544 }
545 let url = registry.url.trim_end_matches('/').to_owned();
546 let authorization = match (&token, &username, &password) {
547 (Some(token), _, _) if registry.kind == "cargo-registry" => Some(token.clone()),
548 (Some(token), _, _) if registry.kind == "npm-registry" => Some(format!("Bearer {token}")),
549 (Some(token), _, _) => Some(format!("Basic {}", base64_encode(format!("__token__:{token}").as_bytes()))),
550 (None, Some(user), Some(password)) => Some(format!("Basic {}", base64_encode(format!("{user}:{password}").as_bytes()))),
551 _ => None,
552 };
553 let bump = BumpRegistry {
554 kind: registry.kind.clone(),
555 url: url.clone(),
556 username,
557 password,
558 token,
559 replaces_base: registry.replaces_base,
560 scopes: registry.scopes.clone(),
561 };
562 ready.push((registry.clone(), bump, Source { url, authorization }));
563 }
564 Ok((ready, notes))
565 }
566
567 async fn get(&self, url: &str, source: Option<&Source>, accept: &str) -> Result<Option<String>> {
568 let headers = Headers::new();
569 headers.set("user-agent", "g1t (+https://g1t.sh)")?;
570 headers.set("accept", accept)?;
571 if let Some(authorization) = source.and_then(|source| source.authorization.as_deref()) {
572 headers.set("authorization", authorization)?;
573 }
574 let mut init = RequestInit::new();
575 init.with_method(Method::Get).with_headers(headers);
576 let mut response = Fetch::Request(Request::new_with_init(url, &init)?).send().await?;
577 match response.status_code() {
578 200..=299 => Ok(Some(response.text().await?)),
579 404 | 410 => Ok(None),
580 status => Err(worker::Error::RustError(format!("{} answered {status}", url.split('/').take(3).collect::<Vec<_>>().join("/")))),
581 }
582 }
583
584 /// What `name`'s registry says, from the private one that serves it if
585 /// the entry names one.
586 async fn package(&self, ecosystem: &str, name: &str, current: &str, registries: &[(Registry, BumpRegistry, Source)]) -> Result<Option<Package>> {
587 let private = registries.iter().find(|(registry, _, _)| {
588 registry.replaces_base || registry.scopes.iter().any(|scope| name.starts_with(&format!("{scope}/")))
589 });
590 let source = private.map(|(_, _, source)| source);
591 let Some(url) = registries::package_url(ecosystem, name, source) else { return Ok(None) };
592 let accept = if ecosystem == "pip" && source.is_some() { "application/vnd.pypi.simple.v1+json" } else { "application/json, text/plain" };
593 let Some(body) = self.get(&url, source, accept).await? else { return Ok(None) };
594 let mut package = registries::read(ecosystem, name, source.is_some(), &body);
595 if ecosystem == "gomod" {
596 // When the newest few were published, for the cooldown.
597 let mut newer: Vec<usize> = (0..package.releases.len())
598 .filter(|at| version::compare(&package.releases[*at].version, current).is_gt())
599 .collect();
600 newer.sort_by(|a, b| version::compare(&package.releases[*b].version, &package.releases[*a].version));
601 for at in newer.into_iter().take(GO_TIMES) {
602 let info = url.replace("/@v/list", &format!("/@v/{}.info", package.releases[at].version));
603 if let Some(text) = self.get(&info, source, "application/json").await? {
604 package.releases[at].published_ms = serde_json::from_str(&text).ok().and_then(|value: Value| registries::go_info_time(&value));
605 }
606 }
607 }
608 Ok(Some(package))
609 }
610
611 /// One run of one entry: what it found, as a sentence, or why it could
612 /// not run.
613 async fn run_entry(&self, repo: &RepoRow, loaded: &Loaded, entry: &Entry) -> Result<std::result::Result<String, String>> {
A desktop release goes out with what can be built. Linux is built on g1t's machines in every release; Windows and macOS each wait on a repository variable: a machine registered for them, or for Windows a cross build on Linux once aka.ms and download.visualstudio.microsoft.com are workflow-only domains, which is where the first release stopped. A platform with no machine is skipped and the download page says it is coming, with a word for the visitor whose platform this release lacks. The guide and CONTRIBUTING say so.614 // Archived, deleted, or a mirror that is not taken over: nothing
615 // opens here on its own (`g1t_contracts::mirrors`).
616 if let Err(why) = self.takes_updates(repo).await? {
617 return Ok(Err(why.replace("Security updates wait", "Version updates wait")));
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar618 }
619 if entry.open_pull_requests_limit == 0 {
620 return Ok(Ok("open-pull-requests-limit is 0, so no version updates open.".to_owned()));
621 }
622 let listed: FileList = g1t_kit::call(
623 &self.repos,
624 "list_files",
625 &ListFilesArgs {
626 repo_id: repo.repo_id.clone(),
627 // The branch its updates are for: `target-branch`, or the default.
628 git_ref: target_of(entry, &loaded.default_branch),
629 skip_dirs: SKIPPED_DIRECTORIES.iter().map(|dir| (*dir).to_owned()).collect(),
630 limit: g1t_contracts::repos::MAX_LISTED_FILES,
Merge GitHub pull request #8 (claude/cool-newton-7nm8r9): Docker deployments on g1t.page behind a switch that stays off until Cloudflare's container APIs are checked live, workflows a workspace manages in its repositories, tarball and zipball downloads, refs/pull/* kept to g1t alone, framework detection for g1t.page builds, and a workflow's event payload matching GitHub's.631 links: false,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar632 },
633 )
634 .await?;
635 let paths: Vec<String> = listed.files.iter().filter(|file| file.hash.is_some()).map(|file| file.path.clone()).collect();
636 let hash_of = |path: &str| listed.files.iter().find(|file| file.path == path).and_then(|file| file.hash.clone());
637 let directories = entry_directories(entry, &paths);
638 if directories.is_empty() {
639 return Ok(Err(format!("No {} manifest was found in {}.", entry.ecosystem, entry.directories.join(", "))));
640 }
641 // Each directory's manifests and lockfiles, read at once.
642 let mut wanted: BTreeMap<String, String> = BTreeMap::new();
643 let mut plan: Vec<(String, Vec<String>, Vec<String>)> = Vec::new();
644 for directory in &directories {
645 let base = directory.trim_start_matches('/');
646 let manifests: Vec<String> = manifests::manifest_names(&entry.ecosystem)
647 .iter()
648 .map(|name| if base.is_empty() { (*name).to_owned() } else { format!("{base}/{name}") })
649 .filter(|path| paths.contains(path))
650 .collect();
651 let lockfiles = lockfiles_for(&entry.ecosystem, directory, &paths);
652 for path in manifests.iter().chain(&lockfiles) {
653 if let Some(hash) = hash_of(path) {
654 wanted.insert(path.clone(), hash);
655 }
656 }
657 plan.push((directory.clone(), manifests, lockfiles));
658 }
659 let mut texts: BTreeMap<String, String> = BTreeMap::new();
660 let entries: Vec<(String, String)> = wanted.into_iter().collect();
661 for chunk in entries.chunks(g1t_contracts::repos::MAX_READ_BLOBS) {
662 let blobs: Vec<BlobText> = g1t_kit::call(
663 &self.repos,
664 "read_blobs",
665 &ReadBlobsArgs { repo_id: repo.repo_id.clone(), hashes: chunk.iter().map(|(_, hash)| hash.clone()).collect(), max_bytes: MAX_BLOB_BYTES },
666 )
667 .await?;
668 for ((path, _), blob) in chunk.iter().zip(blobs) {
669 if let Some(text) = blob.text {
670 texts.insert(path.clone(), text);
671 }
672 }
673 }
674 let mut read: Vec<Directory> = Vec::new();
675 for (path, manifest_paths, lockfiles) in plan {
676 let mut declared = Vec::new();
677 for manifest in &manifest_paths {
678 let name = manifest.rsplit('/').next().unwrap_or(manifest);
679 // A pinned requirements.txt is also read as a lockfile.
680 declared.extend(texts.get(manifest).map(|text| manifests::declared(name, text)).unwrap_or_default());
681 }
682 let mut locked: BTreeMap<String, Vec<String>> = BTreeMap::new();
683 for lockfile in &lockfiles {
684 let Some(kind) = Lockfile::for_path(lockfile) else { continue };
685 for package in texts.get(lockfile).map(|text| kind.parse(text)).unwrap_or_default() {
686 locked.entry(normalize(&entry.ecosystem, &package.name)).or_default().push(package.version);
687 }
688 }
689 read.push(Directory { path, declared, locked, lockfiles });
690 }
691 let (registries, mut notes) = self.entry_registries(repo, loaded, entry).await?;
692 let comments = self.store.ignores(&repo.repo_id).await?;
693 // Every dependency to ask about, once.
694 let mut candidates: Vec<(String, String, DependencyType, String, Option<String>)> = Vec::new();
695 for directory in &read {
696 for (name, kind, current, requirement) in directory_candidates(entry, directory) {
697 if planning::allowed(entry, &name, kind) {
698 candidates.push((directory.path.clone(), name, kind, current, requirement));
699 }
700 }
701 }
702 let names: Vec<(String, String)> = {
703 let mut seen = BTreeMap::new();
704 for (_, name, _, current, _) in &candidates {
705 let lowest = seen.entry(name.clone()).or_insert_with(|| current.clone());
706 if version::compare(current, lowest).is_lt() {
707 *lowest = current.clone();
708 }
709 }
710 seen.into_iter().collect()
711 };
712 if names.len() > MAX_PACKAGES {
713 notes.push(format!("Only the first {MAX_PACKAGES} of {} dependencies were checked this time.", names.len()));
714 }
715 let mut packages: BTreeMap<String, Package> = BTreeMap::new();
716 let mut failures = Vec::new();
717 for chunk in names.iter().take(MAX_PACKAGES).collect::<Vec<_>>().chunks(FETCH_AT_ONCE) {
718 let asks = chunk.iter().map(|(name, current)| self.package(&entry.ecosystem, name, current, &registries));
719 for ((name, _), found) in chunk.iter().zip(futures_util::future::join_all(asks).await) {
720 match found {
721 Ok(Some(package)) => {
722 packages.insert(name.clone(), package);
723 }
724 Ok(None) => {}
725 Err(error) => failures.push(format!("{name} ({error})")),
726 }
727 }
728 }
729 let now = now_ms();
730 let mut planned = Vec::new();
731 let mut up_to_date = 0;
732 let mut ignored = 0;
733 for (directory, name, kind, current, requirement) in candidates {
734 let Some(package) = packages.get(&name) else { continue };
735 let candidate = Candidate { name, directory, kind, current, requirement, package: package.clone() };
736 match planning::target(entry, &comments, &candidate, now) {
737 Ok(update) => planned.push(update),
738 Err(Skip::UpToDate) => up_to_date += 1,
739 Err(Skip::Ignored) => ignored += 1,
740 Err(Skip::NotAllowed) => {}
741 }
742 }
743 let plans = planning::gather(&entry.groups, "version-updates", planned);
744 let existing: Vec<PullRow> = self
745 .store
746 .update_pulls(&repo.repo_id)
747 .await?
748 .into_iter()
749 .filter(|row| row.kind == "version" && row.entry == entry.id())
750 .collect();
751 let (admitted, held) = admit(&plans, &existing, entry.open_pull_requests_limit);
752 let lockfiles_of = |plan: &PullPlan| -> Vec<String> {
753 let mut all: Vec<String> = read
754 .iter()
755 .filter(|directory| plan.directories().contains(&directory.path))
756 .flat_map(|directory| directory.lockfiles.clone())
757 .collect();
758 all.sort();
759 all.dedup();
760 all
761 };
762 let mut asked = 0;
763 for (plan, _) in &admitted {
764 let lockfiles = lockfiles_of(plan);
765 if lockfiles.is_empty() {
766 notes.push(format!("{} has no lockfile, so it was not updated.", plan.directories().join(", ")));
767 continue;
768 }
769 // The same branch as one in progress is replaced in place.
770 let branch = pull_text::branch(entry, plan);
771 let force = existing.iter().any(|row| row.branch == branch && row.state().in_progress());
772 self.ask_version_update(repo, loaded, entry, plan, lockfiles, &registries, force).await?;
773 asked += 1;
774 }
775 let mut summary = format!(
776 "Checked {} {}: {} up to date{}, {} {} asked for",
777 names.len().min(MAX_PACKAGES),
778 if names.len() == 1 { "dependency" } else { "dependencies" },
779 up_to_date,
780 if ignored > 0 { format!(", {ignored} ignored") } else { String::new() },
781 asked,
782 if asked == 1 { "pull request" } else { "pull requests" },
783 );
784 if held > 0 {
785 summary.push_str(&format!(", {held} waiting for open-pull-requests-limit"));
786 }
787 summary.push('.');
788 if !failures.is_empty() {
789 notes.push(format!("These could not be looked up: {}.", failures.join(", ")));
790 }
791 for note in notes {
792 summary.push(' ');
793 summary.push_str(&note);
794 }
795 Ok(Ok(summary))
796 }
797
798 /// Asks the runner for one version update pull request.
799 #[allow(clippy::too_many_arguments)]
800 async fn ask_version_update(
801 &self,
802 repo: &RepoRow,
803 loaded: &Loaded,
804 entry: &Entry,
805 plan: &PullPlan,
806 lockfiles: Vec<String>,
807 registries: &[(Registry, BumpRegistry, Source)],
808 force: bool,
809 ) -> Result<()> {
810 let Some(osv) = osv_ecosystem(&entry.ecosystem) else { return Ok(()) };
811 let branch = pull_text::branch(entry, plan);
812 let title = pull_text::title(entry, plan);
813 let body = pull_text::body(entry, plan, &loaded.file);
814 // One version per package: the highest any directory needs.
815 let mut highest: BTreeMap<String, String> = BTreeMap::new();
816 for update in &plan.updates {
817 let known = highest.entry(update.name.clone()).or_insert_with(|| update.to.clone());
818 if version::compare(&update.to, known).is_gt() {
819 *known = update.to.clone();
820 }
821 }
822 let packages: Vec<BumpPackage> = highest.into_iter().map(|(package, version)| BumpPackage { package, version }).collect();
823 let strategy = match entry.versioning_strategy.as_deref() {
824 Some("lockfile-only") => "lockfile-only",
825 Some("increase-if-necessary") => "increase-if-necessary",
826 Some("widen") => "widen",
827 _ => "increase",
828 };
829 let mut bump = BumpArgs {
830 repo: Self::repo_path(repo),
831 ecosystem: osv.to_owned(),
832 package: packages[0].package.clone(),
833 version: packages[0].version.clone(),
834 lockfiles,
835 branch: branch.clone(),
836 message: pull_text::commit_message(entry, plan),
837 kind: Some("version".to_owned()),
838 packages,
839 strategy: Some(strategy.to_owned()),
840 force,
841 registries: Vec::new(),
842 base: target_of(entry, &loaded.default_branch),
843 };
844 let dependencies: Vec<UpdatedDependency> = plan.updates.iter().map(updated).collect();
845 let people = |names: &[String]| -> Vec<String> { names.iter().filter(|name| !name.contains('/')).cloned().collect() };
846 let id = self
847 .store
848 .add_update_pull(&NewPull {
849 repo_id: &repo.repo_id,
850 kind: "version",
851 entry: &entry.id(),
852 ecosystem: &entry.ecosystem,
853 subject: &plan.subject(),
854 signature: &plan.signature(),
855 group: plan.group.as_deref(),
856 branch: &branch,
857 title: &title,
858 body: &body,
859 dependencies: &dependencies,
860 bump: &bump,
861 assignees: &people(&entry.assignees),
862 reviewers: &people(&entry.reviewers),
863 })
864 .await?;
865 bump.registries = registries.iter().map(|(_, ready, _)| ready.clone()).collect();
866 if let Err(reason) = self.start_bump(&bump).await {
867 self.store.set_update_pull(&id, UpdateState::Failed, None, None, Some(&format!("g1t could not start the update: {reason}"))).await?;
868 }
869 Ok(())
870 }
871
872 async fn start_bump(&self, bump: &BumpArgs) -> std::result::Result<(), String> {
873 match g1t_kit::call::<_, Outcome<bool>>(&self.runner, "bump", bump).await {
874 Ok(Outcome::Ok(_)) => Ok(()),
875 Ok(Outcome::Fail(refused)) => Err(refused.message),
876 Err(error) => Err(format!("the runner could not be reached: {error}")),
877 }
878 }
879
880 /// Makes an open update pull request again from its base, on the same
881 /// branch: a rebase, a recreate, or a reopen.
882 async fn remake(&self, repo: &RepoRow, row: &PullRow) -> std::result::Result<(), String> {
883 let Some(mut bump) = row.bump() else { return Err("it was made before g1t kept how".to_owned()) };
884 bump.force = true;
885 bump.registries.clear();
886 // Credentials are never stored, and g1t's own push is awaited: see
887 // `update_pull_pushed`.
888 self.store.remake_update_pull(&row.id, &bump).await.map_err(|error| error.to_string())?;
889 if row.kind == "version"
890 && let Ok(Some(loaded)) = self.load_config(repo).await
891 && let Some(entry) = loaded.config.updates.iter().find(|entry| entry.id() == row.entry)
892 && let Ok((registries, _)) = self.entry_registries(repo, &loaded, entry).await
893 {
894 bump.registries = registries.into_iter().map(|(_, ready, _)| ready).collect();
895 }
896 self.start_bump(&bump).await
897 }
898
899 /// An update branch was pushed: its pull request opens, or a remade one
900 /// carries on. Returns whether the branch was one of these.
901 pub async fn update_pull_pushed(&self, repo_id: &str, branch: &str, after: &str) -> Result<bool> {
902 let Some(row) = self.store.update_pull_by_branch(repo_id, branch).await? else { return Ok(false) };
903 match row.state() {
904 UpdateState::Open => {
905 // g1t's own push after a rebase leaves `head` empty; anyone
906 // else's leaves it, so a rebase knows not to overwrite them.
907 if row.head.is_none() {
908 self.store.set_update_pull_head(&row.id, after).await?;
909 }
910 Ok(true)
911 }
912 UpdateState::Requested => {
913 let Some(repo) = self.store.repo(repo_id).await? else { return Ok(true) };
914 self.open_update_pull(&repo, &row, after).await?;
915 Ok(true)
916 }
Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches917 // No longer needed by the time its sandbox pushed: the branch goes.
918 UpdateState::Superseded if row.pull().is_none() => {
919 self.drop_pushed(repo_id, branch, after).await?;
920 Ok(true)
921 }
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar922 _ => Ok(true),
923 }
924 }
925
926 async fn open_update_pull(&self, repo: &RepoRow, row: &PullRow, after: &str) -> Result<()> {
927 let system = User::system(&repo.namespace);
928 let opened: Outcome<Pull> = g1t_kit::call(
929 &self.work,
930 "open_pull",
931 &OpenPullArgs {
932 actor: system.clone(),
933 repo: Self::repo_path(repo),
934 issue: None,
935 title: row.title.clone(),
936 body: row.body.clone(),
937 branch: Some(row.branch.clone()),
938 agent: String::new(),
939 runtime: Runtime::External,
940 // Into `target-branch`, which it was made from.
941 base: row.bump().and_then(|bump| bump.base),
The API opens a pull request as a draft with "draft": true942 draft: false,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar943 },
944 )
945 .await?;
946 let pull = match opened {
947 Outcome::Ok(pull) => pull,
948 Outcome::Fail(refused) => {
949 let error = format!("The pull request could not be opened: {}", refused.message);
950 return self.store.set_update_pull(&row.id, UpdateState::Failed, None, None, Some(&error)).await;
951 }
952 };
953 self.store.set_update_pull(&row.id, UpdateState::Open, Some(pull.number), None, None).await?;
954 self.store.set_update_pull_head(&row.id, after).await?;
955 let (assignees, reviewers) = (row.names("assignees"), row.names("reviewers"));
956 // Its labels and milestone, as the entry says now.
957 let entry = match self.load_config(repo).await {
958 Ok(Some(loaded)) => loaded.config.updates.into_iter().find(|entry| entry.id() == row.entry),
959 _ => None,
960 };
961 let labels = config::update_labels(entry.as_ref().and_then(|entry| entry.labels.as_deref()), &row.ecosystem);
962 let milestone = entry.as_ref().and_then(|entry| entry.milestone);
963 let _: Outcome<Value> = g1t_kit::call(
964 &self.work,
965 "update_pull",
966 &UpdatePullArgs {
967 actor: system.clone(),
968 repo: Self::repo_path(repo),
969 number: pull.number,
970 assignees: (!assignees.is_empty()).then_some(assignees),
971 reviewers: (!reviewers.is_empty()).then_some(reviewers),
972 labels: (!labels.is_empty()).then_some(labels),
973 milestone: None,
974 base: None,
975 },
976 )
977 .await?;
978 // Apart, so that a milestone the repository lacks leaves the rest.
979 if milestone.is_some() {
980 let _: Outcome<Value> = g1t_kit::call(
981 &self.work,
982 "update_pull",
983 &UpdatePullArgs {
984 actor: system.clone(),
985 repo: Self::repo_path(repo),
986 number: pull.number,
987 assignees: None,
988 reviewers: None,
989 labels: None,
990 milestone,
991 base: None,
992 },
993 )
994 .await?;
995 }
996 // Older ones for the same dependency or group are replaced.
997 for older in self.store.update_pulls(&repo.repo_id).await? {
998 if older.id == row.id || older.subject != row.subject || older.entry != row.entry || older.state() != UpdateState::Open {
999 continue;
1000 }
1001 self.store.set_update_pull(&older.id, UpdateState::Superseded, older.pull(), None, None).await?;
Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches1002 if let Some(number) = older.pull()
1003 && let Some(found) = self.get_pull(repo, number).await?
1004 && matches!(found.status, PullStatus::Open | PullStatus::Draft)
1005 && self.close_with(repo, number, format!("Closed: superseded by #{}.", pull.number)).await?
1006 && older.branch != row.branch
1007 {
1008 self.delete_pull_branch(repo, &Pull { status: PullStatus::Closed, ..found }).await;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1009 }
1010 }
1011 // A grouped security update stands for each package's own.
1012 if row.kind == "security" {
1013 for dependency in row.dependencies() {
1014 let Some(osv) = osv_ecosystem(&row.ecosystem) else { continue };
1015 if let Some(update) = self.store.update(&repo.repo_id, osv, &dependency.name).await? {
1016 self.store.set_update(&update, UpdateState::Open, Some(pull.number), None, None).await?;
1017 }
1018 }
1019 }
1020 Ok(())
1021 }
1022
1023 /// A pull request merged or closed: if it is an update's, where it
1024 /// stands now. Returns whether it was one.
1025 pub async fn update_pull_closed(&self, kind: &str, repo_id: &str, number: u32) -> Result<bool> {
1026 let Some(row) = self.store.update_pull_by_number(repo_id, number).await? else { return Ok(false) };
1027 if row.state() != UpdateState::Open {
1028 return Ok(true);
1029 }
1030 let state = if kind == "pull.merged" { UpdateState::Merged } else { UpdateState::Closed };
1031 self.store.set_update_pull(&row.id, state, Some(number), None, None).await?;
1032 if row.kind == "security" {
1033 for dependency in row.dependencies() {
1034 let Some(osv) = osv_ecosystem(&row.ecosystem) else { continue };
1035 if let Some(update) = self.store.update(repo_id, osv, &dependency.name).await? {
1036 self.store.set_update(&update, state, Some(number), None, None).await?;
1037 }
1038 }
1039 }
1040 Ok(true)
1041 }
1042
1043 async fn pull_detail(&self, repo: &RepoRow, number: u32) -> Result<Option<PullDetail>> {
1044 let found: Outcome<PullDetail> = g1t_kit::call(
1045 &self.work,
1046 "get_pull",
1047 &ViewArgs { repo: Self::repo_path(repo), number, viewer: Some(User::system(&repo.namespace)), after_seq: 0 },
1048 )
1049 .await?;
1050 Ok(found.into_result().ok())
1051 }
1052
1053 /// Looks at one open update pull request: merged or closed meanwhile,
1054 /// its checks failing or passing, or in conflict with its base.
1055 async fn watch_update_pull(&self, row: &PullRow) -> Result<()> {
1056 self.store.touch_update_pull(&row.id).await?;
1057 let (Some(repo), Some(number)) = (self.store.repo(&row.repo_id).await?, row.pull()) else { return Ok(()) };
1058 let Some(detail) = self.pull_detail(&repo, number).await? else { return Ok(()) };
1059 match detail.pull.status {
1060 PullStatus::Merged => return self.update_pull_closed("pull.merged", &row.repo_id, number).await.map(|_| ()),
1061 PullStatus::Closed => return self.update_pull_closed("pull.closed", &row.repo_id, number).await.map(|_| ()),
1062 _ => {}
1063 }
1064 let ours = row.head.is_some() && row.head == detail.pull.head_commit;
1065 match checks_verdict(&detail) {
1066 Some(true) if row.merge_by.is_some() => {
1067 let merged: Outcome<Pull> = g1t_kit::call(
1068 &self.work,
1069 "merge_pull",
1070 &PullActionArgs {
1071 actor: User::system(&repo.namespace),
1072 repo: Self::repo_path(&repo),
1073 number,
1074 summary: String::new(),
1075 keep_issue_open: false,
1076 ignore_checks: false,
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge1077 bypass_rules: false,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1078 },
1079 )
1080 .await?;
1081 if let Outcome::Fail(refused) = merged {
1082 self.store.set_merge_by(&row.id, None).await?;
1083 self.comment(&User::system(&repo.namespace), &Self::repo_path(&repo), number, format!("I could not merge this: {}", refused.message))
1084 .await?;
1085 }
1086 return Ok(());
1087 }
1088 _ => {}
1089 }
1090 let rebases = row.kind == "security" || self.rebase_strategy(&repo, &row.entry) != "disabled";
1091 if detail.mergeable == Mergeable::Conflicting
1092 && ours
1093 && rebases
1094 && let Err(reason) = self.remake(&repo, row).await
1095 {
1096 worker::console_error!("security: update {} not rebased: {reason}", row.id);
1097 }
1098 Ok(())
1099 }
1100
1101 /// An entry's `rebase-strategy`, as last read.
1102 fn rebase_strategy(&self, repo: &RepoRow, entry: &str) -> String {
1103 repo.version_updates()
1104 .updates
1105 .iter()
1106 .find(|found| found.id == entry)
1107 .and_then(|found| found.options.get("rebase-strategy").and_then(Value::as_str).map(str::to_owned))
1108 .unwrap_or_else(|| "auto".to_owned())
1109 }
1110
1111 /// A comment on an update pull request: a command, if it is one.
1112 pub async fn update_comment(&self, event: &Event) -> Result<()> {
1113 #[derive(Deserialize)]
1114 #[serde(rename_all = "camelCase")]
1115 struct Commented {
1116 comment_id: String,
1117 repo_id: String,
1118 number: u32,
1119 #[serde(default)]
1120 pull_id: Option<String>,
1121 }
1122 let Ok(commented) = serde_json::from_value::<Commented>(event.data.clone()) else { return Ok(()) };
1123 if commented.pull_id.is_none() || event.actor.as_deref().is_some_and(g1t_contracts::system::is_system_id) {
1124 return Ok(());
1125 }
1126 let row = self.store.update_pull_by_number(&commented.repo_id, commented.number).await?;
1127 let single = self.store.update_by_pull(&commented.repo_id, commented.number).await?;
1128 if row.is_none() && single.is_none() {
1129 return Ok(());
1130 }
1131 let Some(repo) = self.store.repo(&commented.repo_id).await? else { return Ok(()) };
1132 let Some(detail) = self.pull_detail(&repo, commented.number).await? else { return Ok(()) };
1133 let Some(comment) = detail.comments.iter().find(|comment| comment.id == commented.comment_id) else { return Ok(()) };
1134 let Some(command) = update_command(&comment.body) else { return Ok(()) };
1135 let author = serde_json::from_value::<CommentAuthor>(serde_json::to_value(&comment.author)?).map(|a| a.username).unwrap_or_default();
1136 let system = User::system(&repo.namespace);
1137 let path = Self::repo_path(&repo);
1138 let needed = if matches!(command, UpdateCommand::Merge | UpdateCommand::SquashAndMerge | UpdateCommand::CancelMerge) {
1139 Capability::Merge
1140 } else {
1141 Capability::Push
1142 };
1143 let permission: Outcome<PermissionInfo> = g1t_kit::call(
1144 &self.identity,
1145 "collaborator_permission",
1146 &CollaboratorPermissionArgs { viewer: Some(system.clone()), path: path.clone(), username: author.clone() },
1147 )
1148 .await?;
1149 let allowed = matches!(&permission, Outcome::Ok(info) if info.capabilities.contains(&needed));
1150 if !allowed {
1151 let text = format!("@{author}, that takes the {} role on this repository.", if needed == Capability::Merge { "Maintain or Write" } else { "Write" });
1152 return self.comment(&system, &path, commented.number, text).await;
1153 }
1154 let Some(row) = row else {
1155 // A security update for one package: the commands that make sense for it.
1156 return self.single_update_command(&repo, single.as_ref(), &command, &author, commented.number, &detail).await;
1157 };
1158 let reply = self.command(&repo, &row, &command, &author, &detail).await?;
1159 if let Some(reply) = reply {
1160 self.comment(&system, &path, commented.number, reply).await?;
1161 }
1162 Ok(())
1163 }
1164
1165 /// Acts on a command on an update pull request; what to say back.
1166 async fn command(&self, repo: &RepoRow, row: &PullRow, command: &UpdateCommand, author: &str, detail: &PullDetail) -> Result<Option<String>> {
1167 let number = detail.pull.number;
1168 let open = detail.pull.status.is_active();
1169 let ignore = |dependency: &str, versions: Option<String>, update_type: Option<String>| IgnoreCondition {
1170 ecosystem: row.ecosystem.clone(),
1171 dependency: dependency.to_owned(),
1172 versions,
1173 update_type,
1174 by: author.to_owned(),
1175 pull: Some(number),
1176 at: String::new(),
1177 };
1178 let dependencies = row.dependencies();
1179 Ok(Some(match command {
1180 UpdateCommand::Rebase | UpdateCommand::Recreate => {
1181 if !open {
1182 return Ok(Some(format!("@{author}, this pull request is {}; `@g1t reopen` opens it again.", detail.pull.status.as_str())));
1183 }
1184 let pushed_by_others = row.head.is_some() && row.head != detail.pull.head_commit;
1185 if *command == UpdateCommand::Rebase && pushed_by_others {
1186 return Ok(Some(format!("@{author}, someone else has pushed to this branch, so a rebase would drop their commits. `@g1t recreate` makes it again from scratch.")));
1187 }
1188 match self.remake(repo, row).await {
1189 Ok(()) => format!("@{author}, {} this pull request from its base branch now.", if *command == UpdateCommand::Rebase { "rebasing" } else { "recreating" }),
1190 Err(reason) => format!("@{author}, I could not start that: {reason}"),
1191 }
1192 }
1193 UpdateCommand::Merge | UpdateCommand::SquashAndMerge => {
1194 if !open {
1195 return Ok(Some(format!("@{author}, this pull request is already {}.", detail.pull.status.as_str())));
1196 }
1197 self.store.set_merge_by(&row.id, Some(author)).await?;
1198 match checks_verdict(detail) {
1199 Some(false) => format!("@{author}, its required checks are failing; it merges once they pass."),
1200 Some(true) => {
1201 self.watch_update_pull(row).await?;
1202 return Ok(None);
1203 }
1204 None => format!("@{author}, it merges once its required checks pass."),
1205 }
1206 }
1207 UpdateCommand::CancelMerge => {
1208 self.store.set_merge_by(&row.id, None).await?;
1209 format!("@{author}, it will not merge on its own now.")
1210 }
1211 UpdateCommand::Close => {
1212 self.store.set_update_pull(&row.id, UpdateState::Closed, row.pull(), None, None).await?;
1213 self.close_with(repo, number, format!("@{author}, closed. g1t will not open a pull request for these versions again, but will when a newer one is out.")).await?;
1214 return Ok(None);
1215 }
1216 UpdateCommand::Reopen => {
1217 if open {
1218 return Ok(Some(format!("@{author}, this pull request is already open.")));
1219 }
1220 self.store.set_update_pull(&row.id, UpdateState::Requested, None, None, None).await?;
1221 match self.remake(repo, row).await {
1222 Ok(()) => format!("@{author}, making it again; it opens as a new pull request on the same branch."),
1223 Err(reason) => format!("@{author}, I could not start that: {reason}"),
1224 }
1225 }
1226 UpdateCommand::IgnoreDependency | UpdateCommand::IgnoreVersion { .. } => {
1227 if row.group_name.is_some() && dependencies.len() > 1 {
1228 return Ok(Some(format!(
1229 "@{author}, this pull request updates several dependencies. Name one: `@g1t ignore <dependency>`, or `@g1t ignore <dependency> major version`."
1230 )));
1231 }
1232 for dependency in &dependencies {
1233 let condition = match command {
1234 UpdateCommand::IgnoreVersion { level } => ignore(&dependency.name, Some(ranges::ignore_level(&dependency.to, level)), None),
1235 _ => ignore(&dependency.name, None, None),
1236 };
1237 self.store.add_ignore(&repo.repo_id, &condition).await?;
1238 }
1239 self.store.set_update_pull(&row.id, UpdateState::Closed, row.pull(), None, None).await?;
1240 let what = match command {
1241 UpdateCommand::IgnoreVersion { level } => format!("this {level} version"),
1242 _ => "this dependency".to_owned(),
1243 };
1244 self.close_with(repo, number, format!("@{author}, g1t will skip {what} from now on. `@g1t unignore {}` undoes it.", dependencies.first().map(|d| d.name.as_str()).unwrap_or("*")))
1245 .await?;
1246 return Ok(None);
1247 }
1248 UpdateCommand::IgnoreNamed { dependency, level } => {
1249 let Some(found) = dependencies.iter().find(|d| d.name.eq_ignore_ascii_case(dependency)) else {
1250 return Ok(Some(format!("@{author}, this pull request does not update {dependency}.")));
1251 };
1252 let condition = match level {
1253 Some(level) => ignore(&found.name, Some(ranges::ignore_level(&found.to, level)), None),
1254 None => ignore(&found.name, None, None),
1255 };
1256 self.store.add_ignore(&repo.repo_id, &condition).await?;
1257 format!("@{author}, g1t will skip {}{}; the next check leaves it out of this group.", found.name, level.as_deref().map(|level| format!(" {level} versions like {}", found.to)).unwrap_or_default())
1258 }
1259 UpdateCommand::Unignore { dependency, level } => {
1260 let condition = match (level, dependencies.iter().find(|d| d.name.eq_ignore_ascii_case(dependency))) {
1261 (Some(level), Some(found)) => Some(ranges::ignore_level(&found.to, level)),
1262 _ => None,
1263 };
1264 let removed = self.store.remove_ignores(&repo.repo_id, &row.ecosystem, dependency, condition.as_deref()).await?;
1265 if removed == 0 {
1266 format!("@{author}, nothing was being ignored for {dependency}.")
1267 } else {
1268 format!("@{author}, removed {removed} ignore {} for {dependency}.", if removed == 1 { "condition" } else { "conditions" })
1269 }
1270 }
1271 UpdateCommand::ShowIgnores { dependency } => {
1272 let names: Vec<String> = match dependency {
1273 Some(name) => vec![name.clone()],
1274 None => dependencies.iter().map(|d| d.name.clone()).collect(),
1275 };
1276 self.ignore_report(repo, &row.ecosystem, &row.entry, &names).await?
1277 }
1278 }))
1279 }
1280
1281 /// The ignore conditions on some dependencies, from the file and from
1282 /// comments, as a reply.
1283 async fn ignore_report(&self, repo: &RepoRow, ecosystem: &str, entry: &str, names: &[String]) -> Result<String> {
1284 let state = repo.version_updates();
1285 let rules = state.updates.iter().find(|found| found.id == entry).map(|found| found.ignore.clone()).unwrap_or_default();
1286 let comments = self.store.ignores(&repo.repo_id).await?;
1287 let mut lines = Vec::new();
1288 for name in names {
1289 for rule in rules.iter().filter(|rule| config::matches(&rule.dependency, name)) {
1290 let what = if !rule.versions.is_empty() {
1291 format!("versions {}", rule.versions.join(", "))
1292 } else if !rule.update_types.is_empty() {
1293 rule.update_types.join(", ")
1294 } else {
1295 "every version".to_owned()
1296 };
1297 lines.push(format!("- `{name}`: {what} (in the dependency update file, `{}`)", rule.dependency));
1298 }
1299 for condition in comments.iter().filter(|c| c.ecosystem == ecosystem && c.dependency.eq_ignore_ascii_case(name)) {
1300 let what = condition
1301 .versions
1302 .clone()
1303 .map(|versions| format!("versions `{versions}`"))
1304 .or_else(|| condition.update_type.clone())
1305 .unwrap_or_else(|| "every version".to_owned());
1306 lines.push(format!(
1307 "- `{name}`: {what} (asked by @{}{})",
1308 condition.by,
1309 condition.pull.map(|pull| format!(" in #{pull}")).unwrap_or_default()
1310 ));
1311 }
1312 }
1313 Ok(if lines.is_empty() {
1314 format!("Nothing is ignored for {}.", names.join(", "))
1315 } else {
1316 format!("Ignore conditions:\n\n{}", lines.join("\n"))
1317 })
1318 }
1319
1320 /// Commands on a security update for one package.
1321 async fn single_update_command(
1322 &self,
1323 repo: &RepoRow,
1324 update: Option<&crate::store::UpdateRow>,
1325 command: &UpdateCommand,
1326 author: &str,
1327 number: u32,
1328 detail: &PullDetail,
1329 ) -> Result<()> {
1330 let Some(update) = update else { return Ok(()) };
1331 let system = User::system(&repo.namespace);
1332 let path = Self::repo_path(repo);
1333 let Some(ecosystem) = package_ecosystem(&update.ecosystem) else { return Ok(()) };
1334 let ignore = |versions: Option<String>| IgnoreCondition {
1335 ecosystem: ecosystem.to_owned(),
1336 dependency: update.package.clone(),
1337 versions,
1338 update_type: None,
1339 by: author.to_owned(),
1340 pull: Some(number),
1341 at: String::new(),
1342 };
1343 let reply = match command {
1344 UpdateCommand::Close => {
1345 self.store.set_update(update, UpdateState::Closed, Some(number), None, None).await?;
1346 self.close_with(repo, number, format!("@{author}, closed. g1t will not open a security update for {} {} again.", update.package, update.target)).await?;
1347 return Ok(());
1348 }
1349 UpdateCommand::IgnoreDependency | UpdateCommand::IgnoreVersion { .. } => {
1350 let versions = match command {
1351 UpdateCommand::IgnoreVersion { level } => Some(ranges::ignore_level(&update.target, level)),
1352 _ => None,
1353 };
1354 self.store.add_ignore(&repo.repo_id, &ignore(versions)).await?;
1355 self.store.set_update(update, UpdateState::Closed, Some(number), None, None).await?;
1356 self.close_with(repo, number, format!("@{author}, g1t will skip that for {} from now on. `@g1t unignore {}` undoes it.", update.package, update.package)).await?;
1357 return Ok(());
1358 }
1359 UpdateCommand::Unignore { dependency, .. } => {
1360 let removed = self.store.remove_ignores(&repo.repo_id, ecosystem, dependency, None).await?;
1361 format!("@{author}, removed {removed} ignore conditions for {dependency}.")
1362 }
1363 UpdateCommand::ShowIgnores { dependency } => {
1364 let name = dependency.clone().unwrap_or_else(|| update.package.clone());
1365 self.ignore_report(repo, ecosystem, "", &[name]).await?
1366 }
1367 UpdateCommand::Merge | UpdateCommand::SquashAndMerge => match checks_verdict(detail) {
1368 Some(true) => {
1369 let merged: Outcome<Pull> = g1t_kit::call(
1370 &self.work,
1371 "merge_pull",
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge1372 &PullActionArgs { actor: system.clone(), repo: path.clone(), number, summary: String::new(), keep_issue_open: false, ignore_checks: false, bypass_rules: false },
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1373 )
1374 .await?;
1375 match merged {
1376 Outcome::Ok(_) => return Ok(()),
1377 Outcome::Fail(refused) => format!("@{author}, I could not merge this: {}", refused.message),
1378 }
1379 }
1380 _ => format!("@{author}, its required checks have not passed yet; merge it once they do."),
1381 },
1382 _ => format!("@{author}, a security update is made again by `Re-scan now` on the Security page; this command is for version updates."),
1383 };
1384 self.comment(&system, &path, number, reply).await
1385 }
1386
1387 /// A pull request that changes the dependency update file gets a
1388 /// status saying whether the file is valid.
1389 pub async fn check_dependabot_file(&self, event: &Event) -> Result<()> {
1390 #[derive(Deserialize)]
1391 #[serde(rename_all = "camelCase")]
1392 struct Changed {
1393 repo_id: String,
1394 number: u32,
1395 }
1396 let Ok(changed) = serde_json::from_value::<Changed>(event.data.clone()) else { return Ok(()) };
1397 let Some(repo) = self.store.repo(&changed.repo_id).await? else { return Ok(()) };
1398 let Some(detail) = self.pull_detail(&repo, changed.number).await? else { return Ok(()) };
1399 let pull = &detail.pull;
1400 let Some(file) = pull.files.iter().find(|file| DEPENDABOT_PATHS.contains(&file.path.as_str())) else { return Ok(()) };
1401 let Some(head) = pull.head_commit.clone() else { return Ok(()) };
1402 let found: Option<RawFile> = g1t_kit::call(
1403 &self.repos,
1404 "raw_file",
1405 &RawFileArgs {
1406 repo_id: pull.fork_repo_id.clone().unwrap_or_else(|| repo.repo_id.clone()),
1407 git_ref: head.clone(),
1408 path: file.path.clone(),
1409 max_bytes: 1_000_000,
1410 },
1411 )
1412 .await?;
1413 let (state, description) = match found.and_then(|raw| base64_decode(&raw.data)).map(String::from_utf8) {
1414 None => ("success".to_owned(), format!("{} is removed.", file.path)),
1415 Some(Err(_)) => ("failure".to_owned(), format!("{} is not text.", file.path)),
1416 Some(Ok(text)) => {
1417 let read = config::read(&text);
1418 match read.problems.first() {
1419 None => ("success".to_owned(), format!("{} is valid: {} entries.", file.path, read.config.updates.len())),
1420 Some(problem) => {
1421 let more = if read.problems.len() > 1 { format!(" (and {} more)", read.problems.len() - 1) } else { String::new() };
1422 ("failure".to_owned(), format!("{}{more}", problem.sentence()))
1423 }
1424 }
1425 }
1426 };
1427 let site = format!("https://g1t.sh/{}/{}/blob/{head}/{}", repo.namespace, repo.name, file.path);
1428 let _: Outcome<bool> = g1t_kit::call(
1429 &self.work,
1430 "set_commit_status",
1431 &SetCommitStatusArgs {
1432 repo_id: repo.repo_id.clone(),
1433 sha: head,
1434 context: DEPENDABOT_CHECK.to_owned(),
1435 state,
1436 description: Some(description.chars().take(400).collect()),
1437 target_url: Some(site),
Merge rulesets: branch and tag rules, agent-first, enforced on push and merge1438 source: Some("security".to_owned()),
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1439 },
1440 )
1441 .await?;
1442 Ok(())
1443 }
1444
1445 /// What the Security page shows: the file as last read, with each
1446 /// entry's runs, the update pull requests, and comment ignores.
1447 pub async fn version_updates_view(&self, repo: &RepoRow) -> Result<VersionUpdatesState> {
1448 let mut state = repo.version_updates();
1449 let runs = self.store.runs(&repo.repo_id).await?;
1450 for entry in &mut state.updates {
1451 if let Some(run) = runs.iter().find(|run| run.entry == entry.id) {
1452 entry.next_run_at = run.next_run_at.clone();
1453 entry.last_checked_at = run.last_checked_at.clone();
1454 entry.last_result = run.last_result.clone();
1455 entry.last_error = run.last_error.clone();
1456 }
1457 }
1458 let rows = self.store.update_pulls(&repo.repo_id).await?;
1459 let (live, done): (Vec<&PullRow>, Vec<&PullRow>) = rows.iter().partition(|row| row.state().in_progress());
1460 state.pulls = live.into_iter().chain(done.into_iter().take(20)).map(PullRow::to_contract).collect();
1461 state.ignores = self.store.ignores(&repo.repo_id).await?;
1462 Ok(state)
1463 }
1464}
1465
1466/// Whether g1t runs an entry: an ecosystem it updates, and pull requests
1467/// it can open (`open-pull-requests-limit` above 0), into the default
1468/// branch or the entry's `target-branch`.
1469pub fn runnable(entry: &Entry, _default_branch: Option<&str>) -> bool {
1470 entry.supported() && entry.open_pull_requests_limit > 0 && entry.schedule.is_some()
1471}
1472
1473/// The branch an entry's updates start from and merge into, when it is not
1474/// the default branch.
1475pub fn target_of(entry: &Entry, default_branch: &str) -> Option<String> {
1476 entry.target_branch.clone().filter(|branch| !branch.is_empty() && branch != default_branch)
1477}
1478
1479fn updated(update: &Planned) -> UpdatedDependency {
1480 UpdatedDependency {
1481 name: update.name.clone(),
1482 from: update.from.clone(),
1483 to: update.to.clone(),
1484 directory: update.directory.clone(),
1485 dependency_type: update.kind.label().to_owned(),
1486 update_type: format!("version-update:semver-{}", update.level),
1487 }
1488}
1489
1490#[cfg(test)]
1491mod tests {
1492 use super::*;
1493 use crate::config::read;
1494 use g1t_contracts::work::{CommitStatus, RequiredCheck};
Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in.1495 use serde_json::json;
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1496
1497 fn entry(extra: &str) -> Entry {
1498 let found = read(&format!("version: 2\nupdates:\n - package-ecosystem: npm\n{extra} schedule: {{interval: daily}}\n"));
1499 assert!(found.problems.is_empty(), "{:?}", found.problems);
1500 found.config.updates.into_iter().next().unwrap()
1501 }
1502
1503 #[test]
1504 fn base64_round_trips() {
1505 for text in ["", "a", "ab", "abc", "user:pa$$word", "version: 2\n"] {
1506 assert_eq!(base64_decode(&base64_encode(text.as_bytes())).unwrap(), text.as_bytes());
1507 }
1508 assert_eq!(base64_encode(b"ci:secret"), "Y2k6c2VjcmV0");
1509 }
1510
1511 #[test]
1512 fn secrets_are_filled_in() {
1513 let mut secrets = serde_json::Map::new();
1514 secrets.insert("TOKEN".into(), json!("t0k"));
1515 assert_eq!(fill_secrets("${{secrets.TOKEN}}", &secrets), ("t0k".to_owned(), vec![]));
1516 assert_eq!(fill_secrets("Bearer ${{ secrets.TOKEN }}!", &secrets).0, "Bearer t0k!");
1517 assert_eq!(fill_secrets("${{secrets.NOPE}}", &secrets).1, ["NOPE"]);
1518 assert_eq!(fill_secrets("plain", &secrets).0, "plain");
1519 }
1520
1521 #[test]
1522 fn directories_from_globs_and_exclusions() {
1523 let files: Vec<String> = ["package.json", "apps/web/package.json", "apps/admin/package.json", "apps/web/vendor/x/package.json", "docs/readme.md"]
1524 .map(str::to_owned)
1525 .to_vec();
1526 assert_eq!(entry_directories(&entry(" directory: /\n"), &files), ["/"]);
1527 assert_eq!(entry_directories(&entry(" directories: [\"/apps/*\"]\n"), &files), ["/apps/admin", "/apps/web"]);
1528 assert_eq!(
1529 entry_directories(&entry(" directories: [\"/apps/**\"]\n exclude-paths: [\"**/vendor/**\"]\n"), &files),
1530 ["/apps/admin", "/apps/web"]
1531 );
1532 assert!(entry_directories(&entry(" directory: /missing\n"), &files).is_empty());
1533 let locks: Vec<String> = ["package-lock.json", "apps/web/package.json", "Cargo.lock"].map(str::to_owned).to_vec();
1534 assert_eq!(lockfiles_for("npm", "/apps/web", &locks), ["package-lock.json"]);
1535 assert!(lockfiles_for("pip", "/", &locks).is_empty());
1536 }
1537
1538 #[test]
1539 fn current_versions_come_from_the_lockfile() {
1540 let directory = Directory {
1541 path: "/".into(),
1542 declared: manifests::package_json(r#"{"dependencies":{"lodash":"^4.17.0","left-pad":"1.0.0"},"devDependencies":{"vitest":"^1"}}"#),
1543 locked: BTreeMap::from([
1544 ("lodash".to_owned(), vec!["4.17.20".to_owned(), "3.10.1".to_owned()]),
1545 ("minimist".to_owned(), vec!["1.2.0".to_owned()]),
1546 ]),
1547 lockfiles: vec!["package-lock.json".into()],
1548 };
1549 let found = directory_candidates(&entry(" directory: /\n"), &directory);
1550 assert_eq!(found, [("lodash".to_owned(), DependencyType::Production, "4.17.20".to_owned(), Some("^4.17.0".to_owned()))]);
1551 let go = Directory {
1552 path: "/".into(),
1553 declared: manifests::go_mod("require golang.org/x/net v0.7.0\n"),
1554 locked: BTreeMap::new(),
1555 lockfiles: vec!["go.mod".into()],
1556 };
1557 let mut go_entry = entry(" directory: /\n");
1558 go_entry.ecosystem = "gomod".into();
1559 let found = directory_candidates(&go_entry, &go);
1560 assert_eq!(found[0].2, "v0.7.0");
1561 }
1562
1563 fn row(subject: &str, signature: &str, state: &str) -> PullRow {
1564 PullRow {
1565 id: format!("upd_{subject}_{state}"),
1566 repo_id: "rep_1".into(),
1567 kind: "version".into(),
1568 entry: "npm:/".into(),
1569 ecosystem: "npm".into(),
1570 subject: subject.into(),
1571 signature: signature.into(),
1572 group_name: None,
1573 branch: "g1t/npm_and_yarn/x".into(),
1574 title: String::new(),
1575 body: String::new(),
1576 dependencies: "[]".into(),
1577 bump: "{}".into(),
1578 assignees: "[]".into(),
1579 reviewers: "[]".into(),
1580 state: state.into(),
1581 pull: Some(3),
Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once.1582 issue: None,
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1583 head: None,
1584 merge_by: None,
1585 error: None,
1586 updated_at: String::new(),
1587 }
1588 }
1589
1590 fn plan(name: &str, to: &str) -> PullPlan {
1591 PullPlan {
1592 group: None,
1593 by_name: false,
1594 updates: vec![Planned {
1595 name: name.into(),
1596 directory: "/".into(),
1597 kind: DependencyType::Production,
1598 from: "1.0.0".into(),
1599 to: to.into(),
1600 level: "minor",
1601 source: None,
1602 changelog: None,
1603 page: None,
1604 }],
1605 }
1606 }
1607
1608 #[test]
1609 fn the_limit_counts_open_pull_requests_and_replacements_do_not() {
1610 let plans = vec![plan("a", "1.1.0"), plan("b", "1.1.0"), plan("c", "1.1.0"), plan("d", "1.1.0")];
1611 let existing = vec![
1612 row("dependency:/:a", "/a@1.0.5", "open"),
1613 row("dependency:/:b", "/b@1.1.0", "open"),
1614 row("dependency:/:c", "/c@1.1.0", "closed"),
1615 ];
1616 let (admitted, held) = admit(&plans, &existing, 3);
1617 let names: Vec<(&str, bool)> = admitted.iter().map(|(plan, older)| (plan.updates[0].name.as_str(), older.is_some())).collect();
1618 // a replaces its older pull request; b is open already; c was closed by a person for this version; d is new, and fits.
1619 assert_eq!(names, [("a", true), ("d", false)]);
1620 assert_eq!(held, 0);
1621 let (admitted, held) = admit(&plans, &existing, 2);
1622 assert_eq!(admitted.len(), 1);
1623 assert_eq!(held, 1);
1624 }
1625
1626 #[test]
1627 fn checks_decide_failure_and_success() {
1628 let check = |state| RequiredCheck { name: "CI".into(), state, description: None, target_url: None };
Merge checks: statuses and check runs on every commit1629 let status = |state: &str| CommitStatus { context: "CI / push".into(), state: state.into(), description: None, target_url: None, updated_at: String::new(), source: None, check_run_id: None };
Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar1630 let detail = |required: Vec<RequiredCheck>, statuses: Vec<CommitStatus>| {
1631 let mut detail: PullDetail = serde_json::from_value(json!({
1632 "pull": {"id": "pul_1", "repoId": "rep_1", "number": 1, "issue": null, "title": "t", "body": null, "agent": "", "runtime": "external",
1633 "status": "open", "fork": null, "forkRepoId": null, "branch": "b", "headCommit": "c", "mergeBase": null, "mergedBy": null,
1634 "mergedAt": null, "supersededBy": null, "checkStatus": null,
1635 "author": {"id": "g1t", "username": "g1t", "kind": "system"}, "createdAt": "", "updatedAt": ""},
1636 "issue": null, "comments": [], "checks": null
1637 }))
1638 .unwrap();
1639 detail.required_checks = required;
1640 detail.statuses = statuses;
1641 detail
1642 };
1643 assert_eq!(checks_verdict(&detail(vec![check(RequiredState::Success)], vec![])), Some(true));
1644 assert_eq!(checks_verdict(&detail(vec![check(RequiredState::Success), check(RequiredState::Failure)], vec![])), Some(false));
1645 assert_eq!(checks_verdict(&detail(vec![check(RequiredState::Pending)], vec![])), None);
1646 assert_eq!(checks_verdict(&detail(vec![], vec![status("error")])), Some(false));
1647 assert_eq!(checks_verdict(&detail(vec![], vec![status("pending")])), None);
1648 assert_eq!(checks_verdict(&detail(vec![], vec![])), Some(true));
1649 }
1650
1651 #[test]
1652 fn which_entries_run() {
1653 assert!(runnable(&entry(" directory: /\n"), Some("main")));
1654 assert!(!runnable(&entry(" directory: /\n open-pull-requests-limit: 0\n"), Some("main")));
1655 assert!(runnable(&entry(" directory: /\n target-branch: main\n"), Some("main")));
1656 assert!(runnable(&entry(" directory: /\n target-branch: develop\n"), Some("main")));
1657 assert_eq!(target_of(&entry(" directory: /\n target-branch: develop\n"), "main").as_deref(), Some("develop"));
1658 assert_eq!(target_of(&entry(" directory: /\n target-branch: main\n"), "main"), None);
1659 assert_eq!(target_of(&entry(" directory: /\n"), "main"), None);
1660 assert_eq!(osv_ecosystem("gomod"), Some("Go"));
1661 assert_eq!(package_ecosystem("crates.io"), Some("cargo"));
1662 }
1663}

This file's history is long; its oldest lines are credited to the oldest commit read.