Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 1 | //! Security updates: for each vulnerable package with a fix, g1t itself |
| 2 | //! opens a pull request that raises its version. | |
| 3 | //! | |
| 4 | //! 1. A dependency scan asks the runner for a `bump` (most severe first): | |
| 5 | //! a sandbox raises the package in each lockfile with the ecosystem's | |
| 6 | //! own tool and pushes that to `g1t/security/<package>-<version>`. | |
| 7 | //! 2. That push (`git.push`) opens the pull request, authored by g1t | |
| 8 | //! (`User::system`). It lands through the branch's required checks like | |
| 9 | //! any other. An older one for the same package is closed as superseded. | |
| 10 | //! 3. When its checks fail because code has to change, or the sandbox | |
| 11 | //! never pushed, the pull request is closed and an issue is opened for | |
| 12 | //! g1t to work on, started by g1t. That is the only time an agent is | |
| 13 | //! used. | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 14 | //! 4. Once none of the alerts it fixes is open (fixed on the default |
| 15 | //! branch, or dismissed), its pull request is closed with a comment | |
| 16 | //! saying why and its branch deleted (`resolved`). | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 17 | //! |
| 18 | //! Each step is written to the alerts' activity log. | |
| 19 | ||
| 20 | use std::collections::BTreeMap; | |
| 21 | ||
| 22 | use g1t_contracts::repos::RepoPath; | |
| 23 | use g1t_contracts::security::{BumpArgs, UpdateState, update_branch}; | |
| 24 | use g1t_contracts::time::rfc3339; | |
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 25 | use g1t_contracts::work::{IssueActionArgs, IssueReason, OpenPullArgs, Pull, PullActionArgs, PullDetail, PullStatus, Runtime, ViewArgs}; |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 26 | use g1t_contracts::{Outcome, User}; |
| 27 | use g1t_kit::now_ms; | |
| 28 | use g1t_scan::osv::{self, Severity}; | |
| 29 | use g1t_scan::version; | |
| Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in. | 30 | use serde_json::Value; |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 31 | use worker::Result; |
| 32 | ||
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 33 | use g1t_contracts::security::UPDATE_BRANCH_PREFIX; |
| 34 | use g1t_contracts::updates::{BumpPackage, IgnoreCondition, UpdatedDependency, VersionUpdateEntry, VersionUpdatesState}; | |
| 35 | use g1t_contracts::work::UpdatePullArgs; | |
| 36 | ||
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 37 | use crate::Security; |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 38 | use crate::config::{CommitMessage, Config, Entry, glob, matches}; |
| Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in. | 39 | use crate::deps::advisory_table; |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 40 | use crate::pull_text; |
| 41 | use crate::ranges; | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 42 | use crate::store::{Activity, RepoRow, UpdateRow, VulnRow}; |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 43 | use crate::update_store::NewPull; |
| 44 | use crate::version_updates::{Loaded, package_ecosystem}; | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 45 | |
| 46 | /// Security updates asked for per scan, most severe first. | |
| 47 | const MAX_NEW_UPDATES: usize = 8; | |
| 48 | /// A sandbox that has not pushed its branch after this long is taken to | |
| 49 | /// have failed. | |
| 50 | const STALLED_MS: u64 = 45 * 60 * 1000; | |
| 51 | /// A failed update is tried again after this long. | |
| 52 | const RETRY_FAILED_MS: u64 = 24 * 60 * 60 * 1000; | |
| 53 | ||
| 54 | /// What to do about a package's existing update, given the version it | |
| 55 | /// should now reach. | |
| 56 | #[derive(Debug, PartialEq, Eq)] | |
| 57 | pub enum Next { | |
| 58 | /// Leave it: in flight, done, or someone closed it. | |
| 59 | Wait, | |
| 60 | /// Ask for a new one. | |
| 61 | Request, | |
| 62 | } | |
| 63 | ||
| 64 | /// Whether a package with an update in `state` to `current`, last changed | |
| 65 | /// at `updated_at`, needs a new one to reach `target`. A higher target | |
| 66 | /// always does; the same one only when the last was superseded (it is | |
| 67 | /// vulnerable again) or failed long enough ago. | |
| 68 | pub fn next_step(state: UpdateState, current: &str, target: &str, updated_at: &str, retry_after: &str) -> Next { | |
| 69 | if version::compare(target, current) == std::cmp::Ordering::Greater { | |
| 70 | return Next::Request; | |
| 71 | } | |
| 72 | match state { | |
| 73 | UpdateState::Superseded => Next::Request, | |
| 74 | UpdateState::Failed if updated_at < retry_after => Next::Request, | |
| 75 | _ => Next::Wait, | |
| 76 | } | |
| 77 | } | |
| 78 | ||
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 79 | /// Why an update that never pushed failed, as its alert shows it. |
| 80 | pub fn cannot_raise(package: &str, target: &str) -> String { | |
| 81 | format!( | |
| 82 | "`{package}` could not be raised to {target} on its own: a package that depends on it may hold it back, or the upgrade needs code changes." | |
| 83 | ) | |
| 84 | } | |
| 85 | ||
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 86 | /// The pull request's title. |
| 87 | pub fn pull_title(package: &str, target: &str) -> String { | |
| 88 | format!("Upgrade {package} to {target}").chars().take(200).collect() | |
| 89 | } | |
| 90 | ||
| 91 | /// The pull request's body: what it fixes, where, and what happens if | |
| 92 | /// raising the version is not enough. | |
| 93 | pub fn pull_text(ecosystem: &str, package: &str, target: &str, vulns: &[&VulnRow]) -> String { | |
| 94 | let mut from: Vec<&str> = vulns.iter().map(|vuln| vuln.version.as_str()).collect(); | |
| 95 | from.sort(); | |
| 96 | from.dedup(); | |
| 97 | let mut lockfiles: Vec<&str> = vulns.iter().map(|vuln| vuln.manifest.as_str()).collect(); | |
| 98 | lockfiles.sort(); | |
| 99 | lockfiles.dedup(); | |
| 100 | let mut body = format!( | |
| 101 | "Upgrades `{package}` ({ecosystem}) from {} to **{target}**, which fixes these known vulnerabilities:\n\n", | |
| 102 | from.join(", ") | |
| 103 | ); | |
| 104 | body.push_str(&advisory_table(vulns)); | |
| 105 | body.push_str(&format!( | |
| 106 | "\nLockfiles changed: {}.\n\n\ | |
| 107 | Only the version changes. This pull request lands through this branch's required checks like any other. \ | |
| 108 | If they fail because code has to change, g1t closes it and puts g1t on an issue to make the change.\n\n\ | |
| 109 | ---\n_Opened by g1t's security updates. Turn them off for this project on its Security page._", | |
| 110 | lockfiles.iter().map(|path| format!("`{path}`")).collect::<Vec<_>>().join(", ") | |
| 111 | )); | |
| 112 | body | |
| 113 | } | |
| 114 | ||
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 115 | /// A package a `security-updates` group gathers. |
| 116 | pub struct GroupMember { | |
| 117 | /// OSV's name. | |
| 118 | pub ecosystem: String, | |
| 119 | pub package: String, | |
| 120 | /// The lowest vulnerable version found. | |
| 121 | pub from: String, | |
| 122 | pub target: String, | |
| 123 | /// The lockfiles that resolve a vulnerable version. | |
| 124 | pub manifests: Vec<String>, | |
| 125 | } | |
| 126 | ||
| 127 | fn lowest(vulns: &[&VulnRow]) -> String { | |
| 128 | vulns.iter().map(|vuln| vuln.version.clone()).min_by(|a, b| version::compare(a, b)).unwrap_or_default() | |
| 129 | } | |
| 130 | ||
| 131 | /// Whether one of the file's directories (`/web`, or a glob) holds the | |
| 132 | /// lockfile at `path`, from the root. | |
| 133 | fn covers(directories: &[String], path: &str) -> bool { | |
| 134 | let directory = format!("/{}", path.rsplit_once('/').map_or("", |(dir, _)| dir)); | |
| 135 | directories.iter().any(|wanted| if wanted.contains(['*', '?']) { glob(wanted, &directory) } else { *wanted == directory }) | |
| 136 | } | |
| 137 | ||
| 138 | /// The `updates` entry that speaks for a vulnerable package: its | |
| 139 | /// ecosystem, a directory holding one of its lockfiles, and no | |
| 140 | /// `target-branch` but the default branch (security updates always go to | |
| 141 | /// the default branch, and such an entry's options are for version | |
| 142 | /// updates only). | |
| 143 | pub fn security_entry<'a>(config: &'a Config, default_branch: &str, osv: &str, manifests: &[&str]) -> Option<&'a Entry> { | |
| 144 | let ecosystem = package_ecosystem(osv)?; | |
| 145 | config.updates.iter().find(|entry| { | |
| 146 | entry.ecosystem == ecosystem | |
| 147 | && entry.target_branch.as_deref().is_none_or(|branch| branch == default_branch) | |
| 148 | && manifests.iter().any(|path| covers(&entry.directories, path)) | |
| 149 | }) | |
| 150 | } | |
| 151 | ||
| 152 | /// Whether the file lets a security update raise `package` to `target`: | |
| 153 | /// not ignored (by name, or for these versions) in the file or by a | |
| 154 | /// comment, and named by `allow` when it names dependencies. | |
| 155 | pub fn security_allowed(entry: &Entry, comments: &[IgnoreCondition], package: &str, target: &str) -> bool { | |
| 156 | let ignored = entry.ignore.iter().filter(|rule| matches(&rule.dependency, package)).any(|rule| { | |
| 157 | (rule.versions.is_empty() && rule.update_types.is_empty()) || rule.versions.iter().any(|versions| ranges::ignored_by(versions, target)) | |
| 158 | }); | |
| 159 | let commented = comments.iter().filter(|c| c.ecosystem == entry.ecosystem && c.dependency.eq_ignore_ascii_case(package)).any(|c| { | |
| 160 | (c.versions.is_none() && c.update_type.is_none()) || c.versions.as_deref().is_some_and(|versions| ranges::ignored_by(versions, target)) | |
| 161 | }); | |
| 162 | let named: Vec<&str> = entry.allow.iter().filter_map(|rule| rule.dependency.as_deref()).collect(); | |
| 163 | let allowed = named.is_empty() || named.iter().any(|pattern| matches(pattern, package)); | |
| 164 | !ignored && !commented && allowed | |
| 165 | } | |
| 166 | ||
| 167 | /// The `security-updates` group that gathers `package`, if any. | |
| 168 | pub fn security_group(entry: &Entry, package: &str, from: &str, target: &str) -> Option<String> { | |
| 169 | let level = ranges::update_level(from, target); | |
| 170 | entry | |
| 171 | .groups | |
| 172 | .iter() | |
| 173 | .filter(|group| group.applies_to == "security-updates") | |
| 174 | .find(|group| { | |
| 175 | (group.patterns.is_empty() || group.patterns.iter().any(|pattern| matches(pattern, package))) | |
| 176 | && !group.exclude_patterns.iter().any(|pattern| matches(pattern, package)) | |
| 177 | && (group.update_types.is_empty() || group.update_types.iter().any(|wanted| wanted == level)) | |
| 178 | }) | |
| 179 | .map(|group| group.name.clone()) | |
| 180 | } | |
| 181 | ||
| 182 | /// The entry, as last read, that speaks for a package's security update. | |
| 183 | pub fn security_settings<'a>(state: &'a VersionUpdatesState, osv: &str, manifests: &[&str]) -> Option<&'a VersionUpdateEntry> { | |
| 184 | let ecosystem = package_ecosystem(osv)?; | |
| 185 | state.updates.iter().find(|entry| { | |
| 186 | entry.ecosystem == ecosystem && entry.target_branch.is_none() && manifests.iter().any(|path| covers(&entry.directories, path)) | |
| 187 | }) | |
| 188 | } | |
| 189 | ||
| 190 | /// An entry's `commit-message`, as last read. | |
| 191 | pub fn commit_message_of(entry: &VersionUpdateEntry) -> Option<CommitMessage> { | |
| 192 | let message = entry.options.get("commit-message")?; | |
| 193 | let text = |key: &str| message.get(key).and_then(Value::as_str).map(str::to_owned); | |
| 194 | Some(CommitMessage { prefix: text("prefix"), prefix_development: text("prefix-development"), scope: text("include").as_deref() == Some("scope") }) | |
| 195 | } | |
| 196 | ||
| 197 | /// A title with the file's commit message prefix, if it has one. | |
| 198 | fn prefixed(prefix: &str, plain: String) -> String { | |
| 199 | let text = if prefix.is_empty() { plain } else { format!("{prefix}{}{}", plain[..1].to_lowercase(), &plain[1..]) }; | |
| 200 | text.chars().take(200).collect() | |
| 201 | } | |
| 202 | ||
| 203 | /// A grouped security update's body. | |
| 204 | pub fn group_text(group: &str, members: &[GroupMember], vulns: &[&VulnRow], file: &str) -> String { | |
| 205 | let mut body = format!("Upgrades the {group} group's vulnerable packages to the versions that fix them:\n\n| Package | From | To |\n| --- | --- | --- |\n"); | |
| 206 | for member in members { | |
| 207 | body.push_str(&format!("| `{}` | {} | **{}** |\n", member.package, member.from, member.target)); | |
| 208 | } | |
| 209 | body.push_str("\nThe known vulnerabilities they fix:\n\n"); | |
| 210 | body.push_str(&advisory_table(vulns)); | |
| 211 | body.push_str(&format!( | |
| 212 | "\nOnly the versions change. This pull request lands through this branch's required checks like any other. \ | |
| 213 | If they fail because code has to change, g1t closes it and puts g1t on an issue to make the change.\n\n{}\n\n---\n\ | |
| 214 | _Opened by g1t's security updates, grouped as `{file}` asks. Turn them off for this project on its Security page._", | |
| 215 | pull_text::COMMANDS | |
| 216 | )); | |
| 217 | body | |
| 218 | } | |
| 219 | ||
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 220 | impl Security { |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 221 | pub(crate) fn path_of(repo: &RepoRow) -> RepoPath { |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 222 | RepoPath { namespace: repo.namespace.clone(), name: repo.name.clone() } |
| 223 | } | |
| 224 | ||
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 225 | pub(crate) async fn get_pull(&self, repo: &RepoRow, number: u32) -> Result<Option<Pull>> { |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 226 | let found: Outcome<PullDetail> = g1t_kit::call( |
| 227 | &self.work, | |
| 228 | "get_pull", | |
| 229 | &ViewArgs { repo: Self::path_of(repo), number, viewer: Some(User::system(&repo.namespace)), after_seq: 0 }, | |
| 230 | ) | |
| 231 | .await?; | |
| 232 | Ok(found.into_result().ok().map(|detail| detail.pull)) | |
| 233 | } | |
| 234 | ||
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 235 | /// Closes one of g1t's pull requests, saying why first. Returns |
| 236 | /// whether it closed. | |
| 237 | pub(crate) async fn close_with(&self, repo: &RepoRow, number: u32, why: String) -> Result<bool> { | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 238 | let system = User::system(&repo.namespace); |
| 239 | self.comment(&system, &Self::path_of(repo), number, why).await?; | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 240 | let closed: Outcome<Value> = g1t_kit::call( |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 241 | &self.work, |
| 242 | "close_pull", | |
| 243 | &PullActionArgs { | |
| 244 | actor: system, | |
| 245 | repo: Self::path_of(repo), | |
| 246 | number, | |
| 247 | summary: String::new(), | |
| 248 | keep_issue_open: false, | |
| 249 | ignore_checks: false, | |
| Merge rulesets: branch and tag rules, agent-first, enforced on push and merge | 250 | bypass_rules: false, |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 251 | }, |
| 252 | ) | |
| 253 | .await?; | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 254 | if let Outcome::Fail(refused) = &closed { |
| 255 | worker::console_error!("security: #{number} of {} not closed: {}", repo.repo_id, refused.message); | |
| 256 | } | |
| 257 | Ok(matches!(closed, Outcome::Ok(_))) | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 258 | } |
| 259 | ||
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 260 | /// Closes an issue g1t opened for an update before it stopped opening |
| 261 | /// them, saying why. One already closed is left as it is. | |
| 262 | pub(crate) async fn close_issue_with(&self, repo: &RepoRow, number: u32, why: String) -> Result<()> { | |
| 263 | let system = User::system(&repo.namespace); | |
| 264 | let path = Self::path_of(repo); | |
| 265 | let closed: Outcome<Value> = g1t_kit::call( | |
| 266 | &self.work, | |
| 267 | "close_issue", | |
| 268 | &IssueActionArgs { actor: system.clone(), repo: path.clone(), number, reason: Some(IssueReason::NotPlanned) }, | |
| 269 | ) | |
| 270 | .await?; | |
| 271 | if matches!(closed, Outcome::Ok(_)) { | |
| 272 | self.comment(&system, &path, number, why).await?; | |
| 273 | } | |
| 274 | Ok(()) | |
| 275 | } | |
| 276 | ||
| 277 | /// Records `action` on every open alert of an update's package, except | |
| 278 | /// on an alert whose latest entry already says the same. | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 279 | async fn note(&self, row: &UpdateRow, action: &str, actor: Option<&str>, number: Option<u32>, comment: Option<&str>) -> Result<()> { |
| 280 | let ids = self.store.open_ids(&row.repo_id, &row.ecosystem, &row.package).await?; | |
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 281 | let latest = self.store.latest_activity(&row.repo_id, &ids).await?; |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 282 | let activity: Vec<Activity> = ids |
| 283 | .iter() | |
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 284 | .filter(|id| latest.get(*id).is_none_or(|(last, said)| last != action || said.as_deref() != comment)) |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 285 | .map(|id| Activity { alert_id: id, action, actor, reason: None, comment, number }) |
| 286 | .collect(); | |
| 287 | self.store.record(&row.repo_id, &activity).await | |
| 288 | } | |
| 289 | ||
| 290 | /// After a dependency scan: asks for an update for each vulnerable | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 291 | /// package with a fix (when `enabled`). Those no longer needed were |
| 292 | /// closed already (`resolved`). | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 293 | /// |
| 294 | /// The dependency update file, when there is one, applies as it does to | |
| 295 | /// version updates: `ignore` and `allow` by name, people's `@g1t ignore` | |
| 296 | /// comments, `groups` with `applies-to: security-updates` (one pull | |
| 297 | /// request for each group), and `assignees`, `reviewers` and | |
| 298 | /// `commit-message`. `open-pull-requests-limit` does not apply. | |
| 299 | pub async fn security_updates(&self, repo: &RepoRow, enabled: bool, rules: Option<&Loaded>) -> Result<()> { | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 300 | let open = self.store.open_vulnerabilities(&repo.repo_id).await?; |
| 301 | let mut by_package: BTreeMap<(String, String), Vec<&VulnRow>> = BTreeMap::new(); | |
| 302 | for vuln in &open { | |
| 303 | by_package.entry((vuln.ecosystem.clone(), vuln.package.clone())).or_default().push(vuln); | |
| 304 | } | |
| 305 | if !enabled { | |
| 306 | return Ok(()); | |
| 307 | } | |
| 308 | let mut groups: Vec<((String, String), Vec<&VulnRow>)> = by_package | |
| 309 | .into_iter() | |
| 310 | .filter(|(_, vulns)| vulns.iter().any(|vuln| vuln.fixed_version.is_some())) | |
| 311 | .collect(); | |
| 312 | groups.sort_by_key(|(_, vulns)| std::cmp::Reverse(vulns.iter().map(|v| Severity::parse(&v.severity)).max())); | |
| 313 | let retry_after = rfc3339(now_ms().saturating_sub(RETRY_FAILED_MS)); | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 314 | let comments = self.store.ignores(&repo.repo_id).await?; |
| 315 | let mut grouped: BTreeMap<(String, String), Vec<GroupMember>> = BTreeMap::new(); | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 316 | let mut asked = 0; |
| 317 | for ((ecosystem, package), vulns) in groups { | |
| 318 | if asked >= MAX_NEW_UPDATES { | |
| 319 | break; | |
| 320 | } | |
| 321 | let Some(target) = osv::upgrade_target(vulns.iter().filter_map(|v| v.fixed_version.as_deref())) else { | |
| 322 | continue; | |
| 323 | }; | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 324 | let manifests: Vec<&str> = vulns.iter().map(|vuln| vuln.manifest.as_str()).collect(); |
| 325 | if let Some(entry) = rules.and_then(|loaded| security_entry(&loaded.config, &loaded.default_branch, &ecosystem, &manifests)) { | |
| 326 | if !security_allowed(entry, &comments, &package, &target) { | |
| 327 | continue; | |
| 328 | } | |
| 329 | let from = lowest(&vulns); | |
| 330 | if let Some(group) = security_group(entry, &package, &from, &target) { | |
| 331 | let key = (entry.id(), group); | |
| 332 | if !grouped.contains_key(&key) { | |
| 333 | asked += 1; | |
| 334 | } | |
| 335 | let manifests = manifests.iter().map(|path| (*path).to_owned()).collect(); | |
| 336 | grouped.entry(key).or_default().push(GroupMember { ecosystem: ecosystem.clone(), package: package.clone(), from, target, manifests }); | |
| 337 | continue; | |
| 338 | } | |
| 339 | } | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 340 | match self.store.update(&repo.repo_id, &ecosystem, &package).await? { |
| 341 | Some(row) => { | |
| 342 | if next_step(row.state(), &row.target, &target, &row.updated_at, &retry_after) == Next::Wait { | |
| 343 | continue; | |
| 344 | } | |
| 345 | } | |
| 346 | None => { | |
| 347 | // An upgrade issue from before security updates, still | |
| 348 | // open, is left to the agent on it. | |
| 349 | if let Some(existing) = self.store.upgrade(&repo.repo_id, &ecosystem, &package).await? | |
| 350 | && self | |
| 351 | .issue(&User::system(&repo.namespace), &Self::path_of(repo), existing.number as u32) | |
| 352 | .await? | |
| 353 | .is_some_and(|issue| issue.state == g1t_contracts::work::State::Open) | |
| 354 | { | |
| 355 | continue; | |
| 356 | } | |
| 357 | } | |
| 358 | } | |
| 359 | asked += 1; | |
| 360 | self.request(repo, &ecosystem, &package, &target, &vulns).await?; | |
| 361 | } | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 362 | for ((entry_id, group), members) in grouped { |
| 363 | let Some(loaded) = rules else { continue }; | |
| 364 | let Some(entry) = loaded.config.updates.iter().find(|entry| entry.id() == entry_id) else { continue }; | |
| 365 | self.request_group(repo, loaded, entry, &group, members, &open).await?; | |
| 366 | } | |
| 367 | Ok(()) | |
| 368 | } | |
| 369 | ||
| 370 | /// Asks for one pull request for a `security-updates` group's | |
| 371 | /// packages, unless one for the same versions is under way or was | |
| 372 | /// closed. | |
| 373 | async fn request_group(&self, repo: &RepoRow, loaded: &Loaded, entry: &Entry, group: &str, members: Vec<GroupMember>, open: &[VulnRow]) -> Result<()> { | |
| 374 | let subject = format!("security:{group}"); | |
| 375 | let mut signature: Vec<String> = members.iter().map(|m| format!("{}@{}", m.package, m.target)).collect(); | |
| 376 | signature.sort(); | |
| 377 | let signature = signature.join(","); | |
| 378 | let existing = self.store.update_pulls(&repo.repo_id).await?; | |
| 379 | let known = |row: &&crate::update_store::PullRow| row.kind == "security" && row.subject == subject && row.signature == signature; | |
| 380 | if existing.iter().filter(known).any(|row| row.state().in_progress() || row.state() == UpdateState::Closed) { | |
| 381 | return Ok(()); | |
| 382 | } | |
| 383 | let branch = format!("{UPDATE_BRANCH_PREFIX}{}-{}", group.to_lowercase().replace('|', "-"), pull_text::digest(&signature)); | |
| 384 | let vulns: Vec<&VulnRow> = open.iter().filter(|vuln| members.iter().any(|m| m.ecosystem == vuln.ecosystem && m.package == vuln.package)).collect(); | |
| 385 | let count = if members.len() == 1 { "1 security update".to_owned() } else { format!("{} security updates", members.len()) }; | |
| 386 | let title = prefixed(&pull_text::prefix(entry.commit_message.as_ref(), false), format!("Bump the {group} group with {count}")); | |
| 387 | let body = group_text(group, &members, &vulns, &loaded.file); | |
| 388 | let mut lockfiles: Vec<String> = members.iter().flat_map(|m| m.manifests.clone()).collect(); | |
| 389 | lockfiles.sort(); | |
| 390 | lockfiles.dedup(); | |
| 391 | let packages: Vec<BumpPackage> = members.iter().map(|m| BumpPackage { package: m.package.clone(), version: m.target.clone() }).collect(); | |
| 392 | let bump = BumpArgs { | |
| 393 | repo: Self::path_of(repo), | |
| 394 | ecosystem: members[0].ecosystem.clone(), | |
| 395 | package: packages[0].package.clone(), | |
| 396 | version: packages[0].version.clone(), | |
| 397 | lockfiles, | |
| 398 | branch: branch.clone(), | |
| 399 | message: title.clone(), | |
| 400 | kind: None, | |
| 401 | packages, | |
| 402 | strategy: None, | |
| 403 | force: existing.iter().any(|row| row.branch == branch && row.state().in_progress()), | |
| 404 | registries: Vec::new(), | |
| 405 | base: None, | |
| 406 | }; | |
| 407 | let dependencies: Vec<UpdatedDependency> = members | |
| 408 | .iter() | |
| 409 | .map(|m| UpdatedDependency { | |
| 410 | name: m.package.clone(), | |
| 411 | from: m.from.clone(), | |
| 412 | to: m.target.clone(), | |
| 413 | directory: m.manifests.first().map(|path| format!("/{}", path.rsplit_once('/').map_or("", |(dir, _)| dir))).unwrap_or_else(|| "/".to_owned()), | |
| 414 | dependency_type: String::new(), | |
| 415 | update_type: ranges::update_type(&m.from, &m.target), | |
| 416 | }) | |
| 417 | .collect(); | |
| 418 | let people = |names: &[String]| -> Vec<String> { names.iter().filter(|name| !name.contains('/')).cloned().collect() }; | |
| 419 | let id = self | |
| 420 | .store | |
| 421 | .add_update_pull(&NewPull { | |
| 422 | repo_id: &repo.repo_id, | |
| 423 | kind: "security", | |
| 424 | entry: &entry.id(), | |
| 425 | ecosystem: &entry.ecosystem, | |
| 426 | subject: &subject, | |
| 427 | signature: &signature, | |
| 428 | group: Some(group), | |
| 429 | branch: &branch, | |
| 430 | title: &title, | |
| 431 | body: &body, | |
| 432 | dependencies: &dependencies, | |
| 433 | bump: &bump, | |
| 434 | assignees: &people(&entry.assignees), | |
| 435 | reviewers: &people(&entry.reviewers), | |
| 436 | }) | |
| 437 | .await?; | |
| 438 | for member in &members { | |
| 439 | self.store.request_update(&repo.repo_id, &member.ecosystem, &member.package, &member.target, &branch).await?; | |
| 440 | } | |
| 441 | let started: std::result::Result<(), String> = match g1t_kit::call::<_, Outcome<bool>>(&self.runner, "bump", &bump).await { | |
| 442 | Ok(Outcome::Ok(_)) => Ok(()), | |
| 443 | Ok(Outcome::Fail(refused)) => Err(refused.message), | |
| 444 | Err(error) => Err(format!("the runner could not be reached: {error}")), | |
| 445 | }; | |
| 446 | for member in &members { | |
| 447 | let Some(row) = self.store.update(&repo.repo_id, &member.ecosystem, &member.package).await? else { continue }; | |
| 448 | match &started { | |
| 449 | Ok(()) => self.note(&row, "update_requested", Some(g1t_contracts::system::USERNAME), None, None).await?, | |
| 450 | Err(reason) => { | |
| 451 | let error = format!("g1t could not start the security update: {reason}"); | |
| 452 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&error)).await?; | |
| 453 | self.note(&row, "update_failed", Some(g1t_contracts::system::USERNAME), None, Some(&error)).await?; | |
| 454 | } | |
| 455 | } | |
| 456 | } | |
| 457 | if let Err(reason) = started { | |
| 458 | self.store.set_update_pull(&id, UpdateState::Failed, None, None, Some(&format!("g1t could not start the security update: {reason}"))).await?; | |
| 459 | } | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 460 | Ok(()) |
| 461 | } | |
| 462 | ||
| 463 | /// Asks the runner to make the change on its branch. | |
| 464 | async fn request(&self, repo: &RepoRow, ecosystem: &str, package: &str, target: &str, vulns: &[&VulnRow]) -> Result<()> { | |
| 465 | let branch = update_branch(package, target); | |
| 466 | let mut lockfiles: Vec<String> = vulns.iter().map(|vuln| vuln.manifest.clone()).collect(); | |
| 467 | lockfiles.sort(); | |
| 468 | lockfiles.dedup(); | |
| 469 | let args = BumpArgs { | |
| 470 | repo: Self::path_of(repo), | |
| 471 | ecosystem: ecosystem.to_owned(), | |
| 472 | package: package.to_owned(), | |
| 473 | version: target.to_owned(), | |
| 474 | lockfiles, | |
| 475 | branch: branch.clone(), | |
| 476 | message: format!("Upgrade {package} to {target}"), | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 477 | kind: None, |
| 478 | packages: Vec::new(), | |
| 479 | strategy: None, | |
| 480 | force: false, | |
| 481 | registries: Vec::new(), | |
| 482 | base: None, | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 483 | }; |
| 484 | let started: std::result::Result<(), String> = match g1t_kit::call::<_, Outcome<bool>>(&self.runner, "bump", &args).await { | |
| 485 | Ok(Outcome::Ok(_)) => Ok(()), | |
| 486 | Ok(Outcome::Fail(refused)) => Err(refused.message), | |
| 487 | Err(error) => Err(format!("the runner could not be reached: {error}")), | |
| 488 | }; | |
| 489 | self.store.request_update(&repo.repo_id, ecosystem, package, target, &branch).await?; | |
| 490 | let Some(row) = self.store.update(&repo.repo_id, ecosystem, package).await? else { | |
| 491 | return Ok(()); | |
| 492 | }; | |
| 493 | match started { | |
| 494 | Ok(()) => self.note(&row, "update_requested", Some(g1t_contracts::system::USERNAME), None, None).await, | |
| 495 | Err(reason) => { | |
| 496 | let error = format!("g1t could not start the security update: {reason}"); | |
| 497 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&error)).await?; | |
| 498 | self.note(&row, "update_failed", Some(g1t_contracts::system::USERNAME), None, Some(&error)).await | |
| 499 | } | |
| 500 | } | |
| 501 | } | |
| 502 | ||
| 503 | /// A security update's branch was pushed: its pull request opens, and | |
| 504 | /// an older one for the same package is closed as superseded. | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 505 | pub async fn update_pushed(&self, repo_id: &str, branch: &str, after: &str) -> Result<()> { |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 506 | let Some(row) = self.store.update_by_branch(repo_id, branch).await? else { |
| 507 | return Ok(()); | |
| 508 | }; | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 509 | if row.state() == UpdateState::Superseded && row.pull().is_none() { |
| 510 | // No longer needed by the time its sandbox pushed: the branch goes. | |
| 511 | return self.drop_pushed(repo_id, branch, after).await; | |
| 512 | } | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 513 | if row.state() != UpdateState::Requested { |
| 514 | return Ok(()); | |
| 515 | } | |
| 516 | let Some(repo) = self.store.repo(repo_id).await? else { | |
| 517 | return Ok(()); | |
| 518 | }; | |
| 519 | let open = self.store.open_vulnerabilities(repo_id).await?; | |
| 520 | let vulns: Vec<&VulnRow> = open | |
| 521 | .iter() | |
| 522 | .filter(|vuln| vuln.ecosystem == row.ecosystem && vuln.package == row.package) | |
| 523 | .collect(); | |
| 524 | let system = User::system(&repo.namespace); | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 525 | // What the dependency update file says for this package's directory. |
| 526 | let state = repo.version_updates(); | |
| 527 | let manifests: Vec<&str> = vulns.iter().map(|vuln| vuln.manifest.as_str()).collect(); | |
| 528 | let settings = security_settings(&state, &row.ecosystem, &manifests); | |
| 529 | let prefix = settings.map(|entry| pull_text::prefix(commit_message_of(entry).as_ref(), false)).unwrap_or_default(); | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 530 | let opened: Outcome<Pull> = g1t_kit::call( |
| 531 | &self.work, | |
| 532 | "open_pull", | |
| 533 | &OpenPullArgs { | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 534 | actor: system.clone(), |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 535 | repo: Self::path_of(&repo), |
| 536 | issue: None, | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 537 | title: prefixed(&prefix, pull_title(&row.package, &row.target)), |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 538 | body: pull_text(&row.ecosystem, &row.package, &row.target, &vulns), |
| 539 | branch: Some(branch.to_owned()), | |
| 540 | agent: String::new(), | |
| 541 | runtime: Runtime::External, | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 542 | // Security updates are for the default branch. |
| 543 | base: None, | |
| The API opens a pull request as a draft with "draft": true | 544 | draft: false, |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 545 | }, |
| 546 | ) | |
| 547 | .await?; | |
| 548 | let pull = match opened { | |
| 549 | Outcome::Ok(pull) => pull, | |
| 550 | Outcome::Fail(refused) => { | |
| 551 | let error = format!("The pull request could not be opened: {}", refused.message); | |
| 552 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&error)).await?; | |
| 553 | return self.note(&row, "update_failed", Some(g1t_contracts::system::USERNAME), None, Some(&error)).await; | |
| 554 | } | |
| 555 | }; | |
| Merge update PRs close themselves: g1t closes its security and version updates once they are no longer needed, and deletes their branches | 556 | if let Some(older) = row.pull().filter(|older| *older != pull.number) |
| 557 | && let Some(found) = self.get_pull(&repo, older).await? | |
| 558 | && matches!(found.status, PullStatus::Open | PullStatus::Draft) | |
| 559 | && self | |
| 560 | .close_with(&repo, older, format!("Closed: superseded by #{}, which upgrades `{}` to {}.", pull.number, row.package, row.target)) | |
| 561 | .await? | |
| 562 | && found.branch.as_deref() != Some(branch) | |
| 563 | { | |
| 564 | self.delete_pull_branch(&repo, &Pull { status: PullStatus::Closed, ..found }).await; | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 565 | } |
| 566 | self.store.set_update(&row, UpdateState::Open, Some(pull.number), None, None).await?; | |
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 567 | // Labelled as the entry for its directory says, or `dependencies` |
| 568 | // and its ecosystem's label; assigned and in a milestone as it says. | |
| 569 | let ecosystem = package_ecosystem(&row.ecosystem).unwrap_or(row.ecosystem.as_str()); | |
| 570 | let labels = crate::config::update_labels(settings.and_then(|entry| entry.labels.as_deref()), ecosystem); | |
| 571 | let people = |names: &[String]| -> Vec<String> { names.iter().filter(|name| !name.contains('/')).cloned().collect() }; | |
| 572 | let (assignees, reviewers) = settings.map(|entry| (people(&entry.assignees), people(&entry.reviewers))).unwrap_or_default(); | |
| 573 | if !assignees.is_empty() || !reviewers.is_empty() || !labels.is_empty() { | |
| 574 | let _: Outcome<Value> = g1t_kit::call( | |
| 575 | &self.work, | |
| 576 | "update_pull", | |
| 577 | &UpdatePullArgs { | |
| 578 | actor: system.clone(), | |
| 579 | repo: Self::path_of(&repo), | |
| 580 | number: pull.number, | |
| 581 | assignees: (!assignees.is_empty()).then_some(assignees), | |
| 582 | reviewers: (!reviewers.is_empty()).then_some(reviewers), | |
| 583 | labels: (!labels.is_empty()).then_some(labels), | |
| 584 | milestone: None, | |
| 585 | base: None, | |
| 586 | }, | |
| 587 | ) | |
| 588 | .await?; | |
| 589 | } | |
| 590 | if let Some(milestone) = settings.and_then(|entry| entry.milestone) { | |
| 591 | let _: Outcome<Value> = g1t_kit::call( | |
| 592 | &self.work, | |
| 593 | "update_pull", | |
| 594 | &UpdatePullArgs { | |
| 595 | actor: system, | |
| 596 | repo: Self::path_of(&repo), | |
| 597 | number: pull.number, | |
| 598 | assignees: None, | |
| 599 | reviewers: None, | |
| 600 | labels: None, | |
| 601 | milestone: Some(milestone), | |
| 602 | base: None, | |
| 603 | }, | |
| 604 | ) | |
| 605 | .await?; | |
| 606 | } | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 607 | self.note(&row, "update_opened", Some(g1t_contracts::system::USERNAME), Some(pull.number), None).await |
| 608 | } | |
| 609 | ||
| Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in. | 610 | /// A pull request merged or closed: if it is a security update's, where |
| 611 | /// the update stands now. One whose checks fail stays open for a person | |
| 612 | /// to decide on. | |
| 613 | pub async fn update_pull_event(&self, kind: &str, repo_id: &str, number: u32, actor: Option<&str>) -> Result<()> { | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 614 | let Some(row) = self.store.update_by_pull(repo_id, number).await? else { |
| 615 | return Ok(()); | |
| 616 | }; | |
| 617 | if row.state() != UpdateState::Open { | |
| 618 | return Ok(()); | |
| 619 | } | |
| 620 | match kind { | |
| 621 | "pull.merged" => { | |
| 622 | self.store.set_update(&row, UpdateState::Merged, Some(number), None, None).await?; | |
| 623 | self.note(&row, "update_merged", actor, Some(number), None).await | |
| 624 | } | |
| 625 | "pull.closed" => { | |
| 626 | self.store.set_update(&row, UpdateState::Closed, Some(number), None, None).await?; | |
| 627 | self.note(&row, "update_closed", actor, Some(number), None).await | |
| 628 | } | |
| 629 | _ => Ok(()), | |
| 630 | } | |
| 631 | } | |
| 632 | ||
| 633 | /// Updates whose sandbox never pushed: raising the version did not | |
| Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in. | 634 | /// work on its own, so the update fails with why, shown on the alert, |
| 635 | /// and is tried again later. Nothing is opened for it: a person can ask | |
| 636 | /// g1t to fix the alert. On a repository that takes no updates now | |
| 637 | /// (security updates off, archived, deleted, or a mirror that is not | |
| 638 | /// taken over, whose pushes are refused) the reason is that instead. | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 639 | pub async fn stalled_updates(&self) -> Result<()> { |
| 640 | let before = rfc3339(now_ms().saturating_sub(STALLED_MS)); | |
| 641 | for row in self.store.stalled_updates(&before, 10).await? { | |
| 642 | let Some(repo) = self.store.repo(&row.repo_id).await? else { continue }; | |
| A desktop release goes out with what can be built. Linux is built on g1t's machines in every release; Windows and macOS each wait on a repository variable: a machine registered for them, or for Windows a cross build on Linux once aka.ms and download.visualstudio.microsoft.com are workflow-only domains, which is where the first release stopped. A platform with no machine is skipped and the download page says it is coming, with a word for the visitor whose platform this release lacks. The guide and CONTRIBUTING say so. | 643 | let waits = if repo.upkeep == 0 { Some("The version could not be raised in a sandbox.".to_owned()) } else { self.takes_updates(&repo).await?.err() }; |
| 644 | if let Some(why) = waits { | |
| 645 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&why)).await?; | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 646 | continue; |
| 647 | } | |
| Updates that g1t opened a "needs code changes" issue for are put where updates go now: the sweep closes the issue as not planned, saying where it is tracked, and marks the update failed, so a security update shows why on its alert and is tried again, and a version update opens again as a pull request with its checks. An alert's history no longer repeats an entry that says the same as its latest one, so an update that fails the same way each day adds it once. | 648 | let why = cannot_raise(&row.package, &row.target); |
| 649 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&why)).await?; | |
| 650 | self.note(&row, "update_failed", Some(g1t_contracts::system::USERNAME), None, Some(&why)).await?; | |
| 651 | } | |
| 652 | // Left from when g1t opened an issue for an upgrade it could not | |
| 653 | // make: the issue closes, and the update fails as one would now. | |
| 654 | for row in self.store.needs_code_updates(10).await? { | |
| 655 | let why = cannot_raise(&row.package, &row.target); | |
| 656 | if let (Some(repo), Some(number)) = (self.store.repo(&row.repo_id).await?, row.issue()) { | |
| 657 | let path = Self::path_of(&repo); | |
| 658 | let alerts = format!("/{}/{}/security/vulnerabilities", path.namespace, path.name); | |
| 659 | self.close_issue_with(&repo, number, format!( | |
| 660 | "g1t no longer opens issues for upgrades it cannot make on its own. This one is on [its alert]({alerts}), \ | |
| 661 | which says why and is tried again later. To have an agent make the change, use **Fix with g1t** there." | |
| 662 | )) | |
| 663 | .await?; | |
| 664 | } | |
| Security and version updates open no issues on their own: an update whose version cannot be raised fails with why on its alert and is tried again a day later, and one whose required checks fail stays open for a person to decide on; an agent takes one only when a person asks with Fix with g1t. That issue now names the lockfiles the package is locked in and the commands that show it is done, which were left empty ("Locked in: .") because the lockfiles were never passed in. | 665 | self.store.set_update(&row, UpdateState::Failed, row.pull(), None, Some(&why)).await?; |
| 666 | self.note(&row, "update_failed", Some(g1t_contracts::system::USERNAME), None, Some(&why)).await?; | |
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 667 | } |
| 668 | Ok(()) | |
| 669 | } | |
| 670 | } | |
| 671 | ||
| 672 | #[cfg(test)] | |
| 673 | mod tests { | |
| 674 | use super::*; | |
| 675 | ||
| Teams and CODEOWNERS, labels and milestones, dependency updates, the security suite, and a clearer top bar | 676 | fn rules(extra: &str) -> Config { |
| 677 | let source = format!( | |
| 678 | "version: 2\nupdates:\n - package-ecosystem: npm\n directories: [\"/\", \"/apps/*\"]\n schedule: {{interval: weekly}}\n{extra} - package-ecosystem: npm\n directory: /legacy\n target-branch: develop\n schedule: {{interval: weekly}}\n" | |
| 679 | ); | |
| 680 | let found = crate::config::read(&source); | |
| 681 | assert!(found.problems.is_empty(), "{:?}", found.problems); | |
| 682 | found.config | |
| 683 | } | |
| 684 | ||
| 685 | #[test] | |
| 686 | fn security_updates_follow_the_file() { | |
| 687 | let config = rules( | |
| 688 | " ignore:\n - dependency-name: left-pad\n - dependency-name: react\n versions: [\">=19\"]\n groups:\n fixes:\n applies-to: security-updates\n patterns: [\"@babel/*\"]\n update-types: [patch, minor]\n minor:\n patterns: [\"*\"]\n", | |
| 689 | ); | |
| 690 | let entry = security_entry(&config, "main", "npm", &["apps/web/package-lock.json"]).unwrap(); | |
| 691 | assert_eq!(entry.id(), "npm:/,/apps/*"); | |
| 692 | // An entry for another branch never speaks for security updates. | |
| 693 | assert!(security_entry(&config, "main", "npm", &["legacy/package-lock.json"]).is_none()); | |
| 694 | assert!(security_entry(&config, "main", "crates.io", &["Cargo.lock"]).is_none()); | |
| 695 | assert!(!security_allowed(entry, &[], "left-pad", "1.3.0")); | |
| 696 | assert!(!security_allowed(entry, &[], "react", "19.0.1")); | |
| 697 | assert!(security_allowed(entry, &[], "react", "18.3.1")); | |
| 698 | let comment = IgnoreCondition { ecosystem: "npm".into(), dependency: "Lodash".into(), versions: None, update_type: None, by: "ana".into(), pull: None, at: String::new() }; | |
| 699 | assert!(!security_allowed(entry, &[comment], "lodash", "4.17.21")); | |
| 700 | assert_eq!(security_group(entry, "@babel/core", "7.0.0", "7.24.1").as_deref(), Some("fixes")); | |
| 701 | // A major bump is outside the group's update types, and only security groups count. | |
| 702 | assert_eq!(security_group(entry, "@babel/core", "6.0.0", "7.24.1"), None); | |
| 703 | assert_eq!(security_group(entry, "lodash", "4.17.20", "4.17.21"), None); | |
| 704 | let named = rules(" allow:\n - dependency-name: \"lodash\"\n"); | |
| 705 | let entry = security_entry(&named, "main", "npm", &["package-lock.json"]).unwrap(); | |
| 706 | assert!(security_allowed(entry, &[], "lodash", "4.17.21") && !security_allowed(entry, &[], "minimist", "1.2.6")); | |
| 707 | } | |
| 708 | ||
| 709 | #[test] | |
| 710 | fn grouped_security_updates_say_what_they_fix() { | |
| 711 | let vuln = row("4.17.20", "4.17.21"); | |
| 712 | let members = vec![GroupMember { ecosystem: "npm".into(), package: "lodash".into(), from: "4.17.20".into(), target: "4.17.21".into(), manifests: vec!["package-lock.json".into()] }]; | |
| 713 | let body = group_text("fixes", &members, &[&vuln], ".github/dependabot.yml"); | |
| 714 | assert!(body.starts_with("Upgrades the fixes group's vulnerable packages")); | |
| 715 | assert!(body.contains("| `lodash` | 4.17.20 | **4.17.21** |")); | |
| 716 | assert!(body.contains("GHSA-35jh-r3h4-6jhm") && body.contains("`@g1t rebase`")); | |
| 717 | assert!(body.ends_with("on its Security page._")); | |
| 718 | assert_eq!(prefixed("build(deps): ", "Upgrade lodash to 4.17.21".into()), "build(deps): upgrade lodash to 4.17.21"); | |
| 719 | assert_eq!(prefixed("", "Upgrade lodash to 4.17.21".into()), "Upgrade lodash to 4.17.21"); | |
| 720 | } | |
| 721 | ||
| Stripe's webhook secret is a Worker secret, STRIPE_WEBHOOK_SECRET, from a destination made in Stripe's dashboard | 722 | fn row(version: &str, fixed: &str) -> VulnRow { |
| 723 | VulnRow { | |
| 724 | id: "vul_1".into(), | |
| 725 | repo_id: "rep_1".into(), | |
| 726 | ecosystem: "npm".into(), | |
| 727 | package: "lodash".into(), | |
| 728 | version: version.into(), | |
| 729 | manifest: "web/package-lock.json".into(), | |
| 730 | osv_id: "GHSA-35jh-r3h4-6jhm".into(), | |
| 731 | advisory: "GHSA-35jh-r3h4-6jhm".into(), | |
| 732 | summary: "Command Injection in lodash".into(), | |
| 733 | severity: "high".into(), | |
| 734 | fixed_version: Some(fixed.into()), | |
| 735 | status: "open".into(), | |
| 736 | found_at: "2026-10-04T00:00:00Z".into(), | |
| 737 | fixed_at: None, | |
| 738 | number: None, | |
| 739 | dismiss_reason: None, | |
| 740 | dismiss_comment: None, | |
| 741 | dismissed_by: None, | |
| 742 | dismissed_at: None, | |
| 743 | } | |
| 744 | } | |
| 745 | ||
| 746 | #[test] | |
| 747 | fn a_newer_fix_asks_again_and_the_same_one_waits() { | |
| 748 | let retry = "2026-10-05T00:00:00Z"; | |
| 749 | let at = "2026-10-06T00:00:00Z"; | |
| 750 | assert_eq!(next_step(UpdateState::Open, "4.17.20", "4.17.21", at, retry), Next::Request); | |
| 751 | assert_eq!(next_step(UpdateState::Open, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 752 | assert_eq!(next_step(UpdateState::Requested, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 753 | assert_eq!(next_step(UpdateState::Merged, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 754 | // A person closed it: left alone until a newer fix. | |
| 755 | assert_eq!(next_step(UpdateState::Closed, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 756 | assert_eq!(next_step(UpdateState::Closed, "4.17.21", "4.17.22", at, retry), Next::Request); | |
| 757 | // Vulnerable again after it was no longer needed. | |
| 758 | assert_eq!(next_step(UpdateState::Superseded, "4.17.21", "4.17.21", at, retry), Next::Request); | |
| 759 | // Failed: tried again a day later. | |
| 760 | assert_eq!(next_step(UpdateState::Failed, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 761 | assert_eq!(next_step(UpdateState::Failed, "4.17.21", "4.17.21", "2026-10-04T00:00:00Z", retry), Next::Request); | |
| 762 | assert_eq!(next_step(UpdateState::NeedsCode, "4.17.21", "4.17.21", at, retry), Next::Wait); | |
| 763 | } | |
| 764 | ||
| 765 | #[test] | |
| 766 | fn the_pull_request_says_what_it_fixes_and_where() { | |
| 767 | let a = row("4.17.20", "4.17.21"); | |
| 768 | let mut b = row("4.17.19", "4.17.21"); | |
| 769 | b.manifest = "package-lock.json".into(); | |
| 770 | let body = pull_text("npm", "lodash", "4.17.21", &[&a, &b]); | |
| 771 | assert!(body.starts_with("Upgrades `lodash` (npm) from 4.17.19, 4.17.20 to **4.17.21**")); | |
| 772 | assert!(body.contains("[GHSA-35jh-r3h4-6jhm](https://osv.dev/vulnerability/GHSA-35jh-r3h4-6jhm)")); | |
| 773 | assert!(body.contains("Lockfiles changed: `package-lock.json`, `web/package-lock.json`.")); | |
| 774 | assert!(body.contains("required checks")); | |
| 775 | assert_eq!(pull_title("lodash", "4.17.21"), "Upgrade lodash to 4.17.21"); | |
| 776 | assert_eq!(update_branch("@babel/core", "7.24.1"), "g1t/security/babel-core-7.24.1"); | |
| 777 | assert_eq!(update_branch("golang.org/x/net", "v0.23.0"), "g1t/security/golang.org-x-net-v0.23.0"); | |
| 778 | } | |
| 779 | } |
This file's history is long; its oldest lines are credited to the oldest commit read.