Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 1 | //! `g1t-runner` itself, kept in the app's data folder and run as a child |
| 2 | //! of the app: for the Docker and On this computer backends, and for the | |
| 3 | //! sandbox launcher, which is the same program. | |
| 4 | //! | |
| 5 | //! The app installs and updates it from the server's releases: the | |
| 6 | //! newest one's manifest is `<server>/downloads/runner/latest.json`, which | |
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 7 | //! names each platform's file and its SHA-256, and `latest.json.sig` is |
| 8 | //! its Ed25519 signature, made with g1t's release key | |
| 9 | //! (scripts/runner-release.mjs). The app trusts a manifest only when that | |
| 10 | //! signature checks out against the key built into it, as the runner does | |
| 11 | //! with its own updates (crates/runner/src/selfhosted/update.rs), and | |
| 12 | //! throws away a download whose SHA-256 is not the manifest's. The | |
| 13 | //! runner's own updating is turned off (`G1T_RUNNER_NO_AUTO_UPDATE`), so | |
| 14 | //! only the app replaces it, and only while it runs nothing. | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 15 | |
| 16 | use std::collections::BTreeMap; | |
| 17 | use std::path::{Path, PathBuf}; | |
| 18 | use std::process::{Child, Stdio}; | |
| 19 | use std::time::Duration; | |
| 20 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 21 | use base64::Engine; |
| 22 | use base64::engine::general_purpose::STANDARD; | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 23 | use serde::Deserialize; |
| 24 | use sha2::{Digest, Sha256}; | |
| 25 | use url::Url; | |
| 26 | ||
| 27 | use super::backend::quiet; | |
| 28 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 29 | /// g1t's release key, as base64 of its 32 bytes: the public half of the |
| 30 | /// key that signs every runner release (the repository variable | |
| 31 | /// RUNNER_RELEASE_PUBLIC_KEY, which the runner's own builds take as | |
| 32 | /// G1T_RUNNER_RELEASE_KEY). It is public, so it is kept here, where every | |
| 33 | /// build has it, development builds included; someone who builds the app | |
| 34 | /// for a g1t of their own, with their own release key, sets | |
| 35 | /// G1T_RUNNER_RELEASE_KEY when they build it. | |
| 36 | const RELEASE_KEY: &str = match option_env!("G1T_RUNNER_RELEASE_KEY") { | |
| 37 | Some(key) => key, | |
| 38 | None => "C8TxhitVxZpx+VLr3gtu0nq7I46q2awBsPZISwDzIgI=", | |
| 39 | }; | |
| 40 | ||
| 41 | /// Why the releases couldn't be had. | |
| 42 | #[derive(Clone, Debug, PartialEq, Eq)] | |
| 43 | pub enum Fault { | |
| 44 | /// The manifest isn't signed with g1t's release key: nothing in it is used. | |
| 45 | Unsigned(String), | |
| 46 | /// It couldn't be fetched or read, or a download didn't check out. | |
| 47 | Failed(String), | |
| 48 | } | |
| 49 | ||
| 50 | impl std::fmt::Display for Fault { | |
| 51 | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | |
| 52 | match self { | |
| 53 | Fault::Unsigned(why) | Fault::Failed(why) => f.write_str(why), | |
| 54 | } | |
| 55 | } | |
| 56 | } | |
| 57 | ||
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 58 | #[derive(Deserialize)] |
| 59 | pub struct File { | |
| 60 | pub name: String, | |
| 61 | pub sha256: String, | |
| 62 | } | |
| 63 | ||
| 64 | #[derive(Deserialize)] | |
| 65 | pub struct Manifest { | |
| 66 | pub version: String, | |
| 67 | pub files: BTreeMap<String, File>, | |
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 68 | /// The sandbox's images, which `sandbox update` takes: none in |
| 69 | /// releases from before them, or published without them. | |
| 70 | #[serde(default)] | |
| 71 | pub sandbox: Option<SandboxRelease>, | |
| 72 | } | |
| 73 | ||
| 74 | #[derive(Default, Deserialize)] | |
| 75 | pub struct SandboxRelease { | |
| 76 | #[serde(default)] | |
| 77 | pub files: BTreeMap<String, serde::de::IgnoredAny>, | |
| 78 | } | |
| 79 | ||
| 80 | impl Manifest { | |
| 81 | /// Whether the release has the sandbox's images, so `sandbox update` has something to take. | |
| 82 | pub fn has_images(&self) -> bool { | |
| 83 | self.sandbox.as_ref().is_some_and(|sandbox| !sandbox.files.is_empty()) | |
| 84 | } | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 85 | } |
| 86 | ||
| 87 | /// This computer's platform, as releases name it. | |
| 88 | pub fn platform() -> String { | |
| 89 | let os = match std::env::consts::OS { | |
| 90 | "macos" => "macos", | |
| 91 | "windows" => "windows", | |
| 92 | _ => "linux", | |
| 93 | }; | |
| 94 | let arch = if std::env::consts::ARCH == "aarch64" { "arm64" } else { "x64" }; | |
| 95 | format!("{os}-{arch}") | |
| 96 | } | |
| 97 | ||
| 98 | /// Where the program is kept. | |
| 99 | pub fn binary(folder: &Path) -> PathBuf { | |
| 100 | folder.join("bin").join(if cfg!(windows) { "g1t-runner.exe" } else { "g1t-runner" }) | |
| 101 | } | |
| 102 | ||
| 103 | /// The version installed, as the manifest named it. | |
| 104 | pub fn installed(folder: &Path) -> Option<String> { | |
| 105 | binary(folder).exists().then(|| std::fs::read_to_string(folder.join("bin").join("version")).ok()).flatten().map(|v| v.trim().to_owned()) | |
| 106 | } | |
| 107 | ||
| 108 | /// `a.b.c` newer than `x.y.z`. | |
| 109 | pub fn newer(candidate: &str, current: &str) -> bool { | |
| 110 | let parts = |v: &str| -> Vec<u64> { v.trim_start_matches('v').split(['.', '-']).take(3).map(|p| p.parse().unwrap_or(0)).collect() }; | |
| 111 | parts(candidate) > parts(current) | |
| 112 | } | |
| 113 | ||
| 114 | pub fn sha256(bytes: &[u8]) -> String { | |
| 115 | Sha256::digest(bytes).iter().map(|byte| format!("{byte:02x}")).collect() | |
| 116 | } | |
| 117 | ||
| 118 | fn downloads(server: &Url) -> String { | |
| 119 | format!("{}downloads/runner", server.as_str()) | |
| 120 | } | |
| 121 | ||
| 122 | fn get(url: &str) -> Result<Vec<u8>, String> { | |
| 123 | tauri::async_runtime::block_on(async { | |
| 124 | let client = reqwest::Client::builder().timeout(Duration::from_secs(300)).build().map_err(|e| e.to_string())?; | |
| 125 | let response = client.get(url).send().await.map_err(|e| e.to_string())?; | |
| 126 | if !response.status().is_success() { | |
| 127 | return Err(format!("{url} answered {}", response.status())); | |
| 128 | } | |
| 129 | response.bytes().await.map(|bytes| bytes.to_vec()).map_err(|e| e.to_string()) | |
| 130 | }) | |
| 131 | } | |
| 132 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 133 | /// Whether `signature` (base64) is the release key's over `message`. |
| 134 | pub fn verify(message: &[u8], signature: &str, key: &str) -> Result<(), Fault> { | |
| 135 | let unsigned = |why: &str| Fault::Unsigned(why.to_owned()); | |
| 136 | let key: [u8; 32] = STANDARD | |
| 137 | .decode(key.trim()) | |
| 138 | .ok() | |
| 139 | .and_then(|bytes| bytes.try_into().ok()) | |
| 140 | .ok_or_else(|| unsigned("the release key built into g1t doesn't read"))?; | |
| 141 | let signature: [u8; 64] = STANDARD | |
| 142 | .decode(signature.trim()) | |
| 143 | .ok() | |
| 144 | .and_then(|bytes| bytes.try_into().ok()) | |
| 145 | .ok_or_else(|| unsigned("the release's signature doesn't read"))?; | |
| 146 | let key = ed25519_dalek::VerifyingKey::from_bytes(&key).map_err(|_| unsigned("the release key built into g1t isn't a key"))?; | |
| 147 | key.verify_strict(message, &ed25519_dalek::Signature::from_bytes(&signature)) | |
| 148 | .map_err(|_| unsigned("the release's signature isn't g1t's")) | |
| 149 | } | |
| 150 | ||
| 151 | /// A manifest, once its signature checks out against `key`. | |
| 152 | fn signed(manifest: &[u8], signature: &str, key: &str) -> Result<Manifest, Fault> { | |
| 153 | verify(manifest, signature, key)?; | |
| 154 | serde_json::from_slice(manifest).map_err(|e| Fault::Failed(format!("the runner's release manifest doesn't read: {e}"))) | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 155 | } |
| 156 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 157 | /// The newest release's manifest, signed by g1t. One without a |
| 158 | /// signature, or whose signature doesn't check out, is refused. | |
| 159 | pub fn latest(server: &Url) -> Result<Manifest, Fault> { | |
| 160 | let base = downloads(server); | |
| 161 | let manifest = get(&format!("{base}/latest.json")).map_err(Fault::Failed)?; | |
| 162 | let signature = get(&format!("{base}/latest.json.sig")).map_err(|why| if why.contains("404") { Fault::Unsigned("the release has no signature".into()) } else { Fault::Failed(why) })?; | |
| 163 | signed(&manifest, &String::from_utf8_lossy(&signature), RELEASE_KEY) | |
| 164 | } | |
| 165 | ||
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 166 | /// Downloads a release's program for this platform and checks it against |
| 167 | /// the manifest's SHA-256. | |
| 168 | pub fn download(server: &Url, manifest: &Manifest) -> Result<Vec<u8>, String> { | |
| 169 | let file = manifest.files.get(&platform()).ok_or_else(|| format!("g1t-runner {} has no build for {}", manifest.version, platform()))?; | |
| 170 | let bytes = get(&format!("{}/{}/{}", downloads(server), manifest.version, file.name))?; | |
| 171 | if sha256(&bytes) != file.sha256.to_ascii_lowercase() { | |
| 172 | return Err("the download's SHA-256 isn't the release's, so it was thrown away".into()); | |
| 173 | } | |
| 174 | Ok(bytes) | |
| 175 | } | |
| 176 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 177 | /// Puts a checked program in place. The runner it replaces must not be |
| 178 | /// running, but a short launcher command may be (a `sandbox status` the | |
| 179 | /// app is asking at that moment): the program there is moved aside first, | |
| 180 | /// under a name of its own, since Windows won't overwrite a program that | |
| 181 | /// is running but lets it be renamed. Those left from earlier updates are | |
| 182 | /// removed here, once nothing runs them. | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 183 | pub fn put(folder: &Path, bytes: &[u8], version: &str) -> Result<(), String> { |
| 184 | let path = binary(folder); | |
| 185 | let dir = path.parent().unwrap_or(folder); | |
| 186 | std::fs::create_dir_all(dir).map_err(|e| e.to_string())?; | |
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 187 | for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() { |
| 188 | if entry.file_name().to_string_lossy().ends_with(".old") { | |
| 189 | let _ = std::fs::remove_file(entry.path()); | |
| 190 | } | |
| 191 | } | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 192 | let new = path.with_extension("new"); |
| 193 | std::fs::write(&new, bytes).map_err(|e| e.to_string())?; | |
| 194 | #[cfg(unix)] | |
| 195 | { | |
| 196 | use std::os::unix::fs::PermissionsExt; | |
| 197 | std::fs::set_permissions(&new, std::fs::Permissions::from_mode(0o755)).map_err(|e| e.to_string())?; | |
| 198 | } | |
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 199 | let stamp = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map_or(0, |since| since.as_nanos()); |
| 200 | let old = dir.join(format!("g1t-runner-{stamp}.old")); | |
| 201 | let moved = path.exists() && std::fs::rename(&path, &old).is_ok(); | |
| 202 | if let Err(error) = std::fs::rename(&new, &path) { | |
| 203 | if moved { | |
| 204 | let _ = std::fs::rename(&old, &path); | |
| 205 | } | |
| 206 | return Err(format!("could not put g1t-runner in place: {error}")); | |
| 207 | } | |
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 208 | std::fs::write(dir.join("version"), version).map_err(|e| e.to_string()) |
| 209 | } | |
| 210 | ||
| 211 | /// Starts `register-and-run`, which registers the first time (from | |
| 212 | /// `G1T_RUNNER_*` in the environment the caller sets) and then runs. | |
| 213 | pub fn spawn(binary: &Path, dir: &Path, envs: &[(&str, String)]) -> std::io::Result<Child> { | |
| 214 | let mut command = quiet(binary); | |
| 215 | command.arg("register-and-run").arg("--dir").arg(dir); | |
| 216 | // What the app's own environment may have set for another runner is not this one's. | |
| 217 | for (name, _) in std::env::vars_os() { | |
| 218 | if name.to_string_lossy().starts_with("G1T_RUNNER_") { | |
| 219 | command.env_remove(&name); | |
| 220 | } | |
| 221 | } | |
| 222 | command.envs(envs.iter().map(|(name, value)| (*name, value.as_str()))); | |
| 223 | command.stdout(Stdio::piped()).stderr(Stdio::piped()); | |
| 224 | // A group of its own, so stopping it reaches what it started. | |
| 225 | #[cfg(unix)] | |
| 226 | std::os::unix::process::CommandExt::process_group(&mut command, 0); | |
| 227 | command.spawn() | |
| 228 | } | |
| 229 | ||
| 230 | /// Unregisters the runner whose configuration is in `dir`, and forgets it. | |
| 231 | pub fn forget(binary: &Path, dir: &Path) { | |
| 232 | if binary.exists() && dir.join("config.json").exists() { | |
| 233 | let _ = quiet(binary).arg("remove").arg("--dir").arg(dir).stdout(Stdio::null()).stderr(Stdio::null()).status(); | |
| 234 | } | |
| 235 | let _ = std::fs::remove_file(dir.join("config.json")); | |
| 236 | } | |
| 237 | ||
| 238 | /// Stops the runner and what it started: the job it ran in host mode | |
| 239 | /// included. A job cut short this way fails on g1t. | |
| 240 | pub fn stop(child: &mut Child) { | |
| 241 | let pid = child.id().to_string(); | |
| 242 | if cfg!(windows) { | |
| 243 | let _ = quiet("taskkill").args(["/T", "/F", "/PID", &pid]).stdout(Stdio::null()).stderr(Stdio::null()).status(); | |
| 244 | } else { | |
| 245 | let _ = quiet("kill").args(["-TERM", &format!("-{pid}")]).stdout(Stdio::null()).stderr(Stdio::null()).status(); | |
| 246 | for _ in 0..40 { | |
| 247 | if matches!(child.try_wait(), Ok(Some(_))) { | |
| 248 | return; | |
| 249 | } | |
| 250 | std::thread::sleep(Duration::from_millis(250)); | |
| 251 | } | |
| 252 | } | |
| 253 | let _ = child.kill(); | |
| 254 | let _ = child.wait(); | |
| 255 | } | |
| 256 | ||
| 257 | /// Removes the containers, networks and volumes of the jobs of the runner | |
| 258 | /// registered in `dir`, as `g1t-runner` does itself when it starts: each | |
| 259 | /// carries the label `sh.g1t.runner=<its id>`. | |
| 260 | pub fn sweep(dir: &Path) { | |
| 261 | #[derive(Deserialize)] | |
| 262 | struct Registered { | |
| 263 | runner: String, | |
| 264 | } | |
| 265 | let Some(registered) = std::fs::read(dir.join("config.json")).ok().and_then(|bytes| serde_json::from_slice::<Registered>(&bytes).ok()) else { return }; | |
| 266 | let label = format!("label=sh.g1t.runner={}", registered.runner); | |
| 267 | for (list, remove) in [ | |
| 268 | (vec!["ps", "-aq", "--filter", &label], vec!["rm", "-f", "-v"]), | |
| 269 | (vec!["network", "ls", "-q", "--filter", &label], vec!["network", "rm"]), | |
| 270 | (vec!["volume", "ls", "-q", "--filter", &label], vec!["volume", "rm", "-f"]), | |
| 271 | ] { | |
| 272 | let Ok(out) = quiet("docker").args(&list).stderr(Stdio::null()).output() else { return }; | |
| 273 | let ids: Vec<String> = String::from_utf8_lossy(&out.stdout).split_whitespace().map(str::to_owned).collect(); | |
| 274 | if !ids.is_empty() { | |
| 275 | let _ = quiet("docker").args(&remove).args(&ids).stdout(Stdio::null()).stderr(Stdio::null()).status(); | |
| 276 | } | |
| 277 | } | |
| 278 | } | |
| 279 | ||
| 280 | /// A runner the app started before and did not stop (it was ended | |
| 281 | /// without quitting): stopped, if that process is still `g1t-runner`. | |
| 282 | pub fn reap(pid: u32) { | |
| 283 | let pid = pid.to_string(); | |
| 284 | if cfg!(windows) { | |
| 285 | let _ = quiet("taskkill") | |
| 286 | .args(["/T", "/F", "/FI", "IMAGENAME eq g1t-runner.exe", "/PID", &pid]) | |
| 287 | .stdout(Stdio::null()) | |
| 288 | .stderr(Stdio::null()) | |
| 289 | .status(); | |
| 290 | } else { | |
| 291 | let named = quiet("ps").args(["-p", &pid, "-o", "comm="]).output().is_ok_and(|out| String::from_utf8_lossy(&out.stdout).trim().ends_with("g1t-runner")); | |
| 292 | if named { | |
| 293 | let _ = quiet("kill").args(["-TERM", &pid]).status(); | |
| 294 | } | |
| 295 | } | |
| 296 | } | |
| 297 | ||
| 298 | #[cfg(test)] | |
| 299 | mod tests { | |
| 300 | use super::*; | |
| 301 | ||
| 302 | #[test] | |
| 303 | fn releases_are_named_and_compared_as_the_runner_does() { | |
| 304 | assert!(["linux-x64", "linux-arm64", "macos-arm64", "macos-x64", "windows-x64", "windows-arm64"].contains(&platform().as_str())); | |
| 305 | assert!(newer("0.10.0", "0.9.9")); | |
| 306 | assert!(!newer("0.2.0", "0.2.0")); | |
| 307 | assert_eq!(sha256(b"abc"), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"); | |
| 308 | let manifest: Manifest = serde_json::from_str(r#"{"version":"0.2.3","agent_image":null,"files":{"windows-x64":{"name":"g1t-runner-windows-x64.exe","sha256":"ab"}}}"#).unwrap(); | |
| 309 | assert_eq!(manifest.files["windows-x64"].name, "g1t-runner-windows-x64.exe"); | |
| 310 | assert_eq!(downloads(&Url::parse("https://g1t.sh/").unwrap()), "https://g1t.sh/downloads/runner"); | |
| 311 | } | |
| 312 | ||
| The desktop app is 0.3.2, and its runner updates are fast, seen and only ever g1t's. The app now trusts a g1t-runner release only when its manifest's Ed25519 signature, latest.json.sig, checks out against g1t's runner release key, which is built into the app (the public half of RUNNER_RELEASE_PUBLIC_KEY, kept in runner/native.rs, where development builds have it too; G1T_RUNNER_RELEASE_KEY overrides it for a g1t with its own key), as the runner already did with its own updates. A manifest with no signature, or one that doesn't check out, is refused and nothing in it is used: the runner keeps the version it has and says "Update refused: not signed by g1t", or, with no runner installed yet, "Not running: the runner's release isn't signed by g1t", each with why and what to do. It looks every hour rather than every six, on a thread of its own so the runner is looked after meanwhile, and what it finds waits, downloaded and checked, until each kind of job is idle, then goes in kind by kind, never cutting a job short: Linux jobs take the release's image (sandbox update) as soon as they have no job, while Windows jobs, whose runner is the program itself, are stopped after their job (sandbox down --only windows --drain), the program is replaced and they start again, the other kind running on throughout; a runner in Docker or on the computer stops between jobs and starts on the new one. The program there is moved aside rather than overwritten, so a status the app is asking at that moment can't stop it. Each kind of job's card in Settings shows its runner version, when the app last looked ("Checked 12 min ago") and a newer one waiting ("0.3.2 ready, installs after this job", "Installing 0.3.2"), and "Check for runner updates" beside the computer's name looks at once and installs what it finds, even with "Download updates by themselves" off, which otherwise stops the hourly look. The desktop report carries the same (snake_case: the diagnosis's update, with checked_at, ready and note, and each kind's update), so the Runners page's Desktops cards show it, "Checked" with how long ago, and this computer's row has the button too. All the words are diagnosis.rs's. The desktop guide says how updates work, what each card shows, the new troubleshooting rows and where the release key lives. Tests cover the signature with a key pair of their own and with release 0.3.0's manifest and signature as g1t.sh published them, kept byte for byte (.gitattributes), the wording, the cards and the report. | 313 | /// A key pair of the tests' own: what it signs checks out, and nothing else does. |
| 314 | #[test] | |
| 315 | fn only_a_manifest_signed_with_the_release_key_is_trusted() { | |
| 316 | use ed25519_dalek::Signer; | |
| 317 | let signing = ed25519_dalek::SigningKey::from_bytes(&[7u8; 32]); | |
| 318 | let key = STANDARD.encode(signing.verifying_key().to_bytes()); | |
| 319 | let manifest = br#"{"version":"0.3.2","files":{"windows-x64":{"name":"g1t-runner-windows-x64.exe","sha256":"ab"}}}"#; | |
| 320 | let signature = STANDARD.encode(signing.sign(manifest).to_bytes()); | |
| 321 | let trusted = signed(manifest, &signature, &key).unwrap(); | |
| 322 | assert_eq!(trusted.version, "0.3.2"); | |
| 323 | assert!(!trusted.has_images()); | |
| 324 | // Changed after it was signed: a newer version slipped in. | |
| 325 | let changed = br#"{"version":"9.9.9","files":{"windows-x64":{"name":"g1t-runner-windows-x64.exe","sha256":"ab"}}}"#; | |
| 326 | assert_eq!(signed(changed, &signature, &key).err(), Some(Fault::Unsigned("the release's signature isn't g1t's".into()))); | |
| 327 | // Signed with another key. | |
| 328 | let other = STANDARD.encode(ed25519_dalek::SigningKey::from_bytes(&[8u8; 32]).verifying_key().to_bytes()); | |
| 329 | assert!(matches!(signed(manifest, &signature, &other), Err(Fault::Unsigned(_)))); | |
| 330 | // No signature, or not one. | |
| 331 | assert!(matches!(signed(manifest, "", &key), Err(Fault::Unsigned(_)))); | |
| 332 | assert!(matches!(signed(manifest, "not base64!", &key), Err(Fault::Unsigned(_)))); | |
| 333 | // Signed, yet not a manifest: that is a failure, not a refusal. | |
| 334 | let junk = b"not json"; | |
| 335 | assert!(matches!(signed(junk, &STANDARD.encode(signing.sign(junk).to_bytes()), &key), Err(Fault::Failed(_)))); | |
| 336 | } | |
| 337 | ||
| 338 | /// Release 0.3.0's manifest and signature, as g1t.sh published them: | |
| 339 | /// the key built in is the one g1t's releases are signed with. | |
| 340 | #[test] | |
| 341 | fn g1ts_published_release_checks_out_against_the_key_built_in() { | |
| 342 | let manifest = include_bytes!("testdata/latest-0.3.0.json"); | |
| 343 | let signature = include_str!("testdata/latest-0.3.0.json.sig"); | |
| 344 | let release = signed(manifest, signature, RELEASE_KEY).unwrap(); | |
| 345 | assert_eq!(release.version, "0.3.0"); | |
| 346 | assert!(release.has_images()); | |
| 347 | let mut changed = manifest.to_vec(); | |
| 348 | let at = changed.windows(5).position(|five| five == b"0.3.0").unwrap(); | |
| 349 | changed[at + 4] = b'9'; | |
| 350 | assert!(matches!(signed(&changed, signature, RELEASE_KEY), Err(Fault::Unsigned(_)))); | |
| 351 | } | |
| 352 | ||
| 353 | #[test] | |
| 354 | fn a_new_program_takes_the_old_ones_place_and_its_version() { | |
| 355 | let dir = std::env::temp_dir().join(format!("g1t-desktop-put-{}", std::process::id())); | |
| 356 | let _ = std::fs::remove_dir_all(&dir); | |
| 357 | put(&dir, b"one", "0.3.1").unwrap(); | |
| 358 | put(&dir, b"two", "0.3.2").unwrap(); | |
| 359 | assert_eq!(std::fs::read(binary(&dir)).unwrap(), b"two"); | |
| 360 | assert_eq!(installed(&dir).as_deref(), Some("0.3.2")); | |
| 361 | // The one moved aside goes with the next update, once nothing runs it. | |
| 362 | put(&dir, b"three", "0.3.3").unwrap(); | |
| 363 | let old = std::fs::read_dir(dir.join("bin")).unwrap().flatten().filter(|e| e.file_name().to_string_lossy().ends_with(".old")).count(); | |
| 364 | assert_eq!(old, 1); | |
| 365 | let _ = std::fs::remove_dir_all(&dir); | |
| 366 | } | |
| 367 | ||
| Merge the desktop app's runner: it starts with the app for the signed-in person, sandboxed by default and running nothing where no sandbox is available yet, registers through the site, keeps itself up to date, and says everywhere what it is doing, why it is not, and how to fix it. | 368 | #[test] |
| 369 | fn nothing_is_installed_in_an_empty_folder() { | |
| 370 | let dir = std::env::temp_dir().join(format!("g1t-desktop-runner-{}", std::process::id())); | |
| 371 | assert_eq!(installed(&dir), None); | |
| 372 | assert!(binary(&dir).ends_with(if cfg!(windows) { "bin/g1t-runner.exe" } else { "bin/g1t-runner" })); | |
| 373 | } | |
| 374 | } |
This file's history is long; its oldest lines are credited to the oldest commit read.