Skip to content
Sign up

flagon-io/g1t

public
0
667 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Fast pages, required checks on the branch, self-hosted runners, honest incidents1//! Running what g1t hands the runner: a workflow job or agent work, with
2//! the same harness g1t's sandboxes run (this program, in another mode).
3//!
4//! - **In Docker** (the default): one container per job, removed when it
5//! ends, from the job's `container:` image or the runner's `--image`. A
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.6//! Linux build of the harness is mounted into it read-only; that and the
7//! job's own work volume, removed after, are its only mounts. It runs
8//! with every capability dropped but the few a build needs, no new
9//! privileges, a process, memory and CPU limit, and a network of its
10//! own. With nested-container isolation (sandbox.rs), all of this is on
11//! the runner's own Docker Engine, inside the runner's container, and a
12//! job that uses Docker gets an engine of its own (engine.rs).
13//! - **Directly** (`--no-docker`, deprecated): the harness runs as a child
14//! process, in a fresh folder under the runner's work folder that is
15//! removed after. Unsandboxed: the job's log says so, and a public
16//! repository's work is refused.
Fast pages, required checks on the branch, self-hosted runners, honest incidents17//!
18//! The runner's own credential never reaches either: a job only ever has
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.19//! its own short-lived token, as it would in g1t's sandbox. Credentials are
20//! passed by name (`-e NAME`), never on the command line.
Fast pages, required checks on the branch, self-hosted runners, honest incidents21
22use std::collections::BTreeMap;
23use std::path::{Path, PathBuf};
24use std::process::{Child, Command, Stdio};
25use std::time::{Duration, Instant};
26
27use anyhow::{Context, Result, bail};
28
29use super::api::Assignment;
30use super::config::Config;
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.31use super::sandbox::{self, Isolation};
Fast pages, required checks on the branch, self-hosted runners, honest incidents32use super::{log, update};
33
34/// The image workflow jobs run in when they name none: Debian with Node,
35/// git and Python, as g1t's sandbox has.
36pub const DEFAULT_IMAGE: &str = "node:24-bookworm";
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.37/// The most processes in a job's container, unless the runner says.
38pub const DEFAULT_PIDS_LIMIT: u32 = 4096;
39
40/// The capabilities a job's container keeps; Docker's others, and
41/// everything else, are dropped. A workflow job runs as its image's user,
42/// root for most CI images, and these are what installing packages and
43/// building need:
44///
45/// - `CHOWN`, `FOWNER`, `FSETID`, `DAC_OVERRIDE`: a package manager (apt,
46/// npm as root) writing files it then hands to another owner, keeping
47/// setgid bits, and root writing files it does not own.
48/// - `SETUID`, `SETGID`: apt and others drop to their own user (`_apt`) to
49/// download; with no new privileges, nothing can gain a user back.
50/// - `KILL`: a step stopping processes another user started.
51/// - `NET_BIND_SERVICE`: a test server on port 80 or 443.
52///
53/// Not kept, among others: `NET_RAW` (raw sockets, `ping`), `MKNOD`,
54/// `SYS_CHROOT`, `SETFCAP`, `AUDIT_WRITE`, and every `SYS_*` and `NET_ADMIN`.
55pub const JOB_CAPS: &[&str] = &["CHOWN", "DAC_OVERRIDE", "FOWNER", "FSETID", "SETUID", "SETGID", "KILL", "NET_BIND_SERVICE"];
Fast pages, required checks on the branch, self-hosted runners, honest incidents56
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.57/// Agent work's image runs as `node` and uses `sudo` once, to put its
58/// guardrails' policy where the agent cannot change it, and then removes
59/// its own sudo (guard.rs). sudo needs a setuid program to work and
60/// `AUDIT_WRITE` to log, so agent work keeps those and has no
61/// `no-new-privileges`; it still has every other limit.
62pub const AGENT_EXTRA_CAPS: &[&str] = &["AUDIT_WRITE"];
63
Fast pages, required checks on the branch, self-hosted runners, honest incidents64/// Work in progress.
65pub struct Running {
66 pub id: String,
67 pub kind: String,
68 pub name: String,
69 child: Child,
70 container: Option<String>,
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.71 /// The job's network and work volume, removed after.
72 network: Option<String>,
73 volume: Option<String>,
74 /// The job's own Docker Engine, when it may have one.
75 engine: Option<JobEngine>,
Fast pages, required checks on the branch, self-hosted runners, honest incidents76 folder: Option<PathBuf>,
77 /// Past its time limit and a grace for g1t to say so: stopped here.
78 deadline: Instant,
79}
80
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.81#[cfg(unix)]
82use super::engine::JobEngine;
83
84/// No job has an engine of its own off Unix.
85#[cfg(not(unix))]
86pub struct JobEngine;
87
88#[cfg(not(unix))]
89impl JobEngine {
90 fn remove(&self) {}
91 fn job_args(&self) -> Vec<String> {
92 Vec::new()
93 }
94}
95
96/// `docker`, at the runner's own engine when it has one.
97fn docker() -> Command {
98 #[cfg(unix)]
99 return super::engine::docker();
100 #[cfg(not(unix))]
101 Command::new("docker")
102}
103
104fn quietly(command: &mut Command) {
105 let _ = command.stdout(Stdio::null()).stderr(Stdio::null()).status();
106}
107
Fast pages, required checks on the branch, self-hosted runners, honest incidents108impl Running {
109 pub fn overdue(&self) -> bool {
110 Instant::now() > self.deadline
111 }
112
113 /// Its exit code once it has ended.
114 pub fn ended(&mut self) -> Option<i32> {
115 match self.child.try_wait() {
116 Ok(Some(status)) => Some(status.code().unwrap_or(1)),
117 Ok(None) => None,
118 Err(_) => Some(1),
119 }
120 }
121
122 /// Stops it: the container, then the process and everything it started.
123 pub fn stop(&mut self) {
124 if let Some(container) = &self.container {
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.125 quietly(docker().args(["kill", container]));
126 }
127 #[cfg(unix)]
128 if let Some(engine) = &self.engine {
129 quietly(docker().args(["kill", &engine.container]));
Fast pages, required checks on the branch, self-hosted runners, honest incidents130 }
131 let pid = self.child.id().to_string();
132 if cfg!(windows) {
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.133 quietly(Command::new("taskkill").args(["/T", "/F", "/PID", &pid]));
Fast pages, required checks on the branch, self-hosted runners, honest incidents134 } else {
135 // Its own process group (see `directly`): the steps' processes too.
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.136 quietly(Command::new("kill").args(["-TERM", &format!("-{pid}")]));
Fast pages, required checks on the branch, self-hosted runners, honest incidents137 }
138 let _ = self.child.kill();
139 let _ = self.child.wait();
140 }
141
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.142 /// Removes what it left: its container, its engine and that engine's
143 /// data, its network, its work volume, its folder.
Fast pages, required checks on the branch, self-hosted runners, honest incidents144 pub fn clean(&mut self) {
145 if let Some(container) = &self.container {
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.146 quietly(docker().args(["rm", "-f", "-v", container]));
147 }
148 if let Some(engine) = self.engine.take() {
149 engine.remove();
Fast pages, required checks on the branch, self-hosted runners, honest incidents150 }
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.151 if let Some(network) = &self.network {
152 quietly(docker().args(["network", "rm", network]));
153 }
154 if let Some(volume) = &self.volume {
155 quietly(docker().args(["volume", "rm", "-f", volume]));
156 }
Fast pages, required checks on the branch, self-hosted runners, honest incidents157 if let Some(folder) = &self.folder {
158 let _ = std::fs::remove_dir_all(folder);
159 }
160 }
161}
162
163/// A name Docker takes, from a job's id.
164fn container_name(id: &str) -> String {
165 let clean: String = id.chars().map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c } else { '-' }).collect();
166 format!("g1t-{clean}")
167}
168
169/// The environment the harness gets for this work.
170fn harness_env(config: &Config, work: &Assignment) -> Result<BTreeMap<String, String>> {
171 let mut env = BTreeMap::new();
172 match work.kind.as_str() {
173 "workflow" => {
174 let Some(token) = &work.token else { bail!("the job came without its token") };
175 env.insert("MODE".into(), "actions".into());
176 env.insert("G1T_API".into(), config.api.clone());
177 env.insert("ACTIONS_JOB".into(), work.id.clone());
178 env.insert("ACTIONS_TOKEN".into(), token.clone());
179 }
180 "agent" => {
181 for (name, value) in work.env.clone().unwrap_or_default() {
182 let value = match value {
183 serde_json::Value::String(text) => text,
184 other => other.to_string(),
185 };
186 env.insert(name, value);
187 }
188 if !env.contains_key("MODE") && !env.contains_key("PROMPT") {
189 bail!("the agent work came without its environment");
190 }
191 // Where its API calls go: this installation's.
192 env.insert("G1T_API".into(), config.api.clone());
193 }
194 other => bail!("this runner does not know work of kind {other}; update it"),
195 }
196 // The machine is the workspace's own: no mining watch.
197 env.insert("G1T_ABUSE".into(), "off".into());
198 Ok(env)
199}
200
201/// Starts work. The harness reports a workflow job's steps and log itself;
202/// the runner tells g1t how the process ended.
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.203pub fn start(config: &Config, isolation: Isolation, folder: &Path, work: &Assignment) -> Result<Running> {
Fast pages, required checks on the branch, self-hosted runners, honest incidents204 let env = harness_env(config, work)?;
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.205 if isolation == Isolation::Host {
206 refuse_public_on_host(work)?;
207 return directly(config, work, env);
208 }
209 in_docker(config, isolation, folder, work, env)
210}
211
212/// Host mode does not take a public repository's work: anyone can propose
213/// code to one, and here it would run on the machine itself.
214fn refuse_public_on_host(work: &Assignment) -> Result<()> {
215 match work.visibility.as_deref() {
216 Some("public") => bail!("{}", sandbox::host_mode_refuses(&work.repo)),
217 Some(_) => Ok(()),
218 None => {
219 log(&format!("g1t did not say whether {} is public (an older g1t), so it runs; host mode refuses public repositories' work once it does.", work.repo));
220 Ok(())
221 }
222 }
223}
224
225/// Each job container's limits.
226#[derive(Debug, Clone, PartialEq)]
227pub struct Limits {
228 pub memory: String,
229 pub cpus: String,
230 pub pids: u32,
231}
232
233/// Three quarters of `MemTotal` in `/proc/meminfo`, in MiB, which under
234/// Sysbox is the runner container's own memory.
235fn default_memory(meminfo: &str) -> Option<String> {
236 let kb: u64 = meminfo.lines().find_map(|l| l.strip_prefix("MemTotal:"))?.trim().trim_end_matches("kB").trim().parse().ok()?;
237 let mib = kb / 1024 * 3 / 4;
238 (mib >= 256).then(|| format!("{mib}m"))
239}
240
241pub fn limits(config: &Config) -> Limits {
242 let memory = config
243 .memory
244 .clone()
245 .or_else(|| default_memory(&std::fs::read_to_string("/proc/meminfo").unwrap_or_default()))
246 .unwrap_or_else(|| "4g".into());
247 let cpus = config.cpus.clone().unwrap_or_else(|| std::thread::available_parallelism().map(|n| n.get()).unwrap_or(2).to_string());
248 Limits { memory, cpus, pids: config.pids_limit.unwrap_or(DEFAULT_PIDS_LIMIT) }
249}
250
251/// Everything a job's `docker run` is made from.
252pub struct Plan<'a> {
253 pub kind: &'a str,
254 pub name: &'a str,
255 pub runner: &'a str,
256 pub image: &'a str,
257 pub harness: &'a Path,
258 pub network: &'a str,
259 /// The job's work volume, for a workflow job's `/home/runner`.
260 pub volume: Option<&'a str>,
261 pub limits: &'a Limits,
262 /// The names of the harness's variables, whose values come from the
263 /// runner's process, never its command line.
264 pub env_names: Vec<&'a str>,
265 /// What the job's own engine adds (`JobEngine::job_args`).
266 pub engine_args: Vec<String>,
267}
268
269/// The arguments of `docker` that start a job's container.
270pub fn run_args(plan: &Plan) -> Vec<String> {
271 let mut args: Vec<String> = Vec::new();
272 fn add(args: &mut Vec<String>, items: &[&str]) {
273 args.extend(items.iter().map(|s| s.to_string()));
274 }
275 add(&mut args, &["run", "--rm", "--name", plan.name, "--label", &format!("sh.g1t.runner={}", plan.runner)]);
276 add(&mut args, &["--pull", "missing", "--init"]);
277 // The walls.
278 add(&mut args, &["--cap-drop", "ALL"]);
279 let mut caps: Vec<&str> = JOB_CAPS.to_vec();
280 if plan.kind == "agent" {
281 caps.extend(AGENT_EXTRA_CAPS);
282 } else {
283 add(&mut args, &["--security-opt", "no-new-privileges"]);
284 }
285 for cap in caps {
286 add(&mut args, &["--cap-add", cap]);
287 }
288 let pids = plan.limits.pids.to_string();
289 add(&mut args, &["--pids-limit", &pids, "--memory", &plan.limits.memory, "--memory-swap", &plan.limits.memory, "--cpus", &plan.limits.cpus]);
290 add(&mut args, &["--network", plan.network, "--ipc", "private"]);
291 // The job is in its `container:` image already: the harness inside
292 // does not start it again (actions/containers.rs).
293 add(&mut args, &["-e", "G1T_JOB_CONTAINER=1"]);
294 for name in &plan.env_names {
295 add(&mut args, &["-e", name]);
296 }
297 // Its only mounts: the harness, read-only, and its own volume.
298 add(&mut args, &["-v", &format!("{}:/opt/g1t/g1t-runner:ro", plan.harness.display())]);
299 if let Some(volume) = plan.volume {
300 add(&mut args, &["-v", &format!("{volume}:/home/runner")]);
301 }
302 args.extend(plan.engine_args.iter().cloned());
303 add(&mut args, &["--entrypoint", "/opt/g1t/g1t-runner", plan.image]);
304 args
Fast pages, required checks on the branch, self-hosted runners, honest incidents305}
306
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.307fn in_docker(config: &Config, isolation: Isolation, folder: &Path, work: &Assignment, env: BTreeMap<String, String>) -> Result<Running> {
Fast pages, required checks on the branch, self-hosted runners, honest incidents308 let image = match work.kind.as_str() {
309 "agent" => match config.agent_image.clone().or_else(|| update::agent_image(config)) {
310 Some(image) => image,
311 None => bail!(
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.312 "agent work needs an image with git, Node and the agent's CLI: register this runner with --agent-image IMAGE"
Fast pages, required checks on the branch, self-hosted runners, honest incidents313 ),
314 },
315 _ => work.image.clone().or_else(|| config.image.clone()).unwrap_or_else(|| DEFAULT_IMAGE.to_owned()),
316 };
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.317 #[cfg(unix)]
318 if super::engine::in_use() {
319 super::engine::ensure_running()?;
320 }
Fast pages, required checks on the branch, self-hosted runners, honest incidents321 let harness = update::linux_harness(config, folder)?;
322 let name = container_name(&work.id);
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.323 // What an earlier run of this job may have left.
324 quietly(docker().args(["rm", "-f", "-v", &name]));
325 quietly(docker().args(["network", "rm", &name]));
326 let output = docker()
327 .args(["network", "create", "--label", &format!("sh.g1t.runner={}", config.runner), &name])
328 .output()
329 .context("could not run docker: is Docker installed and running?")?;
330 if !output.status.success() {
331 bail!("could not make the job's network: {}", String::from_utf8_lossy(&output.stderr).trim());
332 }
333 let volume = (work.kind == "workflow").then(|| format!("{name}-work"));
334 if let Some(volume) = &volume {
335 quietly(docker().args(["volume", "rm", "-f", volume]));
336 quietly(docker().args(["volume", "create", "--label", &format!("sh.g1t.runner={}", config.runner), volume]));
Fast pages, required checks on the branch, self-hosted runners, honest incidents337 }
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.338 // A job's own Docker, only inside the runner's own sandbox: on a
339 // shared engine it would be the machine's.
340 let engine: Option<JobEngine> = job_engine(config, isolation, &name);
341 let limits = limits(config);
342 let plan = Plan {
343 kind: &work.kind,
344 name: &name,
345 runner: &config.runner,
346 image: &image,
347 harness: &harness,
348 network: &name,
349 volume: volume.as_deref(),
350 limits: &limits,
351 env_names: env.keys().map(String::as_str).collect(),
352 engine_args: engine.as_ref().map(JobEngine::job_args).unwrap_or_default(),
353 };
354 let mut command = docker();
355 command.args(run_args(&plan));
Fast pages, required checks on the branch, self-hosted runners, honest incidents356 command.envs(&env);
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.357 log(&format!(
358 "Running {} {} in {image}: its own container and network, capabilities dropped, {} memory, {} CPUs, {} processes{}",
359 work.kind,
360 work.name,
361 limits.memory,
362 limits.cpus,
363 limits.pids,
364 if engine.is_some() { ", Docker of its own when it uses it" } else { "" }
365 ));
366 let child = match command.spawn() {
367 Ok(child) => child,
368 Err(error) => {
369 if let Some(engine) = &engine {
370 engine.remove();
371 }
372 quietly(docker().args(["network", "rm", &name]));
373 return Err(error).context("could not run docker: is Docker installed and running?");
374 }
375 };
376 Ok(Running {
377 id: work.id.clone(),
378 kind: work.kind.clone(),
379 name: work.name.clone(),
380 child,
381 container: Some(name.clone()),
382 network: Some(name),
383 volume,
384 engine,
385 folder: None,
386 deadline: deadline(work),
387 })
388}
389
390#[cfg(unix)]
391fn job_engine(config: &Config, isolation: Isolation, name: &str) -> Option<JobEngine> {
392 if isolation != Isolation::NestedContainer || !config.job_docker {
393 return None;
394 }
395 let image = config.job_docker_image.as_deref().unwrap_or(super::engine::DEFAULT_JOB_ENGINE_IMAGE);
396 match JobEngine::prepare(name, &config.runner, image) {
397 Ok(engine) => Some(engine),
398 Err(error) => {
399 log(&format!("This job has no Docker of its own: {error:#}"));
400 None
401 }
402 }
Fast pages, required checks on the branch, self-hosted runners, honest incidents403}
404
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.405#[cfg(not(unix))]
406fn job_engine(_config: &Config, _isolation: Isolation, _name: &str) -> Option<JobEngine> {
407 None
408}
409
Fast pages, required checks on the branch, self-hosted runners, honest incidents410fn directly(config: &Config, work: &Assignment, env: BTreeMap<String, String>) -> Result<Running> {
411 let me = std::env::current_exe().context("could not find this program to run the job")?;
412 let mut command = Command::new(me);
413 let mut folder = None;
414 if work.kind == "workflow" {
415 // GitHub's layout, in a folder of its own.
416 let root = config.work_dir.join(container_name(&work.id));
417 let _ = std::fs::remove_dir_all(&root);
418 std::fs::create_dir_all(&root).with_context(|| format!("could not make {}", root.display()))?;
419 command.env("G1T_RUNNER_ROOT", &root);
420 folder = Some(root);
421 } else if !cfg!(target_os = "linux") || std::fs::create_dir_all("/work").is_err() {
422 // Agent work expects g1t's sandbox layout (/work), which only a
423 // Linux machine (or a container) set aside for it has.
424 bail!("agent work runs in Docker, or directly on Linux where /work can be written; this runner can do neither");
425 }
426 // Its own variables, less anything of the runner's.
427 for (name, _) in std::env::vars_os() {
428 if name.to_string_lossy().starts_with("G1T_RUNNER_") && name != "G1T_RUNNER_ROOT" {
429 command.env_remove(&name);
430 }
431 }
432 command.envs(&env);
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.433 // The job's log says it ran unsandboxed (actions/mod.rs).
434 command.env(JOB_NOTICE, sandbox::HOST_MODE_IN_JOB);
Fast pages, required checks on the branch, self-hosted runners, honest incidents435 // A group of its own, so stopping it stops what its steps started.
436 #[cfg(unix)]
437 std::os::unix::process::CommandExt::process_group(&mut command, 0);
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.438 log(&format!("Running {} {} on this machine, unsandboxed (host mode, deprecated)", work.kind, work.name));
Fast pages, required checks on the branch, self-hosted runners, honest incidents439 let child = command.spawn().context("could not start the job's process")?;
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.440 Ok(Running {
441 id: work.id.clone(),
442 kind: work.kind.clone(),
443 name: work.name.clone(),
444 child,
445 container: None,
446 network: None,
447 volume: None,
448 engine: None,
449 folder,
450 deadline: deadline(work),
451 })
Fast pages, required checks on the branch, self-hosted runners, honest incidents452}
453
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.454/// A warning the harness puts at the top of the job's log.
455pub const JOB_NOTICE: &str = "G1T_JOB_NOTICE";
456
Fast pages, required checks on the branch, self-hosted runners, honest incidents457fn deadline(work: &Assignment) -> Instant {
458 Instant::now() + Duration::from_secs(u64::from(work.timeout_minutes.max(1)) * 60 + 10 * 60)
459}
460
461/// Whether Docker answers.
462pub fn docker_ready() -> bool {
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.463 docker()
Fast pages, required checks on the branch, self-hosted runners, honest incidents464 .args(["version", "--format", "{{.Server.Version}}"])
465 .stdout(Stdio::null())
466 .stderr(Stdio::null())
467 .status()
468 .is_ok_and(|status| status.success())
469}
470
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.471/// Removes what jobs of this runner's left behind, if it stopped while one ran.
472pub fn sweep(config: &Config) {
473 let label = format!("label=sh.g1t.runner={}", config.runner);
474 for (list, remove) in [
475 (vec!["ps", "-aq", "--filter", &label], vec!["rm", "-f", "-v"]),
476 (vec!["network", "ls", "-q", "--filter", &label], vec!["network", "rm"]),
477 (vec!["volume", "ls", "-q", "--filter", &label], vec!["volume", "rm", "-f"]),
478 ] {
479 let Ok(output) = docker().args(&list).output() else { return };
480 let ids: Vec<String> = String::from_utf8_lossy(&output.stdout).split_whitespace().map(str::to_owned).collect();
481 if !ids.is_empty() {
482 quietly(docker().args(&remove).args(&ids));
483 }
484 }
485}
486
Fast pages, required checks on the branch, self-hosted runners, honest incidents487#[cfg(test)]
488mod tests {
489 use super::*;
490 use serde_json::json;
491
492 fn config() -> Config {
493 Config {
494 url: "https://g1t.sh".into(),
495 api: "https://api.g1t.sh".into(),
496 runner: "rnr_1".into(),
497 name: "a".into(),
498 credential: "g1tr_secret".into(),
499 workspace: "acme".into(),
500 repo: None,
501 group: None,
502 labels: vec![],
503 ephemeral: false,
504 work_dir: PathBuf::from("/tmp/w"),
505 docker: true,
506 image: None,
507 agent_image: None,
508 harness: None,
509 auto_update: false,
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.510 memory: None,
511 cpus: None,
512 pids_limit: None,
513 job_docker: true,
514 job_docker_image: None,
Fast pages, required checks on the branch, self-hosted runners, honest incidents515 }
516 }
517
518 fn assignment(kind: &str) -> Assignment {
519 serde_json::from_value(json!({
520 "kind": kind, "id": "job_1", "name": "build", "repo": "acme/web", "timeout_minutes": 60,
521 "token": "jobtoken", "env": { "MODE": "checks", "G1T_TOKEN": "g1t_run", "G1T_API": "https://elsewhere" }
522 }))
523 .unwrap()
524 }
525
526 #[test]
527 fn a_workflow_job_gets_its_own_token_and_never_the_runners() {
528 let env = harness_env(&config(), &assignment("workflow")).unwrap();
529 assert_eq!(env["MODE"], "actions");
530 assert_eq!(env["ACTIONS_JOB"], "job_1");
531 assert_eq!(env["ACTIONS_TOKEN"], "jobtoken");
532 assert_eq!(env["G1T_API"], "https://api.g1t.sh");
533 assert!(env.values().all(|value| !value.contains("g1tr_")));
534 }
535
536 #[test]
537 fn agent_work_gets_the_sandboxs_environment_and_this_api() {
538 let env = harness_env(&config(), &assignment("agent")).unwrap();
539 assert_eq!(env["MODE"], "checks");
540 assert_eq!(env["G1T_TOKEN"], "g1t_run");
541 assert_eq!(env["G1T_API"], "https://api.g1t.sh");
542 assert_eq!(env["G1T_ABUSE"], "off");
543 }
544
545 #[test]
546 fn unknown_work_is_refused() {
547 assert!(harness_env(&config(), &assignment("deploy")).is_err());
548 }
549
550 #[test]
551 fn container_names_are_dockers() {
552 assert_eq!(container_name("job_01ab"), "g1t-job_01ab");
553 assert_eq!(container_name("a/b c"), "g1t-a-b-c");
554 }
Merge branch 'claude/cool-newton-7nm8r9' (#36): real GitHub Actions on g1t, managed deploy workflows, g1t/deploy and the Sysbox runner 0.2.0. The merge keeps both sides' webhook events and API operations, says agent images need tar, guards the job engine's stop to Unix so the runner builds on Windows again, and keeps the moves tests' previews from going idle as the calendar passes their fixture's date.555
556 fn args_for(kind: &str, engine_args: Vec<String>) -> (Vec<String>, BTreeMap<String, String>) {
557 let config = config();
558 let env = harness_env(&config, &assignment(kind)).unwrap();
559 let limits = Limits { memory: "6144m".into(), cpus: "4".into(), pids: 4096 };
560 let harness = PathBuf::from("/usr/local/bin/g1t-runner");
561 let plan = Plan {
562 kind,
563 name: "g1t-job_1",
564 runner: "rnr_1",
565 image: DEFAULT_IMAGE,
566 harness: &harness,
567 network: "g1t-job_1",
568 volume: (kind == "workflow").then_some("g1t-job_1-work"),
569 limits: &limits,
570 env_names: env.keys().map(String::as_str).collect(),
571 engine_args,
572 };
573 (run_args(&plan), env.clone())
574 }
575
576 fn value_of<'a>(args: &'a [String], flag: &str) -> Vec<&'a str> {
577 args.windows(2).filter(|w| w[0] == flag).map(|w| w[1].as_str()).collect()
578 }
579
580 #[test]
581 fn a_jobs_container_is_walled_in() {
582 let (args, _) = args_for("workflow", Vec::new());
583 assert_eq!(args[0], "run");
584 assert!(args.contains(&"--rm".to_owned()));
585 assert_eq!(value_of(&args, "--cap-drop"), vec!["ALL"]);
586 assert_eq!(value_of(&args, "--cap-add"), JOB_CAPS.to_vec());
587 assert_eq!(value_of(&args, "--security-opt"), vec!["no-new-privileges"]);
588 assert_eq!(value_of(&args, "--pids-limit"), vec!["4096"]);
589 assert_eq!(value_of(&args, "--memory"), vec!["6144m"]);
590 assert_eq!(value_of(&args, "--memory-swap"), vec!["6144m"]);
591 assert_eq!(value_of(&args, "--cpus"), vec!["4"]);
592 assert_eq!(value_of(&args, "--network"), vec!["g1t-job_1"]);
593 for forbidden in ["--privileged", "--pid", "--userns", "--device", "host"] {
594 assert!(!args.iter().any(|a| a == forbidden), "{forbidden} in {args:?}");
595 }
596 assert!(!JOB_CAPS.iter().any(|c| c.starts_with("SYS_") || *c == "NET_ADMIN" || *c == "NET_RAW"));
597 }
598
599 #[test]
600 fn a_jobs_only_mounts_are_the_harness_and_its_own_volume() {
601 let (args, _) = args_for("workflow", Vec::new());
602 let mounts = value_of(&args, "-v");
603 assert_eq!(mounts, vec!["/usr/local/bin/g1t-runner:/opt/g1t/g1t-runner:ro", "g1t-job_1-work:/home/runner"]);
604 assert!(!args.iter().any(|a| a.contains("docker.sock")), "no engine socket: {args:?}");
605 assert!(!args.iter().any(|a| a == "--mount"));
606 }
607
608 #[test]
609 fn credentials_go_by_name_and_the_runners_never() {
610 let (args, env) = args_for("workflow", Vec::new());
611 let passed = value_of(&args, "-e");
612 assert!(passed.contains(&"ACTIONS_TOKEN"));
613 // Never a value on the command line, and never the runner's credential.
614 assert!(!args.iter().any(|a| a.contains("jobtoken")));
615 assert!(!args.iter().any(|a| a.contains("g1tr_")));
616 assert!(env.values().all(|v| !v.contains("g1tr_secret")));
617 assert!(!env.contains_key("G1T_RUNNER_TOKEN"));
618 }
619
620 #[test]
621 fn agent_work_keeps_sudo_for_its_guardrails_and_every_other_wall() {
622 let (args, _) = args_for("agent", Vec::new());
623 assert!(value_of(&args, "--security-opt").is_empty());
624 assert!(value_of(&args, "--cap-add").contains(&"AUDIT_WRITE"));
625 assert_eq!(value_of(&args, "--cap-drop"), vec!["ALL"]);
626 assert_eq!(value_of(&args, "-v"), vec!["/usr/local/bin/g1t-runner:/opt/g1t/g1t-runner:ro"]);
627 }
628
629 #[test]
630 fn a_job_with_docker_gets_a_socket_of_its_own_not_the_runners() {
631 let engine_args = vec!["-v".into(), "/run/g1t-runner/jobs/g1t-job_1/proxy:/run/g1t-docker:ro".into(), "-e".into(), "DOCKER_HOST=unix:///run/g1t-docker/docker.sock".into()];
632 let (args, _) = args_for("workflow", engine_args);
633 let mounts = value_of(&args, "-v");
634 assert!(mounts.contains(&"/run/g1t-runner/jobs/g1t-job_1/proxy:/run/g1t-docker:ro"));
635 assert!(!mounts.iter().any(|m| m.starts_with("/var/run/docker.sock") || m.starts_with("/run/g1t-runner/docker.sock")));
636 // The entrypoint and image stay last.
637 assert_eq!(&args[args.len() - 3..], ["--entrypoint", "/opt/g1t/g1t-runner", DEFAULT_IMAGE]);
638 }
639
640 #[test]
641 fn limits_default_sensibly() {
642 assert_eq!(default_memory("MemTotal: 16384000 kB\nMemFree: 1 kB\n"), Some("12000m".into()));
643 assert_eq!(default_memory(""), None);
644 let mut config = config();
645 config.memory = Some("2g".into());
646 config.pids_limit = Some(100);
647 let limits = limits(&config);
648 assert_eq!(limits.memory, "2g");
649 assert_eq!(limits.pids, 100);
650 assert!(limits.cpus.parse::<f64>().unwrap() >= 1.0);
651 }
652
653 #[test]
654 fn host_mode_refuses_a_public_repositorys_work() {
655 let mut work = assignment("workflow");
656 work.visibility = Some("public".into());
657 let refused = refuse_public_on_host(&work).unwrap_err().to_string();
658 assert!(refused.contains("acme/web is public"), "{refused}");
659 work.visibility = Some("private".into());
660 assert!(refuse_public_on_host(&work).is_ok());
661 work.visibility = None;
662 assert!(refuse_public_on_host(&work).is_ok());
663 // And refuses it before anything runs.
664 work.visibility = Some("public".into());
665 assert!(start(&config(), Isolation::Host, Path::new("/tmp"), &work).is_err());
666 }
Fast pages, required checks on the branch, self-hosted runners, honest incidents667}

This file's history is long; its oldest lines are credited to the oldest commit read.