Skip to content
Sign in

flagon-io/g1t

public
0
821 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

GitHub Actions on g1t, part two: running workflows1//! Runs one GitHub Actions job, as GitHub's runner would: its steps in
2//! order, each `run` in a shell and each `uses` as the action it names,
3//! with the `${{ }}` contexts, the `GITHUB_*` variables and files, and the
4//! workflow commands steps print. It reports every step and the log to
5//! g1t as it goes.
6//!
7//! Configuration comes from the environment: `G1T_API`, and `ACTIONS_JOB`
8//! and `ACTIONS_TOKEN`, the job and its own token. Everything else, the
9//! job's definition, its contexts and its secrets, is fetched with them.
10
A repository has its own sidebar, as settings do11mod blobs;
Merge branch 'main' into actions-toolkit-oidc-artifacts12mod containers;
GitHub Actions on g1t, part two: running workflows13mod files;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R214mod glob;
Fast pages, required checks on the branch, self-hosted runners, honest incidents15mod paths;
GitHub Actions on g1t, part two: running workflows16mod process;
17mod report;
18mod uses;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R219mod zip;
GitHub Actions on g1t, part two: running workflows20
Merge branch 'main' into actions-toolkit-oidc-artifacts21use std::collections::{BTreeMap, BTreeSet};
GitHub Actions on g1t, part two: running workflows22use std::path::{Path, PathBuf};
23use std::process::Command;
24use std::time::{Duration, Instant};
25
26use anyhow::{Context, Result};
27use g1t_actions::events::WORKSPACE;
28use g1t_actions::expr::{self, Scope, Status};
29use serde_json::{Map, Value, json};
30
31use files::StepFiles;
32use process::{Commands, Ended};
33use report::{Api, Log};
34
35const TEMP: &str = "/home/runner/_temp";
36
37/// Who is running steps: the job itself, or a composite action inside it.
38#[derive(Clone, Default)]
39pub(crate) struct Frame {
40 /// The `steps` context.
41 pub(crate) steps: Map<String, Value>,
42 /// A composite action's `inputs`, in place of the workflow's.
43 pub(crate) inputs: Option<Value>,
44 /// A composite action's folder, for `github.action_path`.
45 pub(crate) action_path: Option<String>,
46 /// Variables a composite action's caller set for its steps.
47 pub(crate) env: BTreeMap<String, String>,
48}
49
A repository has its own sidebar, as settings do50/// A step run when the job's steps are done: an action's `post`, or
51/// saving the cache.
GitHub Actions on g1t, part two: running workflows52pub(crate) struct Post {
53 pub(crate) name: String,
54 pub(crate) condition: String,
55 pub(crate) env: BTreeMap<String, String>,
A repository has its own sidebar, as settings do56 pub(crate) run: PostRun,
GitHub Actions on g1t, part two: running workflows57}
58
A repository has its own sidebar, as settings do59pub(crate) enum PostRun {
60 Node { action_dir: PathBuf, script: String },
Merge branch 'worktree-agent-a3abfcce648e87dca'61 CacheSave { key: String, paths: Vec<String>, version: String },
Merge branch 'main' into actions-toolkit-oidc-artifacts62 /// A Docker action's `post-entrypoint`.
63 Docker(containers::DockerRun),
A repository has its own sidebar, as settings do64}
65
GitHub Actions on g1t, part two: running workflows66pub(crate) struct Job {
67 pub(crate) log: Log,
68 pub(crate) spec: Value,
69 pub(crate) workspace: PathBuf,
70 pub(crate) temp: PathBuf,
71 /// This process's own variables, less its credentials, and GitHub's.
72 base_env: BTreeMap<String, String>,
73 /// Written to `GITHUB_ENV` by earlier steps.
74 added_env: BTreeMap<String, String>,
75 /// Written to `GITHUB_PATH` by earlier steps, newest first.
76 path_prepend: Vec<String>,
77 workflow_env: BTreeMap<String, String>,
78 job_env: BTreeMap<String, String>,
79 /// github, vars, secrets, inputs, matrix, needs, strategy, runner.
80 pub(crate) contexts: Map<String, Value>,
81 pub(crate) failed: bool,
82 pub(crate) posts: Vec<Post>,
83 step_names: Vec<String>,
84 deadline: Instant,
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts85 /// The running step's `timeout-minutes`, as an instant: the nearest of
86 /// the step's own and those of the steps around it (a composite
87 /// action's step inside a `uses:` step). Every process a step starts
88 /// stops by it, as every one stops by the job's deadline.
89 step_deadline: Option<Instant>,
GitHub Actions on g1t, part two: running workflows90 debug: bool,
91 /// What the last Node process left, for the step that ran it.
92 pub(crate) last_node_outputs: BTreeMap<String, String>,
93 pub(crate) last_node_state: BTreeMap<String, String>,
Merge branch 'main' into actions-toolkit-oidc-artifacts94 /// The names of the sandbox's own variables, which a container does
95 /// not get.
96 host_env: BTreeSet<String>,
97 /// Whether this job has a Docker Engine of its own (g1t's machines).
98 pub(crate) docker_hosted: bool,
99 /// The job's network, once its containers have one.
100 pub(crate) network: Option<String>,
101 /// `services:`, by their names, and their containers' names.
102 pub(crate) services: Vec<(String, String)>,
103 /// `container:`, once started.
104 pub(crate) container: Option<containers::JobContainer>,
105 /// The `job` context's `container` and `services`.
106 pub(crate) job_context: Map<String, Value>,
107 /// Docker actions' images built in this job.
108 pub(crate) built_actions: containers::Built,
GitHub Actions on g1t, part two: running workflows109}
110
111fn text_map(value: Option<&Value>) -> BTreeMap<String, String> {
112 value
113 .and_then(Value::as_object)
114 .map(|map| map.iter().map(|(k, v)| (k.clone(), expr::to_text(v))).collect())
115 .unwrap_or_default()
116}
117
118/// A step's title when it has no name, as GitHub shows it.
119fn default_title(step: &Map<String, Value>) -> String {
120 if let Some(uses) = step.get("uses").and_then(Value::as_str) {
121 return format!("Run {uses}");
122 }
123 let run = step.get("run").map(expr::to_text).unwrap_or_default();
124 let first = run.lines().find(|line| !line.trim().is_empty()).unwrap_or_default().trim();
125 format!("Run {first}")
126}
127
128impl Job {
A repository has its own sidebar, as settings do129 pub(crate) fn base_env_value(&self, name: &str) -> Option<String> {
130 self.base_env.get(name).cloned()
131 }
132
Merge branch 'main' into actions-toolkit-oidc-artifacts133 /// What earlier steps added to `PATH`, newest first.
134 pub(crate) fn path_prepend_entries(&self) -> &[String] {
135 &self.path_prepend
136 }
137
GitHub Actions on g1t, part two: running workflows138 fn status(&self) -> Status {
139 if self.failed { Status::Failure } else { Status::Success }
140 }
141
142 /// The contexts an expression in a step can use.
143 pub(crate) fn contexts_for(&self, frame: &Frame, env: &BTreeMap<String, String>) -> Map<String, Value> {
144 let mut contexts = self.contexts.clone();
145 contexts.insert("env".into(), Value::Object(env.iter().map(|(k, v)| (k.clone(), Value::String(v.clone()))).collect()));
146 contexts.insert("steps".into(), Value::Object(frame.steps.clone()));
Merge branch 'main' into actions-toolkit-oidc-artifacts147 contexts.insert("job".into(), containers::job_context(if self.failed { "failure" } else { "success" }, &self.job_context));
GitHub Actions on g1t, part two: running workflows148 if let Some(inputs) = &frame.inputs {
149 contexts.insert("inputs".into(), inputs.clone());
150 }
151 if let Some(path) = &frame.action_path
152 && let Some(github) = contexts.get_mut("github")
153 {
154 github["action_path"] = Value::String(path.clone());
155 }
156 contexts
157 }
158
159 /// Runs `f` with a scope over these contexts.
160 pub(crate) fn with_scope<T>(&self, contexts: &Map<String, Value>, f: impl FnOnce(&Scope) -> T) -> T {
161 let workspace = self.workspace.clone();
162 let hash = move |patterns: &[String]| files::hash_files(&workspace, patterns);
163 let scope = Scope {
164 contexts,
165 status: self.status(),
166 hash_files: Some(&hash),
167 };
168 f(&scope)
169 }
170
171 /// The `env` context for a step: the workflow's, the job's, what earlier
172 /// steps wrote to `GITHUB_ENV`, and the frame's.
Merge branch 'main' into actions-toolkit-oidc-artifacts173 pub(crate) fn env_context(&self, frame: &Frame) -> BTreeMap<String, String> {
GitHub Actions on g1t, part two: running workflows174 let mut env = self.added_env.clone();
175 env.extend(self.workflow_env.clone());
176 env.extend(self.job_env.clone());
177 env.extend(frame.env.clone());
178 env
179 }
180
181 /// What a process for a step is given.
182 pub(crate) fn process_env(&self, env: &BTreeMap<String, String>, files: &StepFiles) -> BTreeMap<String, String> {
183 let mut out = self.base_env.clone();
184 out.extend(env.clone());
185 for (name, value) in files.variables() {
186 out.insert(name.to_owned(), value);
187 }
188 if !self.path_prepend.is_empty() {
189 let current = out.get("PATH").cloned().unwrap_or_default();
Fast pages, required checks on the branch, self-hosted runners, honest incidents190 let separator = paths::PATH_SEPARATOR;
191 out.insert("PATH".into(), format!("{}{separator}{current}", self.path_prepend.join(separator)));
GitHub Actions on g1t, part two: running workflows192 }
193 out
194 }
195
196 /// Takes in what a step wrote to its files. Returns its outputs.
197 pub(crate) fn absorb(&mut self, files: &StepFiles, commands: &Commands) -> (BTreeMap<String, String>, BTreeMap<String, String>) {
198 let mut outputs: BTreeMap<String, String> = commands.outputs.clone();
199 match files::key_values(&StepFiles::read(&files.output)) {
200 Ok(values) => outputs.extend(values),
201 Err(problem) => self.log.line(&format!("##[error]$GITHUB_OUTPUT: {problem}")),
202 }
203 match files::key_values(&StepFiles::read(&files.env)) {
204 Ok(values) => {
205 for (name, value) in values {
206 if name.starts_with("GITHUB_") || name == "NODE_OPTIONS" {
207 self.log.line(&format!("##[warning]{name} cannot be set through $GITHUB_ENV."));
208 continue;
209 }
210 self.added_env.insert(name, value);
211 }
212 }
213 Err(problem) => self.log.line(&format!("##[error]$GITHUB_ENV: {problem}")),
214 }
215 for line in StepFiles::read(&files.path).lines().map(str::trim).filter(|l| !l.is_empty()) {
216 self.path_prepend.insert(0, line.to_owned());
217 }
218 let mut state = commands.state.clone();
219 if let Ok(values) = files::key_values(&StepFiles::read(&files.state)) {
220 state.extend(values);
221 }
222 let summary = StepFiles::read(&files.summary);
223 if !summary.trim().is_empty() {
224 self.log.line("##[group]Step summary");
225 for line in summary.lines() {
226 self.log.line(line);
227 }
228 self.log.line("##[endgroup]");
229 }
230 (outputs, state)
231 }
232
233 pub(crate) fn remaining_time(&self) -> Duration {
234 self.remaining()
235 }
236
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts237 /// What is left for the running step: its `timeout-minutes`, within
238 /// the job's.
GitHub Actions on g1t, part two: running workflows239 fn remaining(&self) -> Duration {
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts240 let until = self.step_deadline.map_or(self.deadline, |step| step.min(self.deadline));
241 until.saturating_duration_since(Instant::now())
242 }
243
244 /// What is left of the job's own time, whatever the step's.
245 fn job_remaining(&self) -> Duration {
GitHub Actions on g1t, part two: running workflows246 self.deadline.saturating_duration_since(Instant::now())
247 }
248
249 /// Runs a shell script for a `run` step.
250 pub(crate) fn run_script(
251 &mut self,
252 script: &str,
253 shell: Option<&str>,
254 working_directory: Option<&str>,
255 env: &BTreeMap<String, String>,
256 timeout: Duration,
257 ) -> (bool, BTreeMap<String, String>, BTreeMap<String, String>) {
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look258 crate::abuse::touch();
259 // Mining is never a workflow's job (abuse.rs).
260 if let Some(miner) = crate::abuse::miner_in(script) {
261 self.log.line(&format!("##[error]g1t does not run cryptocurrency miners ({miner}). This step was not run."));
262 return (false, BTreeMap::new(), BTreeMap::new());
263 }
GitHub Actions on g1t, part two: running workflows264 let id = format!("{:x}", rand_id());
265 let shell = shell.map(str::trim).filter(|s| !s.is_empty());
266 let (program, args, extension): (String, Vec<String>, &str) = match shell {
Fast pages, required checks on the branch, self-hosted runners, honest incidents267 // A self-hosted Windows runner, as GitHub's: PowerShell.
268 None if cfg!(windows) => (windows_powershell(), powershell_args(), "ps1"),
Merge branch 'main' into actions-toolkit-oidc-artifacts269 // A job container without bash, as GitHub's runner does.
270 None if self.container.as_ref().is_some_and(|c| c.shell == "sh") => ("sh".into(), vec!["-e".into(), "{0}".into()], "sh"),
GitHub Actions on g1t, part two: running workflows271 None => ("bash".into(), vec!["-e".into(), "{0}".into()], "sh"),
272 Some("bash") => ("bash".into(), vec!["--noprofile".into(), "--norc".into(), "-eo".into(), "pipefail".into(), "{0}".into()], "sh"),
273 Some("sh") => ("sh".into(), vec!["-e".into(), "{0}".into()], "sh"),
274 Some("python") => ("python3".into(), vec!["{0}".into()], "py"),
Fast pages, required checks on the branch, self-hosted runners, honest incidents275 Some("pwsh") if cfg!(windows) || program_exists("pwsh") => ("pwsh".into(), powershell_args(), "ps1"),
276 Some("powershell") if cfg!(windows) => ("powershell".into(), powershell_args(), "ps1"),
277 Some("cmd") if cfg!(windows) => (
278 "cmd".into(),
279 vec!["/D".into(), "/E:ON".into(), "/V:OFF".into(), "/S".into(), "/C".into(), "CALL \"{0}\"".into()],
280 "cmd",
281 ),
GitHub Actions on g1t, part two: running workflows282 Some(other @ ("pwsh" | "powershell" | "cmd")) => {
Fast pages, required checks on the branch, self-hosted runners, honest incidents283 self.log.line(&format!(
284 "##[error]`shell: {other}` needs Windows or PowerShell, which g1t's Linux runners do not have. A self-hosted Windows runner can run it: `runs-on: [self-hosted, windows]`."
285 ));
GitHub Actions on g1t, part two: running workflows286 return (false, BTreeMap::new(), BTreeMap::new());
287 }
288 Some(custom) => {
289 let mut parts = custom.split_whitespace().map(str::to_owned);
290 let program = parts.next().unwrap_or_default();
291 let mut args: Vec<String> = parts.collect();
292 if !args.iter().any(|a| a.contains("{0}")) {
293 args.push("{0}".into());
294 }
295 (program, args, "sh")
296 }
297 };
298 let script_path = self.temp.join(format!("{id}.{extension}"));
299 if let Err(error) = std::fs::write(&script_path, script) {
300 self.log.line(&format!("##[error]Could not write the script: {error}"));
301 return (false, BTreeMap::new(), BTreeMap::new());
302 }
303 let files = match StepFiles::new(&self.temp, &id) {
304 Ok(files) => files,
305 Err(error) => {
306 self.log.line(&format!("##[error]Could not make the step's files: {error}"));
307 return (false, BTreeMap::new(), BTreeMap::new());
308 }
309 };
Fast pages, required checks on the branch, self-hosted runners, honest incidents310 let args: Vec<String> = args.iter().map(|a| a.replace("{0}", &paths::shown(&script_path))).collect();
GitHub Actions on g1t, part two: running workflows311 self.log.line(&format!("shell: {program} {}", args.join(" ")));
312 let dir = match working_directory {
313 Some(dir) if Path::new(dir).is_absolute() => PathBuf::from(dir),
314 Some(dir) => self.workspace.join(dir),
315 None => self.workspace.clone(),
316 };
Merge branch 'main' into actions-toolkit-oidc-artifacts317 let full = self.process_env(env, &files);
318 let in_container = self.container.as_ref().map(|c| c.path.clone()).and_then(|image_path| {
319 let inside = self.container_env(env, full.clone(), &image_path);
320 self.in_container(&program, &args, &dir, inside)
321 });
322 let command = match in_container {
323 Some(command) => command,
324 None => {
325 let mut command = Command::new(&program);
326 command.args(&args).current_dir(&dir).env_clear().envs(full);
327 command
328 }
329 };
GitHub Actions on g1t, part two: running workflows330 let mut commands = Commands {
331 debug: self.debug,
332 ..Commands::default()
333 };
334 let ended = process::run(command, timeout.min(self.remaining()), &mut self.log, &mut commands);
335 let ok = match ended {
336 Ok(Ended::Exited(0)) => true,
337 Ok(Ended::Exited(code)) => {
338 self.log.line(&format!("##[error]Process completed with exit code {code}."));
339 false
340 }
341 Ok(Ended::TimedOut) => {
342 self.log.line("##[error]The step ran past its time limit and was stopped.");
343 false
344 }
345 Err(error) => {
346 self.log.line(&format!("##[error]{program} could not be started: {error}"));
347 false
348 }
349 };
350 let (outputs, state) = self.absorb(&files, &commands);
351 (ok, outputs, state)
352 }
353
354 /// Runs one step of a frame. Returns whether it succeeded (its
355 /// conclusion). `number` is the step the log belongs to.
356 pub(crate) fn step(&mut self, frame: &mut Frame, step: &Map<String, Value>, number: u32, report: bool, defaults: &Map<String, Value>) -> bool {
357 let env_before = self.env_context(frame);
358 let contexts = self.contexts_for(frame, &env_before);
359 let title = match step.get("name").map(expr::to_text) {
360 Some(name) => self.with_scope(&contexts, |scope| expr::interpolate(&name, scope)).unwrap_or(name),
361 None => default_title(step),
362 };
363 let condition = step.get("if").map(expr::to_text).unwrap_or_default();
364 let run_it = match self.with_scope(&contexts, |scope| expr::condition(&condition, scope)) {
365 Ok(run_it) => run_it,
366 Err(problem) => {
367 self.log.line(&format!("##[error]The step's `if` does not read: {problem}"));
368 self.failed = true;
369 if report {
370 self.log.step_state(number, &title, "completed", Some("failure"));
371 }
372 return false;
373 }
374 };
375 let id = step.get("id").map(expr::to_text);
376 if !run_it {
377 if let Some(id) = &id {
378 frame.steps.insert(id.clone(), json!({ "outputs": {}, "outcome": "skipped", "conclusion": "skipped" }));
379 }
380 if report {
381 self.log.step_state(number, &title, "completed", Some("skipped"));
382 }
383 return true;
384 }
385 if report {
386 self.log.step(number);
387 self.log.step_state(number, &title, "in_progress", None);
388 }
389
390 // The step's own env, read with the contexts before it.
391 let mut env = env_before.clone();
392 if let Some(Value::Object(step_env)) = step.get("env") {
393 for (name, value) in step_env {
394 let value = self.with_scope(&contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
395 env.insert(name.clone(), expr::to_text(&value));
396 }
397 }
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts398 let minutes = step
GitHub Actions on g1t, part two: running workflows399 .get("timeout-minutes")
400 .and_then(|v| self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).ok())
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts401 .and_then(|v| v.as_f64().or_else(|| expr::to_text(&v).parse().ok()));
402 let timeout = step_timeout(minutes);
403 // Every step stops at its own `timeout-minutes`, `run` or `uses`: a
404 // `uses:` step's action, and every process it starts, included.
405 let outer_deadline = self.step_deadline;
406 self.step_deadline = step_deadline(outer_deadline, Instant::now(), minutes);
GitHub Actions on g1t, part two: running workflows407 let continue_on_error = step
408 .get("continue-on-error")
409 .and_then(|v| self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).ok())
410 .is_some_and(|v| expr::truthy(&v));
411
412 let (ok, outputs) = if let Some(run) = step.get("run").map(expr::to_text) {
413 let script = match self.with_scope(&contexts, |scope| expr::interpolate(&run, scope)) {
414 Ok(script) => script,
415 Err(problem) => {
416 self.log.line(&format!("##[error]The script does not read: {problem}"));
417 String::new()
418 }
419 };
420 self.log.line(&format!("##[group]{title}"));
421 for line in script.lines() {
422 self.log.line(line);
423 }
424 self.log.line("##[endgroup]");
425 let shell = step
426 .get("shell")
427 .map(expr::to_text)
428 .or_else(|| defaults.get("shell").map(expr::to_text));
429 if frame.action_path.is_some() && shell.is_none() {
430 self.log.line("##[error]A composite action's `run` steps need a `shell`.");
431 (false, BTreeMap::new())
432 } else {
433 let working_directory = step
434 .get("working-directory")
435 .or_else(|| defaults.get("working-directory"))
436 .map(|v| self.with_scope(&contexts, |scope| expr::interpolate(&expr::to_text(v), scope)).unwrap_or_else(|_| expr::to_text(v)));
437 let mut env = env;
438 if let Some(path) = &frame.action_path {
439 env.insert("GITHUB_ACTION_PATH".into(), path.clone());
440 }
441 let (ok, outputs, _) = self.run_script(&script, shell.as_deref(), working_directory.as_deref(), &env, timeout);
442 (ok, outputs)
443 }
444 } else if let Some(uses) = step.get("uses").map(expr::to_text) {
445 let with: BTreeMap<String, String> = match step.get("with") {
446 Some(Value::Object(with)) => with
447 .iter()
448 .map(|(k, v)| {
449 let value = self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).unwrap_or(Value::Null);
450 (k.clone(), expr::to_text(&value))
451 })
452 .collect(),
453 _ => BTreeMap::new(),
454 };
455 self.uses(&uses, &with, &env, frame, &title, id.as_deref(), timeout)
456 } else {
457 self.log.line("##[error]A step needs `run` or `uses`.");
458 (false, BTreeMap::new())
459 };
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts460 let own_deadline = self.step_deadline.filter(|_| self.step_deadline != outer_deadline);
461 self.step_deadline = outer_deadline;
462 let ok = if own_deadline.is_some_and(|until| Instant::now() >= until) {
463 self.log.line(&format!(
464 "##[error]The step ran past its timeout-minutes ({}) and was stopped.",
465 minutes.unwrap_or_default()
466 ));
467 false
468 } else {
469 ok
470 };
GitHub Actions on g1t, part two: running workflows471
472 let outcome = if ok { "success" } else { "failure" };
473 let conclusion = if ok || continue_on_error { "success" } else { "failure" };
474 if !ok && continue_on_error {
475 self.log.line("##[warning]The step failed, and `continue-on-error` lets the job go on.");
476 }
477 if let Some(id) = &id {
478 let outputs: Map<String, Value> = outputs.iter().map(|(k, v)| (k.clone(), Value::String(v.clone()))).collect();
479 frame.steps.insert(id.clone(), json!({ "outputs": outputs, "outcome": outcome, "conclusion": conclusion }));
480 }
481 if conclusion == "failure" {
482 self.failed = true;
483 }
484 if report {
485 self.log.step_state(number, &title, "completed", Some(conclusion));
486 }
487 conclusion == "success"
488 }
489
490 fn report_steps(&self) {
491 self.log.steps(&self.step_names);
492 }
493}
494
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts495/// How long a step's processes may run: its `timeout-minutes`, or six
496/// hours (the job's own limit still applies).
497fn step_timeout(minutes: Option<f64>) -> Duration {
498 match minutes {
499 Some(minutes) if minutes.is_finite() && minutes > 0.0 => Duration::from_secs_f64(minutes * 60.0),
500 Some(_) => Duration::ZERO,
501 None => Duration::from_secs(6 * 3600),
502 }
503}
504
505/// When a step must end by: its `timeout-minutes` from `now`, or `outer`
506/// (the deadline of the `uses:` step it runs inside, for a composite
507/// action's step), whichever is nearer. None when neither sets one.
508fn step_deadline(outer: Option<Instant>, now: Instant, minutes: Option<f64>) -> Option<Instant> {
509 let own = minutes.map(|minutes| now + step_timeout(Some(minutes)));
510 match (outer, own) {
511 (Some(outer), Some(own)) => Some(outer.min(own)),
512 (outer, own) => outer.or(own),
513 }
514}
515
Fast pages, required checks on the branch, self-hosted runners, honest incidents516/// PowerShell on Windows: `pwsh` (PowerShell 7) if it is installed, as on
517/// GitHub's Windows runners, else Windows PowerShell.
518fn windows_powershell() -> String {
519 if program_exists("pwsh") { "pwsh".into() } else { "powershell".into() }
520}
521
522/// How GitHub runs a PowerShell step: the script, stopping at the first error.
523fn powershell_args() -> Vec<String> {
524 vec!["-NoLogo".into(), "-NoProfile".into(), "-NonInteractive".into(), "-Command".into(), ". '{0}'".into()]
525}
526
527/// Whether `program` is on `PATH`.
528fn program_exists(program: &str) -> bool {
529 Command::new(program)
530 .arg(if program == "cmd" { "/C" } else { "-Version" })
531 .stdout(std::process::Stdio::null())
532 .stderr(std::process::Stdio::null())
533 .status()
534 .is_ok()
535}
536
GitHub Actions on g1t, part two: running workflows537/// An id for files, unique enough within one job.
538fn rand_id() -> u64 {
539 use std::sync::atomic::{AtomicU64, Ordering};
540 static NEXT: AtomicU64 = AtomicU64::new(1);
541 let nanos = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_nanos() as u64).unwrap_or(0);
542 nanos ^ (NEXT.fetch_add(1, Ordering::Relaxed) << 48)
543}
544
545fn interpolated_map(job: &Job, value: Option<&Value>, contexts: &Map<String, Value>) -> BTreeMap<String, String> {
546 let mut out = BTreeMap::new();
547 if let Some(Value::Object(map)) = value {
548 for (name, value) in map {
549 let value = job.with_scope(contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
550 out.insert(name.clone(), expr::to_text(&value));
551 }
552 }
553 out
554}
555
Fast pages, required checks on the branch, self-hosted runners, honest incidents556fn setup(mut spec: Value, api: Api) -> Result<Job> {
GitHub Actions on g1t, part two: running workflows557 let masks: Vec<String> = spec["masks"].as_array().map(|m| m.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect()).unwrap_or_default();
558 let log = Log::new(api, masks);
Fast pages, required checks on the branch, self-hosted runners, honest incidents559 // On a self-hosted runner's own machine, GitHub's layout lives in a
560 // folder of the runner's (paths.rs).
561 for part in ["variables", "github"] {
562 paths::relocate(&mut spec[part]);
563 }
564 paths::relocate(&mut spec["contexts"]["runner"]);
565 let workspace = paths::under_home(WORKSPACE);
566 let temp = paths::under_home(TEMP);
GitHub Actions on g1t, part two: running workflows567 std::fs::create_dir_all(&workspace).context("could not make the workspace")?;
568 std::fs::create_dir_all(&temp).context("could not make the temporary folder")?;
569 std::fs::write(temp.join("event.json"), serde_json::to_string_pretty(&spec["event"])?)?;
570
Merge branch 'main' into actions-toolkit-oidc-artifacts571 let host_env: BTreeSet<String> = std::env::vars().map(|(name, _)| name).collect();
572 // Docker of the job's own, on g1t's machines (crate::docker).
573 let docker_hosted = cfg!(target_os = "linux")
574 && std::env::var("G1T_DOCKER").as_deref() == Ok("on")
575 && spec["variables"]["RUNNER_ENVIRONMENT"].as_str() != Some("self-hosted");
GitHub Actions on g1t, part two: running workflows576 // This process's environment, less what only it should see.
577 let mut base_env: BTreeMap<String, String> =
578 std::env::vars().filter(|(name, _)| !matches!(name.as_str(), "ACTIONS_TOKEN" | "ACTIONS_JOB" | "MODE") && !name.starts_with("G1T_")).collect();
579 base_env.insert("HOME".into(), std::env::var("HOME").unwrap_or_else(|_| "/home/node".into()));
580 base_env.extend(text_map(spec.get("variables")));
Fast pages, required checks on the branch, self-hosted runners, honest incidents581 base_env.insert("GITHUB_EVENT_PATH".into(), paths::shown(&temp.join("event.json")));
GitHub Actions on g1t, part two: running workflows582
583 let mut contexts: Map<String, Value> = spec["contexts"].as_object().cloned().unwrap_or_default();
584 contexts.insert("github".into(), spec["github"].clone());
585 let debug = contexts
586 .get("secrets")
587 .and_then(|s| s.get("ACTIONS_STEP_DEBUG"))
588 .or_else(|| contexts.get("vars").and_then(|v| v.get("ACTIONS_STEP_DEBUG")))
589 .is_some_and(|v| expr::to_text(v) == "true");
590
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API591 // `timeoutMinutes` is how the API spelled it before its bodies were
592 // `snake_case`.
593 let timeout = spec["timeout_minutes"]
594 .as_u64()
595 .or_else(|| spec["timeoutMinutes"].as_u64())
596 .unwrap_or(60);
GitHub Actions on g1t, part two: running workflows597 let mut job = Job {
598 log,
599 spec,
600 workspace,
601 temp,
602 base_env,
603 added_env: BTreeMap::new(),
604 path_prepend: Vec::new(),
605 workflow_env: BTreeMap::new(),
606 job_env: BTreeMap::new(),
607 contexts,
608 failed: false,
609 posts: Vec::new(),
610 step_names: Vec::new(),
611 deadline: Instant::now() + Duration::from_secs(timeout * 60),
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts612 step_deadline: None,
GitHub Actions on g1t, part two: running workflows613 debug,
614 last_node_outputs: BTreeMap::new(),
615 last_node_state: BTreeMap::new(),
Merge branch 'main' into actions-toolkit-oidc-artifacts616 host_env,
617 docker_hosted,
618 network: None,
619 services: Vec::new(),
620 container: None,
621 job_context: Map::new(),
622 built_actions: containers::Built::new(),
GitHub Actions on g1t, part two: running workflows623 };
624
625 // The workflow's env reads github, secrets, inputs and vars; the job's
626 // also its matrix, needs and strategy.
627 let mut contexts = job.contexts.clone();
628 contexts.insert("env".into(), json!({}));
629 job.workflow_env = interpolated_map(&job, job.spec["workflow"].get("env"), &contexts);
630 contexts.insert("env".into(), Value::Object(job.workflow_env.iter().map(|(k, v)| (k.clone(), json!(v))).collect()));
631 job.job_env = interpolated_map(&job, job.spec["spec"].get("env"), &contexts);
632 Ok(job)
633}
634
635/// The job's `defaults.run`, its own over the workflow's.
636fn run_defaults(spec: &Value) -> Map<String, Value> {
637 let mut defaults = spec["workflow"]["defaults"]["run"].as_object().cloned().unwrap_or_default();
638 if let Some(own) = spec["spec"]["defaults"]["run"].as_object() {
639 defaults.extend(own.clone());
640 }
641 defaults
642}
643
644fn run_job(job: &mut Job) {
645 let steps: Vec<Map<String, Value>> = job.spec["spec"]["steps"]
646 .as_array()
647 .map(|steps| steps.iter().filter_map(|s| s.as_object().cloned()).collect())
648 .unwrap_or_default();
649 let defaults = run_defaults(&job.spec);
650
651 // Step names as they read before anything has run.
652 let frame = Frame::default();
653 let env = job.env_context(&frame);
654 let contexts = job.contexts_for(&frame, &env);
655 job.step_names = steps
656 .iter()
657 .map(|step| match step.get("name").map(expr::to_text) {
658 Some(name) => job.with_scope(&contexts, |scope| expr::interpolate(&name, scope)).unwrap_or(name),
659 None => default_title(step),
660 })
661 .collect();
662 job.report_steps();
663
664 job.log.step(0);
665 job.log.line(&format!("Job: {}", job.spec["name"].as_str().unwrap_or_default()));
Fast pages, required checks on the branch, self-hosted runners, honest incidents666 let variables = &job.spec["variables"];
667 if variables["RUNNER_ENVIRONMENT"] == "self-hosted" {
668 let text = |name: &str| variables[name].as_str().unwrap_or_default().to_owned();
669 job.log.line(&format!("Runner: {}, self-hosted, {} {}", text("RUNNER_NAME"), text("RUNNER_OS"), text("RUNNER_ARCH")));
670 } else {
671 job.log.line("Runner: g1t, Linux X64 (Debian bookworm, Node 24, Python 3, Go, Rust)");
672 }
GitHub Actions on g1t, part two: running workflows673 if let Some(Value::Object(matrix)) = job.contexts.get("matrix")
674 && !matrix.is_empty()
675 {
676 job.log.line(&format!("Matrix: {}", serde_json::to_string(matrix).unwrap_or_default()));
677 }
Merge branch 'worktree-agent-a3abfcce648e87dca'678 // What its G1T_TOKEN may do, as its `permissions:` gave it.
679 if let Some(Value::Object(permissions)) = job.spec.get("permissions").cloned() {
680 job.log.line("##[group]G1T_TOKEN permissions");
681 for (name, access) in &permissions {
682 if access.as_str() != Some("none") {
683 job.log.line(&format!("{name}: {}", access.as_str().unwrap_or_default()));
684 }
685 }
686 job.log.line("##[endgroup]");
687 }
Merge branch 'main' into actions-toolkit-oidc-artifacts688 if job.docker_hosted {
689 let registry = job.contexts["github"]["server_url"].as_str().and_then(crate::docker::engine::registry_host);
690 let token = job.contexts.get("secrets").and_then(|s| s.get("G1T_TOKEN")).map(expr::to_text).filter(|t| !t.is_empty());
691 let options = crate::docker::engine::Options { registry: registry.zip(token) };
692 if let Err(problem) = crate::docker::engine::enable(options) {
693 job.log.line(&format!("##[warning]Docker is not available in this job: {problem}"));
694 job.docker_hosted = false;
695 }
696 }
697 let containers_started = job.start_containers();
GitHub Actions on g1t, part two: running workflows698 job.log.flush();
699
700 let mut frame = Frame::default();
Merge branch 'main' into actions-toolkit-oidc-artifacts701 if !containers_started {
702 job.failed = true;
703 for (index, name) in job.step_names.clone().iter().enumerate() {
704 job.log.step_state(index as u32 + 1, name, "completed", Some("skipped"));
705 }
706 }
707 for (index, step) in steps.iter().enumerate().filter(|_| containers_started) {
GitHub Actions on g1t, part two: running workflows708 job.step(&mut frame, step, index as u32 + 1, true, &defaults);
Merge branch 'main' into actions-toolkit-oidc-artifacts709 job.log_docker_notes();
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts710 if job.job_remaining().is_zero() {
GitHub Actions on g1t, part two: running workflows711 job.log.line("##[error]The job ran past its time limit.");
712 job.failed = true;
713 break;
714 }
715 }
716
717 // Post steps, last registered first.
718 let posts: Vec<Post> = std::mem::take(&mut job.posts);
719 for post in posts.into_iter().rev() {
720 let number = job.step_names.len() as u32 + 1;
721 job.step_names.push(post.name.clone());
722 job.report_steps();
723 let contexts = job.contexts_for(&frame, &job.env_context(&frame));
724 let run_it = job.with_scope(&contexts, |scope| expr::condition(&post.condition, scope)).unwrap_or(true);
725 if !run_it {
726 job.log.step_state(number, &post.name, "completed", Some("skipped"));
727 continue;
728 }
729 job.log.step(number);
730 job.log.step_state(number, &post.name, "in_progress", None);
A repository has its own sidebar, as settings do731 let ok = match &post.run {
732 PostRun::Node { action_dir, script } => job.run_node(action_dir, script, &post.env),
Merge branch 'worktree-agent-a3abfcce648e87dca'733 PostRun::CacheSave { key, paths, version } => job.cache_save(key, paths, version),
Merge branch 'main' into actions-toolkit-oidc-artifacts734 PostRun::Docker(run) => job.run_docker(run).0,
A repository has its own sidebar, as settings do735 };
GitHub Actions on g1t, part two: running workflows736 job.log.step_state(number, &post.name, "completed", Some(if ok { "success" } else { "failure" }));
737 if !ok {
738 job.failed = true;
739 }
740 }
741
Merge branch 'main' into actions-toolkit-oidc-artifacts742 // GitHub's "Stop containers": services' logs, and everything removed.
743 if job.has_containers() {
744 let number = job.step_names.len() as u32 + 1;
745 job.step_names.push("Stop containers".into());
746 job.report_steps();
747 job.log.step(number);
748 job.log.step_state(number, "Stop containers", "in_progress", None);
749 job.stop_containers();
750 job.log.step_state(number, "Stop containers", "completed", Some("success"));
751 }
752
GitHub Actions on g1t, part two: running workflows753 // The job's outputs, read now that every step has run.
754 let env = job.env_context(&frame);
755 let contexts = job.contexts_for(&frame, &env);
756 let mut outputs = Map::new();
757 if let Some(Value::Object(declared)) = job.spec["spec"].get("outputs") {
758 for (name, value) in declared {
759 let value = job.with_scope(&contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
760 outputs.insert(name.clone(), Value::String(expr::to_text(&value)));
761 }
762 }
763 let conclusion = if job.failed { "failure" } else { "success" };
764 job.log.done(conclusion, &outputs, None);
765}
766
767pub(crate) fn main() -> i32 {
768 let api = match (crate::env("G1T_API"), crate::env("ACTIONS_JOB"), crate::env("ACTIONS_TOKEN")) {
769 (Ok(base), Ok(job), Ok(token)) => Api { base, job, token },
770 _ => {
771 eprintln!("g1t-runner: G1T_API, ACTIONS_JOB and ACTIONS_TOKEN are needed");
772 return 2;
773 }
774 };
775 let spec = match api.spec() {
776 Ok(spec) => spec,
777 Err(error) => {
778 eprintln!("g1t-runner: could not fetch the job: {error:#}");
779 api.report(json!({ "kind": "done", "conclusion": "failure", "reason": format!("The runner could not fetch the job: {error}") }));
780 return 1;
781 }
782 };
783 let reporter = Api {
784 base: api.base.clone(),
785 job: api.job.clone(),
786 token: api.token.clone(),
787 };
788 let mut job = match setup(spec, reporter) {
789 Ok(job) => job,
790 Err(error) => {
791 api.report(json!({ "kind": "done", "conclusion": "failure", "reason": format!("The runner could not set up: {error:#}") }));
792 return 1;
793 }
794 };
795 run_job(&mut job);
796 if job.failed { 1 } else { 0 }
797}
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts798
799#[cfg(test)]
800mod tests {
801 use super::*;
802
803 #[test]
804 fn a_steps_timeout_is_its_own_or_the_nearer_one_around_it() {
805 let now = Instant::now();
806 // No timeout-minutes anywhere: only the job's deadline applies.
807 assert_eq!(step_deadline(None, now, None), None);
808 // A step's own, in minutes, fractions included.
809 assert_eq!(step_deadline(None, now, Some(1.5)), Some(now + Duration::from_secs(90)));
810 // A composite action's step inside a `uses:` step with its own:
811 // the nearer of the two.
812 let outer = now + Duration::from_secs(60);
813 assert_eq!(step_deadline(Some(outer), now, Some(10.0)), Some(outer));
814 assert_eq!(step_deadline(Some(outer), now, Some(0.5)), Some(now + Duration::from_secs(30)));
815 assert_eq!(step_deadline(Some(outer), now, None), Some(outer));
816 // Zero or less is a time already up.
817 assert_eq!(step_deadline(None, now, Some(0.0)), Some(now));
818 assert_eq!(step_timeout(None), Duration::from_secs(6 * 3600));
819 assert_eq!(step_timeout(Some(-1.0)), Duration::ZERO);
820 }
821}

This file's history is long; its oldest lines are credited to the oldest commit read.