Skip to content
Sign in

flagon-io/g1t

public
0
906 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

GitHub Actions on g1t, part two: running workflows1//! Runs one GitHub Actions job, as GitHub's runner would: its steps in
2//! order, each `run` in a shell and each `uses` as the action it names,
3//! with the `${{ }}` contexts, the `GITHUB_*` variables and files, and the
4//! workflow commands steps print. It reports every step and the log to
5//! g1t as it goes.
6//!
7//! Configuration comes from the environment: `G1T_API`, and `ACTIONS_JOB`
8//! and `ACTIONS_TOKEN`, the job and its own token. Everything else, the
9//! job's definition, its contexts and its secrets, is fetched with them.
10
A repository has its own sidebar, as settings do11mod blobs;
Merge branch 'main' into actions-toolkit-oidc-artifacts12mod containers;
GitHub Actions on g1t, part two: running workflows13mod files;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R214mod glob;
Fast pages, required checks on the branch, self-hosted runners, honest incidents15mod paths;
GitHub Actions on g1t, part two: running workflows16mod process;
17mod report;
18mod uses;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R219mod zip;
GitHub Actions on g1t, part two: running workflows20
Merge branch 'main' into actions-toolkit-oidc-artifacts21use std::collections::{BTreeMap, BTreeSet};
GitHub Actions on g1t, part two: running workflows22use std::path::{Path, PathBuf};
23use std::process::Command;
24use std::time::{Duration, Instant};
25
26use anyhow::{Context, Result};
27use g1t_actions::events::WORKSPACE;
28use g1t_actions::expr::{self, Scope, Status};
29use serde_json::{Map, Value, json};
30
31use files::StepFiles;
32use process::{Commands, Ended};
33use report::{Api, Log};
34
35const TEMP: &str = "/home/runner/_temp";
36
37/// Who is running steps: the job itself, or a composite action inside it.
38#[derive(Clone, Default)]
39pub(crate) struct Frame {
40 /// The `steps` context.
41 pub(crate) steps: Map<String, Value>,
42 /// A composite action's `inputs`, in place of the workflow's.
43 pub(crate) inputs: Option<Value>,
44 /// A composite action's folder, for `github.action_path`.
45 pub(crate) action_path: Option<String>,
46 /// Variables a composite action's caller set for its steps.
47 pub(crate) env: BTreeMap<String, String>,
48}
49
A repository has its own sidebar, as settings do50/// A step run when the job's steps are done: an action's `post`, or
51/// saving the cache.
GitHub Actions on g1t, part two: running workflows52pub(crate) struct Post {
53 pub(crate) name: String,
54 pub(crate) condition: String,
55 pub(crate) env: BTreeMap<String, String>,
A repository has its own sidebar, as settings do56 pub(crate) run: PostRun,
GitHub Actions on g1t, part two: running workflows57}
58
A repository has its own sidebar, as settings do59pub(crate) enum PostRun {
60 Node { action_dir: PathBuf, script: String },
Merge branch 'worktree-agent-a3abfcce648e87dca'61 CacheSave { key: String, paths: Vec<String>, version: String },
Merge branch 'main' into actions-toolkit-oidc-artifacts62 /// A Docker action's `post-entrypoint`.
63 Docker(containers::DockerRun),
A repository has its own sidebar, as settings do64}
65
GitHub Actions on g1t, part two: running workflows66pub(crate) struct Job {
67 pub(crate) log: Log,
68 pub(crate) spec: Value,
69 pub(crate) workspace: PathBuf,
70 pub(crate) temp: PathBuf,
71 /// This process's own variables, less its credentials, and GitHub's.
72 base_env: BTreeMap<String, String>,
73 /// Written to `GITHUB_ENV` by earlier steps.
74 added_env: BTreeMap<String, String>,
75 /// Written to `GITHUB_PATH` by earlier steps, newest first.
76 path_prepend: Vec<String>,
77 workflow_env: BTreeMap<String, String>,
78 job_env: BTreeMap<String, String>,
79 /// github, vars, secrets, inputs, matrix, needs, strategy, runner.
80 pub(crate) contexts: Map<String, Value>,
81 pub(crate) failed: bool,
82 pub(crate) posts: Vec<Post>,
83 step_names: Vec<String>,
84 deadline: Instant,
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts85 /// The running step's `timeout-minutes`, as an instant: the nearest of
86 /// the step's own and those of the steps around it (a composite
87 /// action's step inside a `uses:` step). Every process a step starts
88 /// stops by it, as every one stops by the job's deadline.
89 step_deadline: Option<Instant>,
GitHub Actions on g1t, part two: running workflows90 debug: bool,
91 /// What the last Node process left, for the step that ran it.
92 pub(crate) last_node_outputs: BTreeMap<String, String>,
93 pub(crate) last_node_state: BTreeMap<String, String>,
Merge branch 'main' into actions-toolkit-oidc-artifacts94 /// The names of the sandbox's own variables, which a container does
95 /// not get.
96 host_env: BTreeSet<String>,
97 /// Whether this job has a Docker Engine of its own (g1t's machines).
98 pub(crate) docker_hosted: bool,
99 /// The job's network, once its containers have one.
100 pub(crate) network: Option<String>,
101 /// `services:`, by their names, and their containers' names.
102 pub(crate) services: Vec<(String, String)>,
103 /// `container:`, once started.
104 pub(crate) container: Option<containers::JobContainer>,
105 /// The `job` context's `container` and `services`.
106 pub(crate) job_context: Map<String, Value>,
107 /// Docker actions' images built in this job.
108 pub(crate) built_actions: containers::Built,
GitHub Actions on g1t, part two: running workflows109}
110
111fn text_map(value: Option<&Value>) -> BTreeMap<String, String> {
112 value
113 .and_then(Value::as_object)
114 .map(|map| map.iter().map(|(k, v)| (k.clone(), expr::to_text(v))).collect())
115 .unwrap_or_default()
116}
117
118/// A step's title when it has no name, as GitHub shows it.
119fn default_title(step: &Map<String, Value>) -> String {
120 if let Some(uses) = step.get("uses").and_then(Value::as_str) {
121 return format!("Run {uses}");
122 }
123 let run = step.get("run").map(expr::to_text).unwrap_or_default();
124 let first = run.lines().find(|line| !line.trim().is_empty()).unwrap_or_default().trim();
125 format!("Run {first}")
126}
127
128impl Job {
A repository has its own sidebar, as settings do129 pub(crate) fn base_env_value(&self, name: &str) -> Option<String> {
130 self.base_env.get(name).cloned()
131 }
132
Merge branch 'main' into actions-toolkit-oidc-artifacts133 /// What earlier steps added to `PATH`, newest first.
134 pub(crate) fn path_prepend_entries(&self) -> &[String] {
135 &self.path_prepend
136 }
137
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)138 /// How the job is going, for `success()`, `failure()`, `cancelled()`
139 /// and `always()`: once the run is cancelled, only steps that ask for
140 /// `always()` or `cancelled()` run.
GitHub Actions on g1t, part two: running workflows141 fn status(&self) -> Status {
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)142 if report::cancelled() {
143 Status::Cancelled
144 } else if self.failed {
145 Status::Failure
146 } else {
147 Status::Success
148 }
149 }
150
151 /// `job.status`.
152 fn status_word(&self) -> &'static str {
153 if report::cancelled() {
154 "cancelled"
155 } else if self.failed {
156 "failure"
157 } else {
158 "success"
159 }
GitHub Actions on g1t, part two: running workflows160 }
161
162 /// The contexts an expression in a step can use.
163 pub(crate) fn contexts_for(&self, frame: &Frame, env: &BTreeMap<String, String>) -> Map<String, Value> {
164 let mut contexts = self.contexts.clone();
165 contexts.insert("env".into(), Value::Object(env.iter().map(|(k, v)| (k.clone(), Value::String(v.clone()))).collect()));
166 contexts.insert("steps".into(), Value::Object(frame.steps.clone()));
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)167 contexts.insert("job".into(), containers::job_context(self.status_word(), &self.job_context));
GitHub Actions on g1t, part two: running workflows168 if let Some(inputs) = &frame.inputs {
169 contexts.insert("inputs".into(), inputs.clone());
170 }
171 if let Some(path) = &frame.action_path
172 && let Some(github) = contexts.get_mut("github")
173 {
174 github["action_path"] = Value::String(path.clone());
175 }
176 contexts
177 }
178
179 /// Runs `f` with a scope over these contexts.
180 pub(crate) fn with_scope<T>(&self, contexts: &Map<String, Value>, f: impl FnOnce(&Scope) -> T) -> T {
181 let workspace = self.workspace.clone();
182 let hash = move |patterns: &[String]| files::hash_files(&workspace, patterns);
183 let scope = Scope {
184 contexts,
185 status: self.status(),
186 hash_files: Some(&hash),
187 };
188 f(&scope)
189 }
190
191 /// The `env` context for a step: the workflow's, the job's, what earlier
192 /// steps wrote to `GITHUB_ENV`, and the frame's.
Merge branch 'main' into actions-toolkit-oidc-artifacts193 pub(crate) fn env_context(&self, frame: &Frame) -> BTreeMap<String, String> {
GitHub Actions on g1t, part two: running workflows194 let mut env = self.added_env.clone();
195 env.extend(self.workflow_env.clone());
196 env.extend(self.job_env.clone());
197 env.extend(frame.env.clone());
198 env
199 }
200
201 /// What a process for a step is given.
202 pub(crate) fn process_env(&self, env: &BTreeMap<String, String>, files: &StepFiles) -> BTreeMap<String, String> {
203 let mut out = self.base_env.clone();
204 out.extend(env.clone());
205 for (name, value) in files.variables() {
206 out.insert(name.to_owned(), value);
207 }
208 if !self.path_prepend.is_empty() {
209 let current = out.get("PATH").cloned().unwrap_or_default();
Fast pages, required checks on the branch, self-hosted runners, honest incidents210 let separator = paths::PATH_SEPARATOR;
211 out.insert("PATH".into(), format!("{}{separator}{current}", self.path_prepend.join(separator)));
GitHub Actions on g1t, part two: running workflows212 }
213 out
214 }
215
216 /// Takes in what a step wrote to its files. Returns its outputs.
217 pub(crate) fn absorb(&mut self, files: &StepFiles, commands: &Commands) -> (BTreeMap<String, String>, BTreeMap<String, String>) {
218 let mut outputs: BTreeMap<String, String> = commands.outputs.clone();
219 match files::key_values(&StepFiles::read(&files.output)) {
220 Ok(values) => outputs.extend(values),
221 Err(problem) => self.log.line(&format!("##[error]$GITHUB_OUTPUT: {problem}")),
222 }
223 match files::key_values(&StepFiles::read(&files.env)) {
224 Ok(values) => {
225 for (name, value) in values {
226 if name.starts_with("GITHUB_") || name == "NODE_OPTIONS" {
227 self.log.line(&format!("##[warning]{name} cannot be set through $GITHUB_ENV."));
228 continue;
229 }
230 self.added_env.insert(name, value);
231 }
232 }
233 Err(problem) => self.log.line(&format!("##[error]$GITHUB_ENV: {problem}")),
234 }
235 for line in StepFiles::read(&files.path).lines().map(str::trim).filter(|l| !l.is_empty()) {
236 self.path_prepend.insert(0, line.to_owned());
237 }
238 let mut state = commands.state.clone();
239 if let Ok(values) = files::key_values(&StepFiles::read(&files.state)) {
240 state.extend(values);
241 }
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)242 // The step's job summary, for the run's page (report.rs).
243 self.log.summary(&StepFiles::read(&files.summary));
GitHub Actions on g1t, part two: running workflows244 (outputs, state)
245 }
246
247 pub(crate) fn remaining_time(&self) -> Duration {
248 self.remaining()
249 }
250
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts251 /// What is left for the running step: its `timeout-minutes`, within
252 /// the job's.
GitHub Actions on g1t, part two: running workflows253 fn remaining(&self) -> Duration {
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts254 let until = self.step_deadline.map_or(self.deadline, |step| step.min(self.deadline));
255 until.saturating_duration_since(Instant::now())
256 }
257
258 /// What is left of the job's own time, whatever the step's.
259 fn job_remaining(&self) -> Duration {
GitHub Actions on g1t, part two: running workflows260 self.deadline.saturating_duration_since(Instant::now())
261 }
262
263 /// Runs a shell script for a `run` step.
264 pub(crate) fn run_script(
265 &mut self,
266 script: &str,
267 shell: Option<&str>,
268 working_directory: Option<&str>,
269 env: &BTreeMap<String, String>,
270 timeout: Duration,
271 ) -> (bool, BTreeMap<String, String>, BTreeMap<String, String>) {
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look272 crate::abuse::touch();
273 // Mining is never a workflow's job (abuse.rs).
274 if let Some(miner) = crate::abuse::miner_in(script) {
275 self.log.line(&format!("##[error]g1t does not run cryptocurrency miners ({miner}). This step was not run."));
276 return (false, BTreeMap::new(), BTreeMap::new());
277 }
GitHub Actions on g1t, part two: running workflows278 let id = format!("{:x}", rand_id());
279 let shell = shell.map(str::trim).filter(|s| !s.is_empty());
280 let (program, args, extension): (String, Vec<String>, &str) = match shell {
Fast pages, required checks on the branch, self-hosted runners, honest incidents281 // A self-hosted Windows runner, as GitHub's: PowerShell.
282 None if cfg!(windows) => (windows_powershell(), powershell_args(), "ps1"),
Merge branch 'main' into actions-toolkit-oidc-artifacts283 // A job container without bash, as GitHub's runner does.
284 None if self.container.as_ref().is_some_and(|c| c.shell == "sh") => ("sh".into(), vec!["-e".into(), "{0}".into()], "sh"),
GitHub Actions on g1t, part two: running workflows285 None => ("bash".into(), vec!["-e".into(), "{0}".into()], "sh"),
286 Some("bash") => ("bash".into(), vec!["--noprofile".into(), "--norc".into(), "-eo".into(), "pipefail".into(), "{0}".into()], "sh"),
287 Some("sh") => ("sh".into(), vec!["-e".into(), "{0}".into()], "sh"),
288 Some("python") => ("python3".into(), vec!["{0}".into()], "py"),
Fast pages, required checks on the branch, self-hosted runners, honest incidents289 Some("pwsh") if cfg!(windows) || program_exists("pwsh") => ("pwsh".into(), powershell_args(), "ps1"),
290 Some("powershell") if cfg!(windows) => ("powershell".into(), powershell_args(), "ps1"),
291 Some("cmd") if cfg!(windows) => (
292 "cmd".into(),
293 vec!["/D".into(), "/E:ON".into(), "/V:OFF".into(), "/S".into(), "/C".into(), "CALL \"{0}\"".into()],
294 "cmd",
295 ),
GitHub Actions on g1t, part two: running workflows296 Some(other @ ("pwsh" | "powershell" | "cmd")) => {
Fast pages, required checks on the branch, self-hosted runners, honest incidents297 self.log.line(&format!(
298 "##[error]`shell: {other}` needs Windows or PowerShell, which g1t's Linux runners do not have. A self-hosted Windows runner can run it: `runs-on: [self-hosted, windows]`."
299 ));
GitHub Actions on g1t, part two: running workflows300 return (false, BTreeMap::new(), BTreeMap::new());
301 }
302 Some(custom) => {
303 let mut parts = custom.split_whitespace().map(str::to_owned);
304 let program = parts.next().unwrap_or_default();
305 let mut args: Vec<String> = parts.collect();
306 if !args.iter().any(|a| a.contains("{0}")) {
307 args.push("{0}".into());
308 }
309 (program, args, "sh")
310 }
311 };
312 let script_path = self.temp.join(format!("{id}.{extension}"));
313 if let Err(error) = std::fs::write(&script_path, script) {
314 self.log.line(&format!("##[error]Could not write the script: {error}"));
315 return (false, BTreeMap::new(), BTreeMap::new());
316 }
317 let files = match StepFiles::new(&self.temp, &id) {
318 Ok(files) => files,
319 Err(error) => {
320 self.log.line(&format!("##[error]Could not make the step's files: {error}"));
321 return (false, BTreeMap::new(), BTreeMap::new());
322 }
323 };
Fast pages, required checks on the branch, self-hosted runners, honest incidents324 let args: Vec<String> = args.iter().map(|a| a.replace("{0}", &paths::shown(&script_path))).collect();
GitHub Actions on g1t, part two: running workflows325 self.log.line(&format!("shell: {program} {}", args.join(" ")));
326 let dir = match working_directory {
327 Some(dir) if Path::new(dir).is_absolute() => PathBuf::from(dir),
328 Some(dir) => self.workspace.join(dir),
329 None => self.workspace.clone(),
330 };
Merge branch 'main' into actions-toolkit-oidc-artifacts331 let full = self.process_env(env, &files);
332 let in_container = self.container.as_ref().map(|c| c.path.clone()).and_then(|image_path| {
333 let inside = self.container_env(env, full.clone(), &image_path);
334 self.in_container(&program, &args, &dir, inside)
335 });
336 let command = match in_container {
337 Some(command) => command,
338 None => {
339 let mut command = Command::new(&program);
340 command.args(&args).current_dir(&dir).env_clear().envs(full);
341 command
342 }
343 };
GitHub Actions on g1t, part two: running workflows344 let mut commands = Commands {
345 debug: self.debug,
346 ..Commands::default()
347 };
348 let ended = process::run(command, timeout.min(self.remaining()), &mut self.log, &mut commands);
349 let ok = match ended {
350 Ok(Ended::Exited(0)) => true,
351 Ok(Ended::Exited(code)) => {
352 self.log.line(&format!("##[error]Process completed with exit code {code}."));
353 false
354 }
355 Ok(Ended::TimedOut) => {
356 self.log.line("##[error]The step ran past its time limit and was stopped.");
357 false
358 }
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)359 Ok(Ended::Cancelled) => false,
GitHub Actions on g1t, part two: running workflows360 Err(error) => {
361 self.log.line(&format!("##[error]{program} could not be started: {error}"));
362 false
363 }
364 };
365 let (outputs, state) = self.absorb(&files, &commands);
366 (ok, outputs, state)
367 }
368
369 /// Runs one step of a frame. Returns whether it succeeded (its
370 /// conclusion). `number` is the step the log belongs to.
371 pub(crate) fn step(&mut self, frame: &mut Frame, step: &Map<String, Value>, number: u32, report: bool, defaults: &Map<String, Value>) -> bool {
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)372 // A cancellation heard during an earlier step (which it stopped) or
373 // since: from here on, only cleanup steps run, and none is stopped
374 // for it again.
375 if report::cancelled() {
376 report::take_cancel();
377 }
GitHub Actions on g1t, part two: running workflows378 let env_before = self.env_context(frame);
379 let contexts = self.contexts_for(frame, &env_before);
380 let title = match step.get("name").map(expr::to_text) {
381 Some(name) => self.with_scope(&contexts, |scope| expr::interpolate(&name, scope)).unwrap_or(name),
382 None => default_title(step),
383 };
384 let condition = step.get("if").map(expr::to_text).unwrap_or_default();
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)385 let read = self.with_scope(&contexts, |scope| expr::condition(&condition, scope));
386 // Debug logging: how the step's `if` read, as GitHub's runner says.
387 if self.debug {
388 let shown = if condition.trim().is_empty() { "success()" } else { condition.trim() };
389 self.log.line(&format!("##[debug]Evaluating condition for step: '{title}'"));
390 self.log.line(&format!("##[debug]Evaluating: {shown}"));
391 match &read {
392 Ok(result) => self.log.line(&format!("##[debug]Result: {result}")),
393 Err(problem) => self.log.line(&format!("##[debug]Failed: {problem}")),
394 }
395 }
396 let run_it = match read {
GitHub Actions on g1t, part two: running workflows397 Ok(run_it) => run_it,
398 Err(problem) => {
399 self.log.line(&format!("##[error]The step's `if` does not read: {problem}"));
400 self.failed = true;
401 if report {
402 self.log.step_state(number, &title, "completed", Some("failure"));
403 }
404 return false;
405 }
406 };
407 let id = step.get("id").map(expr::to_text);
408 if !run_it {
409 if let Some(id) = &id {
410 frame.steps.insert(id.clone(), json!({ "outputs": {}, "outcome": "skipped", "conclusion": "skipped" }));
411 }
412 if report {
413 self.log.step_state(number, &title, "completed", Some("skipped"));
414 }
415 return true;
416 }
417 if report {
418 self.log.step(number);
419 self.log.step_state(number, &title, "in_progress", None);
420 }
421
422 // The step's own env, read with the contexts before it.
423 let mut env = env_before.clone();
424 if let Some(Value::Object(step_env)) = step.get("env") {
425 for (name, value) in step_env {
426 let value = self.with_scope(&contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
427 env.insert(name.clone(), expr::to_text(&value));
428 }
429 }
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts430 let minutes = step
GitHub Actions on g1t, part two: running workflows431 .get("timeout-minutes")
432 .and_then(|v| self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).ok())
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts433 .and_then(|v| v.as_f64().or_else(|| expr::to_text(&v).parse().ok()));
434 let timeout = step_timeout(minutes);
435 // Every step stops at its own `timeout-minutes`, `run` or `uses`: a
436 // `uses:` step's action, and every process it starts, included.
437 let outer_deadline = self.step_deadline;
438 self.step_deadline = step_deadline(outer_deadline, Instant::now(), minutes);
GitHub Actions on g1t, part two: running workflows439 let continue_on_error = step
440 .get("continue-on-error")
441 .and_then(|v| self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).ok())
442 .is_some_and(|v| expr::truthy(&v));
443
444 let (ok, outputs) = if let Some(run) = step.get("run").map(expr::to_text) {
445 let script = match self.with_scope(&contexts, |scope| expr::interpolate(&run, scope)) {
446 Ok(script) => script,
447 Err(problem) => {
448 self.log.line(&format!("##[error]The script does not read: {problem}"));
449 String::new()
450 }
451 };
452 self.log.line(&format!("##[group]{title}"));
453 for line in script.lines() {
454 self.log.line(line);
455 }
456 self.log.line("##[endgroup]");
457 let shell = step
458 .get("shell")
459 .map(expr::to_text)
460 .or_else(|| defaults.get("shell").map(expr::to_text));
461 if frame.action_path.is_some() && shell.is_none() {
462 self.log.line("##[error]A composite action's `run` steps need a `shell`.");
463 (false, BTreeMap::new())
464 } else {
465 let working_directory = step
466 .get("working-directory")
467 .or_else(|| defaults.get("working-directory"))
468 .map(|v| self.with_scope(&contexts, |scope| expr::interpolate(&expr::to_text(v), scope)).unwrap_or_else(|_| expr::to_text(v)));
469 let mut env = env;
470 if let Some(path) = &frame.action_path {
471 env.insert("GITHUB_ACTION_PATH".into(), path.clone());
472 }
473 let (ok, outputs, _) = self.run_script(&script, shell.as_deref(), working_directory.as_deref(), &env, timeout);
474 (ok, outputs)
475 }
476 } else if let Some(uses) = step.get("uses").map(expr::to_text) {
477 let with: BTreeMap<String, String> = match step.get("with") {
478 Some(Value::Object(with)) => with
479 .iter()
480 .map(|(k, v)| {
481 let value = self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).unwrap_or(Value::Null);
482 (k.clone(), expr::to_text(&value))
483 })
484 .collect(),
485 _ => BTreeMap::new(),
486 };
487 self.uses(&uses, &with, &env, frame, &title, id.as_deref(), timeout)
488 } else {
489 self.log.line("##[error]A step needs `run` or `uses`.");
490 (false, BTreeMap::new())
491 };
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts492 let own_deadline = self.step_deadline.filter(|_| self.step_deadline != outer_deadline);
493 self.step_deadline = outer_deadline;
494 let ok = if own_deadline.is_some_and(|until| Instant::now() >= until) {
495 self.log.line(&format!(
496 "##[error]The step ran past its timeout-minutes ({}) and was stopped.",
497 minutes.unwrap_or_default()
498 ));
499 false
500 } else {
501 ok
502 };
GitHub Actions on g1t, part two: running workflows503
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)504 // A step the cancellation stopped ends cancelled, not failed.
505 let stopped = !ok && report::cancelled();
506 let outcome = if ok {
507 "success"
508 } else if stopped {
509 "cancelled"
510 } else {
511 "failure"
512 };
513 let conclusion = if ok || continue_on_error {
514 "success"
515 } else {
516 outcome
517 };
518 if !ok && continue_on_error && !stopped {
GitHub Actions on g1t, part two: running workflows519 self.log.line("##[warning]The step failed, and `continue-on-error` lets the job go on.");
520 }
521 if let Some(id) = &id {
522 let outputs: Map<String, Value> = outputs.iter().map(|(k, v)| (k.clone(), Value::String(v.clone()))).collect();
523 frame.steps.insert(id.clone(), json!({ "outputs": outputs, "outcome": outcome, "conclusion": conclusion }));
524 }
525 if conclusion == "failure" {
526 self.failed = true;
527 }
528 if report {
529 self.log.step_state(number, &title, "completed", Some(conclusion));
530 }
531 conclusion == "success"
532 }
533
534 fn report_steps(&self) {
535 self.log.steps(&self.step_names);
536 }
537}
538
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts539/// How long a step's processes may run: its `timeout-minutes`, or six
540/// hours (the job's own limit still applies).
541fn step_timeout(minutes: Option<f64>) -> Duration {
542 match minutes {
543 Some(minutes) if minutes.is_finite() && minutes > 0.0 => Duration::from_secs_f64(minutes * 60.0),
544 Some(_) => Duration::ZERO,
545 None => Duration::from_secs(6 * 3600),
546 }
547}
548
549/// When a step must end by: its `timeout-minutes` from `now`, or `outer`
550/// (the deadline of the `uses:` step it runs inside, for a composite
551/// action's step), whichever is nearer. None when neither sets one.
552fn step_deadline(outer: Option<Instant>, now: Instant, minutes: Option<f64>) -> Option<Instant> {
553 let own = minutes.map(|minutes| now + step_timeout(Some(minutes)));
554 match (outer, own) {
555 (Some(outer), Some(own)) => Some(outer.min(own)),
556 (outer, own) => outer.or(own),
557 }
558}
559
Fast pages, required checks on the branch, self-hosted runners, honest incidents560/// PowerShell on Windows: `pwsh` (PowerShell 7) if it is installed, as on
561/// GitHub's Windows runners, else Windows PowerShell.
562fn windows_powershell() -> String {
563 if program_exists("pwsh") { "pwsh".into() } else { "powershell".into() }
564}
565
566/// How GitHub runs a PowerShell step: the script, stopping at the first error.
567fn powershell_args() -> Vec<String> {
568 vec!["-NoLogo".into(), "-NoProfile".into(), "-NonInteractive".into(), "-Command".into(), ". '{0}'".into()]
569}
570
571/// Whether `program` is on `PATH`.
572fn program_exists(program: &str) -> bool {
573 Command::new(program)
574 .arg(if program == "cmd" { "/C" } else { "-Version" })
575 .stdout(std::process::Stdio::null())
576 .stderr(std::process::Stdio::null())
577 .status()
578 .is_ok()
579}
580
GitHub Actions on g1t, part two: running workflows581/// An id for files, unique enough within one job.
582fn rand_id() -> u64 {
583 use std::sync::atomic::{AtomicU64, Ordering};
584 static NEXT: AtomicU64 = AtomicU64::new(1);
585 let nanos = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_nanos() as u64).unwrap_or(0);
586 nanos ^ (NEXT.fetch_add(1, Ordering::Relaxed) << 48)
587}
588
589fn interpolated_map(job: &Job, value: Option<&Value>, contexts: &Map<String, Value>) -> BTreeMap<String, String> {
590 let mut out = BTreeMap::new();
591 if let Some(Value::Object(map)) = value {
592 for (name, value) in map {
593 let value = job.with_scope(contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
594 out.insert(name.clone(), expr::to_text(&value));
595 }
596 }
597 out
598}
599
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)600/// Whether `::debug::` lines are shown and steps' conditions explained:
601/// `ACTIONS_STEP_DEBUG` as a secret or a variable set to `true`, or a
602/// re-run with debug logging, which sets it and `RUNNER_DEBUG=1` among the
603/// job's variables.
604fn step_debug(contexts: &Map<String, Value>, variables: &Value) -> bool {
605 let named = |name: &str| {
606 contexts
607 .get("secrets")
608 .and_then(|s| s.get(name))
609 .or_else(|| contexts.get("vars").and_then(|v| v.get(name)))
610 .or_else(|| variables.get(name))
611 .is_some_and(|v| expr::to_text(v).eq_ignore_ascii_case("true"))
612 };
613 named("ACTIONS_STEP_DEBUG") || variables.get("RUNNER_DEBUG").is_some_and(|v| expr::to_text(v) == "1")
614}
615
Fast pages, required checks on the branch, self-hosted runners, honest incidents616fn setup(mut spec: Value, api: Api) -> Result<Job> {
GitHub Actions on g1t, part two: running workflows617 let masks: Vec<String> = spec["masks"].as_array().map(|m| m.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect()).unwrap_or_default();
618 let log = Log::new(api, masks);
Fast pages, required checks on the branch, self-hosted runners, honest incidents619 // On a self-hosted runner's own machine, GitHub's layout lives in a
620 // folder of the runner's (paths.rs).
621 for part in ["variables", "github"] {
622 paths::relocate(&mut spec[part]);
623 }
624 paths::relocate(&mut spec["contexts"]["runner"]);
625 let workspace = paths::under_home(WORKSPACE);
626 let temp = paths::under_home(TEMP);
GitHub Actions on g1t, part two: running workflows627 std::fs::create_dir_all(&workspace).context("could not make the workspace")?;
628 std::fs::create_dir_all(&temp).context("could not make the temporary folder")?;
629 std::fs::write(temp.join("event.json"), serde_json::to_string_pretty(&spec["event"])?)?;
630
Merge branch 'main' into actions-toolkit-oidc-artifacts631 let host_env: BTreeSet<String> = std::env::vars().map(|(name, _)| name).collect();
632 // Docker of the job's own, on g1t's machines (crate::docker).
633 let docker_hosted = cfg!(target_os = "linux")
634 && std::env::var("G1T_DOCKER").as_deref() == Ok("on")
635 && spec["variables"]["RUNNER_ENVIRONMENT"].as_str() != Some("self-hosted");
GitHub Actions on g1t, part two: running workflows636 // This process's environment, less what only it should see.
637 let mut base_env: BTreeMap<String, String> =
638 std::env::vars().filter(|(name, _)| !matches!(name.as_str(), "ACTIONS_TOKEN" | "ACTIONS_JOB" | "MODE") && !name.starts_with("G1T_")).collect();
639 base_env.insert("HOME".into(), std::env::var("HOME").unwrap_or_else(|_| "/home/node".into()));
640 base_env.extend(text_map(spec.get("variables")));
Fast pages, required checks on the branch, self-hosted runners, honest incidents641 base_env.insert("GITHUB_EVENT_PATH".into(), paths::shown(&temp.join("event.json")));
GitHub Actions on g1t, part two: running workflows642
643 let mut contexts: Map<String, Value> = spec["contexts"].as_object().cloned().unwrap_or_default();
644 contexts.insert("github".into(), spec["github"].clone());
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)645 let debug = step_debug(&contexts, &spec["variables"]);
GitHub Actions on g1t, part two: running workflows646
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API647 // `timeoutMinutes` is how the API spelled it before its bodies were
648 // `snake_case`.
649 let timeout = spec["timeout_minutes"]
650 .as_u64()
651 .or_else(|| spec["timeoutMinutes"].as_u64())
652 .unwrap_or(60);
GitHub Actions on g1t, part two: running workflows653 let mut job = Job {
654 log,
655 spec,
656 workspace,
657 temp,
658 base_env,
659 added_env: BTreeMap::new(),
660 path_prepend: Vec::new(),
661 workflow_env: BTreeMap::new(),
662 job_env: BTreeMap::new(),
663 contexts,
664 failed: false,
665 posts: Vec::new(),
666 step_names: Vec::new(),
667 deadline: Instant::now() + Duration::from_secs(timeout * 60),
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts668 step_deadline: None,
GitHub Actions on g1t, part two: running workflows669 debug,
670 last_node_outputs: BTreeMap::new(),
671 last_node_state: BTreeMap::new(),
Merge branch 'main' into actions-toolkit-oidc-artifacts672 host_env,
673 docker_hosted,
674 network: None,
675 services: Vec::new(),
676 container: None,
677 job_context: Map::new(),
678 built_actions: containers::Built::new(),
GitHub Actions on g1t, part two: running workflows679 };
680
681 // The workflow's env reads github, secrets, inputs and vars; the job's
682 // also its matrix, needs and strategy.
683 let mut contexts = job.contexts.clone();
684 contexts.insert("env".into(), json!({}));
685 job.workflow_env = interpolated_map(&job, job.spec["workflow"].get("env"), &contexts);
686 contexts.insert("env".into(), Value::Object(job.workflow_env.iter().map(|(k, v)| (k.clone(), json!(v))).collect()));
687 job.job_env = interpolated_map(&job, job.spec["spec"].get("env"), &contexts);
688 Ok(job)
689}
690
691/// The job's `defaults.run`, its own over the workflow's.
692fn run_defaults(spec: &Value) -> Map<String, Value> {
693 let mut defaults = spec["workflow"]["defaults"]["run"].as_object().cloned().unwrap_or_default();
694 if let Some(own) = spec["spec"]["defaults"]["run"].as_object() {
695 defaults.extend(own.clone());
696 }
697 defaults
698}
699
700fn run_job(job: &mut Job) {
701 let steps: Vec<Map<String, Value>> = job.spec["spec"]["steps"]
702 .as_array()
703 .map(|steps| steps.iter().filter_map(|s| s.as_object().cloned()).collect())
704 .unwrap_or_default();
705 let defaults = run_defaults(&job.spec);
706
707 // Step names as they read before anything has run.
708 let frame = Frame::default();
709 let env = job.env_context(&frame);
710 let contexts = job.contexts_for(&frame, &env);
711 job.step_names = steps
712 .iter()
713 .map(|step| match step.get("name").map(expr::to_text) {
714 Some(name) => job.with_scope(&contexts, |scope| expr::interpolate(&name, scope)).unwrap_or(name),
715 None => default_title(step),
716 })
717 .collect();
718 job.report_steps();
719
720 job.log.step(0);
721 job.log.line(&format!("Job: {}", job.spec["name"].as_str().unwrap_or_default()));
Fast pages, required checks on the branch, self-hosted runners, honest incidents722 let variables = &job.spec["variables"];
723 if variables["RUNNER_ENVIRONMENT"] == "self-hosted" {
724 let text = |name: &str| variables[name].as_str().unwrap_or_default().to_owned();
725 job.log.line(&format!("Runner: {}, self-hosted, {} {}", text("RUNNER_NAME"), text("RUNNER_OS"), text("RUNNER_ARCH")));
726 } else {
727 job.log.line("Runner: g1t, Linux X64 (Debian bookworm, Node 24, Python 3, Go, Rust)");
728 }
GitHub Actions on g1t, part two: running workflows729 if let Some(Value::Object(matrix)) = job.contexts.get("matrix")
730 && !matrix.is_empty()
731 {
732 job.log.line(&format!("Matrix: {}", serde_json::to_string(matrix).unwrap_or_default()));
733 }
Merge branch 'worktree-agent-a3abfcce648e87dca'734 // What its G1T_TOKEN may do, as its `permissions:` gave it.
735 if let Some(Value::Object(permissions)) = job.spec.get("permissions").cloned() {
736 job.log.line("##[group]G1T_TOKEN permissions");
737 for (name, access) in &permissions {
738 if access.as_str() != Some("none") {
739 job.log.line(&format!("{name}: {}", access.as_str().unwrap_or_default()));
740 }
741 }
742 job.log.line("##[endgroup]");
743 }
Merge branch 'main' into actions-toolkit-oidc-artifacts744 if job.docker_hosted {
745 let registry = job.contexts["github"]["server_url"].as_str().and_then(crate::docker::engine::registry_host);
746 let token = job.contexts.get("secrets").and_then(|s| s.get("G1T_TOKEN")).map(expr::to_text).filter(|t| !t.is_empty());
747 let options = crate::docker::engine::Options { registry: registry.zip(token) };
748 if let Err(problem) = crate::docker::engine::enable(options) {
749 job.log.line(&format!("##[warning]Docker is not available in this job: {problem}"));
750 job.docker_hosted = false;
751 }
752 }
753 let containers_started = job.start_containers();
GitHub Actions on g1t, part two: running workflows754 job.log.flush();
755
756 let mut frame = Frame::default();
Merge branch 'main' into actions-toolkit-oidc-artifacts757 if !containers_started {
758 job.failed = true;
759 for (index, name) in job.step_names.clone().iter().enumerate() {
760 job.log.step_state(index as u32 + 1, name, "completed", Some("skipped"));
761 }
762 }
763 for (index, step) in steps.iter().enumerate().filter(|_| containers_started) {
GitHub Actions on g1t, part two: running workflows764 job.step(&mut frame, step, index as u32 + 1, true, &defaults);
Merge branch 'main' into actions-toolkit-oidc-artifacts765 job.log_docker_notes();
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts766 if job.job_remaining().is_zero() {
GitHub Actions on g1t, part two: running workflows767 job.log.line("##[error]The job ran past its time limit.");
768 job.failed = true;
769 break;
770 }
771 }
772
773 // Post steps, last registered first.
774 let posts: Vec<Post> = std::mem::take(&mut job.posts);
775 for post in posts.into_iter().rev() {
776 let number = job.step_names.len() as u32 + 1;
777 job.step_names.push(post.name.clone());
778 job.report_steps();
779 let contexts = job.contexts_for(&frame, &job.env_context(&frame));
780 let run_it = job.with_scope(&contexts, |scope| expr::condition(&post.condition, scope)).unwrap_or(true);
781 if !run_it {
782 job.log.step_state(number, &post.name, "completed", Some("skipped"));
783 continue;
784 }
785 job.log.step(number);
786 job.log.step_state(number, &post.name, "in_progress", None);
A repository has its own sidebar, as settings do787 let ok = match &post.run {
788 PostRun::Node { action_dir, script } => job.run_node(action_dir, script, &post.env),
Merge branch 'worktree-agent-a3abfcce648e87dca'789 PostRun::CacheSave { key, paths, version } => job.cache_save(key, paths, version),
Merge branch 'main' into actions-toolkit-oidc-artifacts790 PostRun::Docker(run) => job.run_docker(run).0,
A repository has its own sidebar, as settings do791 };
GitHub Actions on g1t, part two: running workflows792 job.log.step_state(number, &post.name, "completed", Some(if ok { "success" } else { "failure" }));
793 if !ok {
794 job.failed = true;
795 }
796 }
797
Merge branch 'main' into actions-toolkit-oidc-artifacts798 // GitHub's "Stop containers": services' logs, and everything removed.
799 if job.has_containers() {
800 let number = job.step_names.len() as u32 + 1;
801 job.step_names.push("Stop containers".into());
802 job.report_steps();
803 job.log.step(number);
804 job.log.step_state(number, "Stop containers", "in_progress", None);
805 job.stop_containers();
806 job.log.step_state(number, "Stop containers", "completed", Some("success"));
807 }
808
GitHub Actions on g1t, part two: running workflows809 // The job's outputs, read now that every step has run.
810 let env = job.env_context(&frame);
811 let contexts = job.contexts_for(&frame, &env);
812 let mut outputs = Map::new();
813 if let Some(Value::Object(declared)) = job.spec["spec"].get("outputs") {
814 for (name, value) in declared {
815 let value = job.with_scope(&contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
816 outputs.insert(name.clone(), Value::String(expr::to_text(&value)));
817 }
818 }
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)819 // Cancelled, the job ends so however its cleanup went (and g1t holds
820 // to that whatever it is told).
821 let conclusion = if report::cancelled() {
822 "cancelled"
823 } else if job.failed {
824 "failure"
825 } else {
826 "success"
827 };
GitHub Actions on g1t, part two: running workflows828 job.log.done(conclusion, &outputs, None);
829}
830
831pub(crate) fn main() -> i32 {
832 let api = match (crate::env("G1T_API"), crate::env("ACTIONS_JOB"), crate::env("ACTIONS_TOKEN")) {
833 (Ok(base), Ok(job), Ok(token)) => Api { base, job, token },
834 _ => {
835 eprintln!("g1t-runner: G1T_API, ACTIONS_JOB and ACTIONS_TOKEN are needed");
836 return 2;
837 }
838 };
839 let spec = match api.spec() {
840 Ok(spec) => spec,
841 Err(error) => {
842 eprintln!("g1t-runner: could not fetch the job: {error:#}");
843 api.report(json!({ "kind": "done", "conclusion": "failure", "reason": format!("The runner could not fetch the job: {error}") }));
844 return 1;
845 }
846 };
847 let reporter = Api {
848 base: api.base.clone(),
849 job: api.job.clone(),
850 token: api.token.clone(),
851 };
852 let mut job = match setup(spec, reporter) {
853 Ok(job) => job,
854 Err(error) => {
855 api.report(json!({ "kind": "done", "conclusion": "failure", "reason": format!("The runner could not set up: {error:#}") }));
856 return 1;
857 }
858 };
859 run_job(&mut job);
Merge Actions runs: summaries, attempts and re-runs, graceful cancel, log downloads, badges (actions 0009)860 if job.failed || report::cancelled() { 1 } else { 0 }
861}
862
863#[cfg(test)]
864mod tests {
865 use serde_json::{Map, Value, json};
866
867 use super::step_debug;
868
869 #[test]
870 fn debug_logging_comes_from_a_secret_a_variable_or_a_debug_rerun() {
871 let contexts = |value: Value| -> Map<String, Value> { serde_json::from_value(value).unwrap() };
872 let none = json!({});
873 assert!(!step_debug(&contexts(json!({ "secrets": {}, "vars": {} })), &none));
874 assert!(step_debug(&contexts(json!({ "secrets": { "ACTIONS_STEP_DEBUG": "true" } })), &none));
875 assert!(step_debug(&contexts(json!({ "vars": { "ACTIONS_STEP_DEBUG": "TRUE" } })), &none));
876 assert!(!step_debug(&contexts(json!({ "vars": { "ACTIONS_STEP_DEBUG": "false" } })), &none));
877 // A re-run with debug logging sets these among the job's variables.
878 assert!(step_debug(&contexts(json!({})), &json!({ "RUNNER_DEBUG": "1" })));
879 assert!(step_debug(&contexts(json!({})), &json!({ "ACTIONS_STEP_DEBUG": "true" })));
880 assert!(!step_debug(&contexts(json!({})), &json!({ "RUNNER_DEBUG": "0" })));
881 }
GitHub Actions on g1t, part two: running workflows882}
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts883
884#[cfg(test)]
885mod tests {
886 use super::*;
887
888 #[test]
889 fn a_steps_timeout_is_its_own_or_the_nearer_one_around_it() {
890 let now = Instant::now();
891 // No timeout-minutes anywhere: only the job's deadline applies.
892 assert_eq!(step_deadline(None, now, None), None);
893 // A step's own, in minutes, fractions included.
894 assert_eq!(step_deadline(None, now, Some(1.5)), Some(now + Duration::from_secs(90)));
895 // A composite action's step inside a `uses:` step with its own:
896 // the nearer of the two.
897 let outer = now + Duration::from_secs(60);
898 assert_eq!(step_deadline(Some(outer), now, Some(10.0)), Some(outer));
899 assert_eq!(step_deadline(Some(outer), now, Some(0.5)), Some(now + Duration::from_secs(30)));
900 assert_eq!(step_deadline(Some(outer), now, None), Some(outer));
901 // Zero or less is a time already up.
902 assert_eq!(step_deadline(None, now, Some(0.0)), Some(now));
903 assert_eq!(step_timeout(None), Duration::from_secs(6 * 3600));
904 assert_eq!(step_timeout(Some(-1.0)), Duration::ZERO);
905 }
906}

This file's history is long; its oldest lines are credited to the oldest commit read.