Skip to content

Commit

Merge branch 'worktree-agent-a41ebf64a05c894fe' into claude/cool-newton-7nm8r9

Claudeagentcommitted Parents3d4403f4810c20Browse files
10 files+1260−1330/10 viewed
+1−0
11071107 "g1t-scan",
11081108 "hex",
11091109 "libc",
1110+ "regex",
11101111 "serde",
11111112 "serde_json",
11121113 "serde_yaml",
+68−5
6262 | `concurrency` with `cancel-in-progress`, for the workflow or for one job | The same: one run, or one job, of a group at a time. |
6363 | `permissions:` for the workflow or for one job, `read-all`, `write-all` | The same: they decide what [the job's token](#the-jobs-token) may do. |
6464 | `${{ }}` expressions: every operator, function and context | The same, including `hashFiles`, `success()`, `failure()`, `always()` and `cancelled()`. |
65−| `run:` with `bash`, `sh`, `python` or a custom shell | The same. |
66−| JavaScript actions (`uses: owner/repo@v7`, `owner/repo/path@v7`) | Fetched from that repository on g1t when g1t has it and your repository may use it, otherwise from GitHub, and run as they are, on Node 24, the runtime current actions declare. See [actions and workflows from other repositories](#actions-and-workflows-from-other-repositories). |
65+| `run:` with `bash`, `sh`, `python`, `pwsh` or a custom shell | The same, with GitHub's file for each: `.sh` for `bash` and `sh`, `.py` for `python`, `.ps1` for PowerShell (stopping at the first error, and exiting with the last program's code), `.cmd` for `cmd`, and no extension for another program. `python` runs `python`, or `python3` on a machine without it. |
66+| JavaScript actions (`uses: owner/repo@v7`, `owner/repo/path@v7`) | Fetched from that repository on g1t when g1t has it and your repository may use it, otherwise from GitHub, and run as they are, on Node 24, the runtime current actions declare. An action that declares `node20` runs on Node 20 where the machine has one. See [actions and workflows from other repositories](#actions-and-workflows-from-other-repositories) and [an action's pre and post](#an-actions-pre-and-post). |
6767 | Composite actions | The same. |
6868 | Reusable workflows (`jobs.<id>.uses: ./.g1t/workflows/build.yml`, or `owner/repo/.g1t/workflows/build.yml@v1` in another repository) | The same: `with:` inputs checked against their `type`, `secrets:` by name or `secrets: inherit`, `on.workflow_call` outputs, `strategy.matrix` on the calling job (one call for each combination), `concurrency` on the calling job, and up to ten levels of workflows with at most 50 different reusable workflows in a run. `.github/workflows/…` finds the workflow under `.g1t/` after the move. See [actions and workflows from other repositories](#actions-and-workflows-from-other-repositories). |
6969 | `actions/checkout` | Checks out from g1t, with `ref`, `fetch-depth`, `path`, `repository`, `token` and `submodules`. |
7070 | `GITHUB_OUTPUT`, `GITHUB_ENV`, `GITHUB_PATH`, `GITHUB_STATE`, `GITHUB_STEP_SUMMARY` | The same. Step summaries show on the run's page; see [job summaries](#job-summaries). |
71−| `::error::`, `::warning::`, `::notice::`, `::group::`, `::add-mask::` | The same: errors and warnings become annotations on the run, and [masked](#masking-secrets) values stay hidden. |
71+| `::error::`, `::warning::`, `::notice::`, `::group::`, `::add-mask::`, `::echo::on` | The same: errors and warnings become annotations on the run, with their `file`, `line`, `col`, `endLine` and `endColumn`, and [masked](#masking-secrets) values stay hidden. |
72+| Problem matchers (`::add-matcher::`, `::remove-matcher::`) | The same, so `actions/setup-node`, `setup-go` and `setup-python` turn compiler and linter output into annotations. See [problem matchers](#problem-matchers). |
7273 | `secrets.*`, `vars.*`, `secrets.GITHUB_TOKEN` | The same. `secrets.G1T_TOKEN` is [the job's own token](#the-jobs-token); `GITHUB_TOKEN` is its alias. |
7374 | `environment:` on a job | The job waits for the environment's [protection rules](#environments), then reads each key's row for that environment, as environment secrets work, and the run records a [deployment](/guides/deployments-api/#deployments-from-g1t-actions) to it. `url` gives the deployment its address; `deployment: false` reads the environment's values without making one. The name may be an expression. |
7475 | `actions/upload-artifact`, `actions/download-artifact`, `actions/upload-artifact/merge` | The same inputs and outputs as version 4: `retention-days`, `overwrite`, `compression-level`, `include-hidden-files`, `!` exclusions, download by `pattern` with `merge-multiple`, and from another run with `run-id` and `github-token`. Up to 5 GiB each; see [artifacts](#artifacts). |
379380 Jobs run in a fresh sandbox each: Debian with Node 24, Python 3, Go, Rust,
380381 Java 21, .NET 8, Ruby 3.3, `build-essential`, `git`, `curl`, `jq`, Docker
381382 (with Buildx and Compose) and passwordless `sudo`, in GitHub's layout
382−(`/home/runner/work`, `RUNNER_TEMP`, `RUNNER_TOOL_CACHE`).
383−`runner.os` is `Linux`. `ubuntu-latest`, `ubuntu-24.04` and other Linux
383+(`/home/runner/work`, `RUNNER_WORKSPACE`, `RUNNER_TEMP`, `RUNNER_TOOL_CACHE`).
384+`runner.os` is `Linux`. Each step's `GITHUB_ACTION` is its `id`, or
385+GitHub's name for it: `__run`, `__run_2` and on for `run:` steps,
386+`__actions_checkout` for `uses: actions/checkout`. `ImageOS` and
387+`ImageVersion` are set when the machine's image sets them. `ubuntu-latest`, `ubuntu-24.04` and other Linux
384388 labels all run here. A job whose `runs-on` names `self-hosted` waits for one
385389 of your [self-hosted runners](/guides/self-hosted-runners/) instead. Setup actions such as
386390 `actions/setup-node` and `actions/setup-python` install other versions as
775779 `actions/download-artifact` and `actions/upload-artifact/merge` work:
776780 g1t runs those itself.
777781
782+## An action's pre and post
783+
784+A JavaScript action's `pre` runs before the job's first step, as on
785+GitHub: each one is a step of its own, *Pre* and the step's name, in the
786+order of the steps, when its `pre-if` holds (`always()` when it has
787+none). Its `post` runs after the last step, *Post* and the step's name,
788+last first. What `pre` saves with `GITHUB_STATE` or `::save-state::`
789+reaches the step's main as `STATE_<name>`, and what either saves reaches
790+`post`.
791+
792+An action in your repository (`uses: ./…`) is not checked out when the
793+job starts, so its `pre` is not run and the step warns, as on GitHub. An
794+action inside a composite action runs its `pre` right before its main.
795+
796+A step that gives an input the action does not declare is warned of:
797+*Unexpected input(s) 'foo', valid inputs are ['token', 'path']*. The
798+step still runs.
799+
800+## Problem matchers
801+
802+`::add-matcher::path/to/matcher.json` adds the matchers in a file of
803+GitHub's format, and they read every line of output from then on, for
804+the rest of the job. A line one matches becomes an error, warning or
805+notice annotation, with the file, line, column and message it found, in
806+place of the line in the log.
807+
808+```json
809+{
810+ "problemMatcher": [
811+ {
812+ "owner": "my-linter",
813+ "pattern": [
814+ {
815+ "regexp": "^(.+):(\\d+):(\\d+): (error|warning) (.*)$",
816+ "file": 1, "line": 2, "column": 3, "severity": 4, "message": 5
817+ }
818+ ]
819+ }
820+ ]
821+}
822+```
823+
824+- A pattern of several lines matches them in order. The last one can
825+ set `loop` to match each issue after a header line, as ESLint's
826+ stylish output has.
827+- `severity` is the matcher's own default, `error` when it gives none.
828+ A severity other than `error`, `warning` or `notice` is no annotation.
829+- A file is kept when it exists in a repository in the workspace, as a
830+ path inside it; `fromPath` names the file a relative path is from.
831+- A matcher added with an owner already there takes its place.
832+ `::remove-matcher owner=my-linter::` removes it.
833+- Patterns are regular expressions as Rust's `regex` reads them, which
834+ has no lookaround or backreferences. A file whose patterns do not
835+ read is refused, with an error in the log.
836+
837+`::echo::on` shows each workflow command in the log as it is read,
838+until `::echo::off`; with [debug logging](#debug-logging) they are shown
839+from the start.
840+
778841 ## Caching Rust builds
779842
780843 A checkout gives every file a new modification time, so Cargo compiles
+36−1
104104 self.repository.split('/').next().unwrap_or_default()
105105 }
106106
107+ /// The owner's id, as the event's `repository.owner.id` gives it.
108+ fn owner_id(&self) -> String {
109+ owner_id(&self.event).unwrap_or_default()
110+ }
111+
107112 /// The `github` context for a job. `token` is `github.token` (and
108113 /// `secrets.GITHUB_TOKEN`); `job` is the job's id.
109114 pub fn context(&self, job: &str, token: &str, action: Option<&str>) -> Value {
131136 "repository": self.repository,
132137 "repository_id": self.repository_id,
133138 "repository_owner": self.owner(),
134− "repository_owner_id": "",
139+ "repository_owner_id": self.owner_id(),
135140 "repositoryUrl": format!("{}/{}.git", self.server_url, self.repository),
136141 "retention_days": 14,
137142 "run_attempt": self.run_attempt.to_string(),
174179 set("GITHUB_REPOSITORY", self.repository.clone());
175180 set("GITHUB_REPOSITORY_ID", self.repository_id.clone());
176181 set("GITHUB_REPOSITORY_OWNER", self.owner().to_owned());
182+ set("GITHUB_REPOSITORY_OWNER_ID", self.owner_id());
177183 set("GITHUB_RETENTION_DAYS", "14".into());
178184 set("GITHUB_RUN_ATTEMPT", self.run_attempt.to_string());
179185 set("GITHUB_RUN_ID", self.run_id.clone());
191197 set("RUNNER_TEMP", "/home/runner/_temp".into());
192198 set("RUNNER_TOOL_CACHE", "/home/runner/_tool".into());
193199 set("RUNNER_ENVIRONMENT", "github-hosted".into());
200+ set("RUNNER_WORKSPACE", RUNNER_WORKSPACE.into());
194201 vars
195202 }
196203 }
197204
205+/// The owner's id in a webhook-shaped payload, `repository.owner.id`, a
206+/// number or a string.
207+pub fn owner_id(event: &Value) -> Option<String> {
208+ match &event["repository"]["owner"]["id"] {
209+ Value::String(id) => Some(id.clone()),
210+ Value::Number(id) => Some(id.to_string()),
211+ _ => None,
212+ }
213+}
214+
198215 /// Where a job's repository is checked out, as on GitHub's runners.
199216 pub const WORKSPACE: &str = "/home/runner/work/repo";
200217
218+/// The folder the workspace is in, as GitHub's `RUNNER_WORKSPACE` is.
219+pub const RUNNER_WORKSPACE: &str = "/home/runner/work";
220+
201221 /// The `runner` context.
202222 /// Whether a head commit's `message` asks, as GitHub lets it, that the
203223 /// `push` and `pull_request` workflows it would start don't run: `[skip ci]`,
280300 assert_eq!(vars["GITHUB_REF_NAME"], "v1.2.0");
281301 assert_eq!(vars["GITHUB_JOB"], "build");
282302 assert_eq!(vars["RUNNER_OS"], "Linux");
303+ assert_eq!(vars["RUNNER_WORKSPACE"], "/home/runner/work");
304+ assert_eq!(vars["GITHUB_REPOSITORY_OWNER_ID"], "");
305+ }
306+
307+ #[test]
308+ fn the_owners_id_comes_from_the_event() {
309+ let info = RunInfo {
310+ repository: "acme/web".into(),
311+ event: json!({ "repository": { "owner": { "id": 42 } } }),
312+ ..RunInfo::default()
313+ };
314+ assert_eq!(info.variables("build")["GITHUB_REPOSITORY_OWNER_ID"], "42");
315+ assert_eq!(info.context("build", "", None)["repository_owner_id"], "42");
316+ assert_eq!(owner_id(&json!({ "repository": { "owner": { "id": "ws_1" } } })).as_deref(), Some("ws_1"));
317+ assert_eq!(owner_id(&json!({})), None);
283318 }
284319 }
285320
+2−0
1818 crc32fast = "1"
1919 # Checking that a release of the self-hosted runner is g1t's.
2020 ed25519-dalek = "3"
21+# Problem matchers' patterns (actions/matchers.rs).
22+regex = { version = "1.13", default-features = false, features = ["std", "perf", "unicode-perl", "unicode-case"] }
2123 anyhow = "1"
2224 base64 = "0.22"
2325 serde = { workspace = true }
+10−1
3535 pub(crate) id: String,
3636 /// `bash`, or `sh` in an image without it.
3737 pub(crate) shell: &'static str,
38+ /// `python`, or `python3` in an image without it, for `shell: python`.
39+ pub(crate) python: &'static str,
3840 /// Whether the runner's Node runs in it.
3941 pub(crate) node: bool,
4042 /// The image's own `PATH`.
419421 self.log.line("##[warning]The runner's Node does not run in this image (it needs glibc and libstdc++), so JavaScript actions run beside the container, on g1t's image, with the same files.");
420422 }
421423 self.job_context.insert("container".into(), json!({ "id": id, "network": network }));
422− self.container = Some(JobContainer { id: name, shell: if has_bash { "bash" } else { "sh" }, node, path });
424+ let has_python = self.docker_output(&["exec", &name, "sh", "-c", "command -v python"]).is_some_and(|out| !out.is_empty());
425+ self.container = Some(JobContainer {
426+ id: name,
427+ shell: if has_bash { "bash" } else { "sh" },
428+ python: if has_python { "python" } else { "python3" },
429+ node,
430+ path,
431+ });
423432 }
424433
425434 // Services with a health check are waited for.
+430−0
1+//! Problem matchers, as GitHub's runner has them. `::add-matcher::` reads a
2+//! JSON file of patterns (`actions/setup-node`, `setup-go` and
3+//! `setup-python` each add some), and from then on, for the rest of the
4+//! job, an output line they match becomes an annotation in place of the
5+//! line. A pattern can span lines, and the last one of several can `loop`
6+//! to match many issues after one header.
7+
8+use std::path::{Path, PathBuf};
9+
10+use regex::Regex;
11+use serde_json::{Map, Value};
12+
13+/// One line's pattern, and which of its groups hold what.
14+#[derive(Debug)]
15+struct Pattern {
16+ regex: Regex,
17+ file: Option<usize>,
18+ line: Option<usize>,
19+ column: Option<usize>,
20+ severity: Option<usize>,
21+ code: Option<usize>,
22+ message: Option<usize>,
23+ from_path: Option<usize>,
24+ looping: bool,
25+}
26+
27+/// What the lines matched so far say about an issue.
28+#[derive(Clone, Debug, Default, PartialEq)]
29+pub(crate) struct Found {
30+ pub(crate) file: Option<String>,
31+ pub(crate) line: Option<String>,
32+ pub(crate) column: Option<String>,
33+ pub(crate) severity: Option<String>,
34+ pub(crate) code: Option<String>,
35+ pub(crate) message: Option<String>,
36+ pub(crate) from_path: Option<String>,
37+}
38+
39+impl Found {
40+ /// `running` with what `pattern` read from `caps` over it.
41+ fn read(running: Option<&Found>, pattern: &Pattern, caps: &regex::Captures) -> Found {
42+ let mut found = running.cloned().unwrap_or_default();
43+ let group = |index: Option<usize>, into: &mut Option<String>| {
44+ if let Some(index) = index {
45+ *into = caps.get(index).map(|m| m.as_str().to_owned());
46+ }
47+ };
48+ group(pattern.file, &mut found.file);
49+ group(pattern.line, &mut found.line);
50+ group(pattern.column, &mut found.column);
51+ group(pattern.severity, &mut found.severity);
52+ group(pattern.code, &mut found.code);
53+ group(pattern.message, &mut found.message);
54+ group(pattern.from_path, &mut found.from_path);
55+ found
56+ }
57+}
58+
59+/// One matcher: its owner, the severity it gives by default, its patterns,
60+/// and how far into them the output has gone.
61+#[derive(Debug)]
62+pub(crate) struct Matcher {
63+ pub(crate) owner: String,
64+ severity: Option<String>,
65+ patterns: Vec<Pattern>,
66+ /// For each pattern but the last, what it matched on the line before.
67+ state: Vec<Option<Found>>,
68+}
69+
70+impl Matcher {
71+ fn reset(&mut self) {
72+ self.state.iter_mut().for_each(|s| *s = None);
73+ }
74+
75+ /// The issue `line` completes, if it does, as GitHub's runner reads one:
76+ /// a single pattern matches by itself; several are worked through from
77+ /// the last back, each needing the one before it to have matched the
78+ /// line before.
79+ pub(crate) fn matches(&mut self, line: &str) -> Option<Found> {
80+ let last = self.patterns.len() - 1;
81+ if last == 0 {
82+ let caps = self.patterns[0].regex.captures(line)?;
83+ return Some(self.with_default(Found::read(None, &self.patterns[0], &caps)));
84+ }
85+ for i in (0..=last).rev() {
86+ let running = if i > 0 { self.state[i - 1].clone() } else { None };
87+ if i > 0 && running.is_none() {
88+ continue;
89+ }
90+ let pattern = &self.patterns[i];
91+ match pattern.regex.captures(line) {
92+ Some(caps) if i == last => {
93+ let found = Found::read(running.as_ref(), pattern, &caps);
94+ let looping = pattern.looping;
95+ self.reset();
96+ // A loop keeps what led up to it, for the next line.
97+ if looping {
98+ self.state[i - 1] = running;
99+ }
100+ return Some(self.with_default(found));
101+ }
102+ Some(caps) => self.state[i] = Some(Found::read(running.as_ref(), pattern, &caps)),
103+ None if i == last => self.state[i - 1] = None,
104+ None => self.state[i] = None,
105+ }
106+ }
107+ None
108+ }
109+
110+ fn with_default(&self, mut found: Found) -> Found {
111+ if found.severity.as_deref().is_none_or(str::is_empty) {
112+ found.severity = self.severity.clone();
113+ }
114+ found
115+ }
116+}
117+
118+/// Reads a problem matcher file: `{"problemMatcher": [{"owner",
119+/// "severity"?, "pattern": [{"regexp", "file", "line", "column",
120+/// "severity", "code", "message", "fromPath", "loop"}]}]}`.
121+pub(crate) fn parse(text: &str) -> Result<Vec<Matcher>, String> {
122+ let config: Value = serde_json::from_str(text).map_err(|e| format!("The problem matcher file is not JSON: {e}"))?;
123+ let Some(list) = config.get("problemMatcher").and_then(Value::as_array) else {
124+ return Err("The problem matcher file has no `problemMatcher` list.".into());
125+ };
126+ list.iter().map(parse_matcher).collect()
127+}
128+
129+fn parse_matcher(config: &Value) -> Result<Matcher, String> {
130+ let owner = config.get("owner").and_then(Value::as_str).filter(|o| !o.trim().is_empty()).ok_or("A problem matcher needs an `owner`.")?;
131+ let severity = config.get("severity").and_then(Value::as_str).map(str::to_owned);
132+ let patterns: Vec<&Value> = match config.get("pattern") {
133+ Some(Value::Array(items)) => items.iter().collect(),
134+ Some(item @ Value::Object(_)) => vec![item],
135+ _ => Vec::new(),
136+ };
137+ if patterns.is_empty() {
138+ return Err(format!("The problem matcher `{owner}` has no patterns."));
139+ }
140+ let count = patterns.len();
141+ let mut read = Vec::new();
142+ for (index, pattern) in patterns.into_iter().enumerate() {
143+ let regexp = pattern.get("regexp").and_then(Value::as_str).filter(|r| !r.is_empty()).ok_or(format!("A pattern of `{owner}` needs a `regexp`."))?;
144+ let regex = Regex::new(regexp).map_err(|e| format!("A pattern of `{owner}` does not read: {e}"))?;
145+ let group = |key: &str| -> Result<Option<usize>, String> {
146+ match pattern.get(key) {
147+ None | Some(Value::Null) => Ok(None),
148+ Some(value) => value.as_u64().map(|n| Some(n as usize)).ok_or(format!("`{key}` in a pattern of `{owner}` must be a group's number.")),
149+ }
150+ };
151+ let looping = pattern.get("loop").and_then(Value::as_bool).unwrap_or(false);
152+ let read_pattern = Pattern {
153+ regex,
154+ file: group("file")?,
155+ line: group("line")?,
156+ column: group("column")?,
157+ severity: group("severity")?,
158+ code: group("code")?,
159+ message: group("message")?,
160+ from_path: group("fromPath")?,
161+ looping,
162+ };
163+ if looping && (count == 1 || index != count - 1) {
164+ return Err(format!("Only the last of several patterns may `loop` (`{owner}`)."));
165+ }
166+ if looping && read_pattern.message.is_none() {
167+ return Err(format!("The looping pattern of `{owner}` must set `message`."));
168+ }
169+ read.push(read_pattern);
170+ }
171+ if !read.iter().any(|p| p.message.is_some()) {
172+ return Err(format!("One of the patterns of `{owner}` must set `message`."));
173+ }
174+ Ok(Matcher { owner: owner.to_owned(), severity, state: vec![None; read.len() - 1], patterns: read })
175+}
176+
177+/// An issue a matcher found: the annotation's level, its message, and its
178+/// `file`, `line`, `col` and `code`, as an `::error::` command's properties.
179+#[derive(Debug, PartialEq)]
180+pub(crate) struct Issue {
181+ pub(crate) level: &'static str,
182+ pub(crate) message: String,
183+ pub(crate) properties: Map<String, Value>,
184+}
185+
186+/// The job's matchers, in the order they were added.
187+#[derive(Default)]
188+pub(crate) struct Matchers {
189+ list: Vec<Matcher>,
190+ /// Where relative files are read from: `github.workspace`.
191+ pub(crate) workspace: PathBuf,
192+}
193+
194+impl Matchers {
195+ /// Adds matchers; one with an owner already here takes its place.
196+ pub(crate) fn add(&mut self, new: Vec<Matcher>) {
197+ for matcher in new {
198+ match self.list.iter_mut().find(|m| m.owner.eq_ignore_ascii_case(&matcher.owner)) {
199+ Some(old) => *old = matcher,
200+ None => self.list.push(matcher),
201+ }
202+ }
203+ }
204+
205+ pub(crate) fn remove(&mut self, owner: &str) {
206+ self.list.retain(|m| !m.owner.eq_ignore_ascii_case(owner));
207+ }
208+
209+ /// The issue `line` makes, if a matcher reads one in it. The first
210+ /// matcher to match has it, and the others start over.
211+ pub(crate) fn issue(&mut self, line: &str) -> Option<Issue> {
212+ if self.list.is_empty() {
213+ return None;
214+ }
215+ let line = strip_colors(line);
216+ for index in 0..self.list.len() {
217+ let Some(found) = self.list[index].matches(&line) else { continue };
218+ for (other, matcher) in self.list.iter_mut().enumerate() {
219+ if other != index {
220+ matcher.reset();
221+ }
222+ }
223+ if let Some(issue) = to_issue(&found, &self.workspace) {
224+ return Some(issue);
225+ }
226+ }
227+ None
228+ }
229+}
230+
231+/// `\x1b[…m` colour codes, which a matcher never sees.
232+fn strip_colors(line: &str) -> String {
233+ let mut out = String::with_capacity(line.len());
234+ let mut rest = line;
235+ while let Some(start) = rest.find("\x1b[") {
236+ out.push_str(&rest[..start]);
237+ let after = &rest[start + 2..];
238+ let end = after.find(|c: char| !(c.is_ascii_digit() || c == ';'));
239+ match end {
240+ Some(end) if after[end..].starts_with('m') => rest = &after[end + 1..],
241+ _ => {
242+ out.push_str("\x1b[");
243+ rest = after;
244+ }
245+ }
246+ }
247+ out.push_str(rest);
248+ out
249+}
250+
251+/// What GitHub's runner makes of a match: nothing without a message or
252+/// with a severity other than error, warning or notice; a line and column
253+/// only when they are numbers; a file only when it is there, inside a
254+/// repository in the workspace, and then relative to that repository.
255+fn to_issue(found: &Found, workspace: &Path) -> Option<Issue> {
256+ let message = found.message.as_deref().filter(|m| !m.trim().is_empty())?;
257+ let level = match found.severity.as_deref().map(str::to_ascii_lowercase).as_deref() {
258+ None | Some("") | Some("error") => "error",
259+ Some("warning") => "warning",
260+ Some("notice") => "notice",
261+ Some(_) => return None,
262+ };
263+ let mut properties = Map::new();
264+ for (key, value) in [("line", &found.line), ("col", &found.column)] {
265+ if let Some(number) = value.as_deref().and_then(|v| v.trim().parse::<u32>().ok()) {
266+ properties.insert(key.into(), Value::String(number.to_string()));
267+ }
268+ }
269+ if let Some(code) = found.code.as_deref().map(str::trim).filter(|c| !c.is_empty()) {
270+ properties.insert("code".into(), Value::String(code.to_owned()));
271+ }
272+ if let Some(file) = found.file.as_deref().map(str::trim).filter(|f| !f.is_empty())
273+ && let Some(file) = repository_file(file, found.from_path.as_deref(), workspace)
274+ {
275+ properties.insert("file".into(), Value::String(file));
276+ }
277+ Some(Issue { level, message: message.to_owned(), properties })
278+}
279+
280+/// `file` as a path inside its repository, if it is in one in the workspace.
281+fn repository_file(file: &str, from_path: Option<&str>, workspace: &Path) -> Option<String> {
282+ let mut path = PathBuf::from(file.replace('\\', "/"));
283+ if !path.is_absolute()
284+ && let Some(from) = from_path.map(Path::new).and_then(Path::parent).filter(|p| !p.as_os_str().is_empty())
285+ {
286+ path = from.join(path);
287+ }
288+ if !path.is_absolute() {
289+ path = workspace.join(path);
290+ }
291+ if !path.is_file() {
292+ return None;
293+ }
294+ let repository = path.ancestors().skip(1).take_while(|dir| dir.starts_with(workspace)).find(|dir| dir.join(".git").exists())?;
295+ let relative = path.strip_prefix(repository).ok()?;
296+ let parts: Vec<String> = relative.iter().map(|part| part.to_string_lossy().into_owned()).collect();
297+ Some(parts.join("/"))
298+}
299+
300+#[cfg(test)]
301+mod tests {
302+ use super::*;
303+
304+ /// actions/setup-node's `tsc` matcher, as it ships.
305+ const TSC: &str = r#"{
306+ "problemMatcher": [{
307+ "owner": "tsc",
308+ "pattern": [{
309+ "regexp": "^([^\\s].*)[\\(:](\\d+)[,:](\\d+)(?:\\):\\s+|\\s+-\\s+)(error|warning|info)\\s+TS(\\d+)\\s*:\\s*(.*)$",
310+ "file": 1, "line": 2, "column": 3, "severity": 4, "code": 5, "message": 6
311+ }]
312+ }]
313+ }"#;
314+
315+ /// actions/setup-node's `eslint-stylish`: a file, then its issues.
316+ const STYLISH: &str = r#"{
317+ "problemMatcher": [{
318+ "owner": "eslint-stylish",
319+ "pattern": [
320+ { "regexp": "^([^\\s].*)$", "file": 1 },
321+ { "regexp": "^\\s+(\\d+):(\\d+)\\s+(error|warning|info)\\s+(.*)\\s\\s+(.*)$", "line": 1, "column": 2, "severity": 3, "message": 4, "code": 5, "loop": true }
322+ ]
323+ }]
324+ }"#;
325+
326+ #[test]
327+ fn a_single_pattern_reads_its_groups() {
328+ let mut matchers = parse(TSC).unwrap();
329+ assert_eq!(matchers[0].owner, "tsc");
330+ let found = matchers[0].matches("src/app.ts(12,5): error TS2322: Type 'string' is not assignable.").unwrap();
331+ assert_eq!(found.file.as_deref(), Some("src/app.ts"));
332+ assert_eq!(found.line.as_deref(), Some("12"));
333+ assert_eq!(found.column.as_deref(), Some("5"));
334+ assert_eq!(found.severity.as_deref(), Some("error"));
335+ assert_eq!(found.code.as_deref(), Some("2322"));
336+ assert_eq!(found.message.as_deref(), Some("Type 'string' is not assignable."));
337+ assert!(matchers[0].matches("Compiled successfully.").is_none());
338+ }
339+
340+ #[test]
341+ fn several_patterns_span_lines_and_a_loop_goes_on() {
342+ let mut matcher = parse(STYLISH).unwrap().remove(0);
343+ assert!(matcher.matches("/w/src/a.js").is_none(), "the header alone is no issue");
344+ let first = matcher.matches(" 1:10 error 'x' is defined but never used no-unused-vars").unwrap();
345+ assert_eq!(first.file.as_deref(), Some("/w/src/a.js"));
346+ assert_eq!((first.line.as_deref(), first.column.as_deref()), (Some("1"), Some("10")));
347+ assert_eq!(first.code.as_deref(), Some("no-unused-vars"));
348+ // The loop keeps the file for the next issue.
349+ let second = matcher.matches(" 3:1 warning Unexpected console statement no-console").unwrap();
350+ assert_eq!(second.file.as_deref(), Some("/w/src/a.js"));
351+ assert_eq!(second.severity.as_deref(), Some("warning"));
352+ // A line that is neither ends the run.
353+ assert!(matcher.matches("").is_none());
354+ assert!(matcher.matches(" 4:1 error Late semi").is_none(), "no file is held after the run ended");
355+ }
356+
357+ #[test]
358+ fn without_a_loop_each_issue_needs_its_header() {
359+ let config = r#"{"problemMatcher": [{"owner": "two", "severity": "warning", "pattern": [
360+ {"regexp": "^FILE (.*)$", "file": 1},
361+ {"regexp": "^MSG (.*)$", "message": 1}
362+ ]}]}"#;
363+ let mut matcher = parse(config).unwrap().remove(0);
364+ assert!(matcher.matches("MSG orphan").is_none());
365+ assert!(matcher.matches("FILE a.txt").is_none());
366+ let found = matcher.matches("MSG broke").unwrap();
367+ assert_eq!((found.file.as_deref(), found.message.as_deref()), (Some("a.txt"), Some("broke")));
368+ assert_eq!(found.severity.as_deref(), Some("warning"), "the matcher's own severity");
369+ assert!(matcher.matches("MSG again").is_none());
370+ // A line between header and message breaks the pair.
371+ matcher.matches("FILE b.txt");
372+ matcher.matches("something else");
373+ assert!(matcher.matches("MSG late").is_none());
374+ }
375+
376+ #[test]
377+ fn bad_matchers_are_refused() {
378+ assert!(parse("not json").is_err());
379+ assert!(parse(r#"{"problemMatcher": [{"pattern": [{"regexp": "x", "message": 0}]}]}"#).unwrap_err().contains("owner"));
380+ assert!(parse(r#"{"problemMatcher": [{"owner": "a", "pattern": [{"regexp": "x"}]}]}"#).unwrap_err().contains("message"));
381+ assert!(parse(r#"{"problemMatcher": [{"owner": "a", "pattern": [{"regexp": "x", "message": 0, "loop": true}]}]}"#).unwrap_err().contains("loop"));
382+ assert!(parse(r#"{"problemMatcher": [{"owner": "a", "pattern": [{"regexp": "(", "message": 1}]}]}"#).is_err());
383+ }
384+
385+ #[test]
386+ fn matchers_are_replaced_by_owner_and_removed() {
387+ let mut matchers = Matchers::default();
388+ matchers.add(parse(TSC).unwrap());
389+ matchers.add(parse(TSC).unwrap());
390+ assert_eq!(matchers.list.len(), 1);
391+ matchers.add(parse(STYLISH).unwrap());
392+ assert_eq!(matchers.list.len(), 2);
393+ matchers.remove("TSC");
394+ assert_eq!(matchers.list[0].owner, "eslint-stylish");
395+ }
396+
397+ #[test]
398+ fn issues_carry_numbers_and_files_inside_the_repository() {
399+ let workspace = std::env::temp_dir().join(format!("g1t-matchers-{}", std::process::id()));
400+ std::fs::create_dir_all(workspace.join(".git")).unwrap();
401+ std::fs::create_dir_all(workspace.join("src")).unwrap();
402+ std::fs::write(workspace.join("src/app.ts"), "").unwrap();
403+ let mut matchers = Matchers { workspace: workspace.clone(), ..Matchers::default() };
404+ matchers.add(parse(TSC).unwrap());
405+ let issue = matchers.issue("\x1b[31msrc/app.ts(12,5): error TS2322: Bad.\x1b[0m").unwrap();
406+ assert_eq!(issue.level, "error");
407+ assert_eq!(issue.message, "Bad.");
408+ assert_eq!(issue.properties["file"], "src/app.ts");
409+ assert_eq!(issue.properties["line"], "12");
410+ assert_eq!(issue.properties["col"], "5");
411+ assert_eq!(issue.properties["code"], "2322");
412+ // A file that is not there is dropped; the issue stays.
413+ let issue = matchers.issue("gone.ts(1,1): warning TS1: Missing.").unwrap();
414+ assert_eq!(issue.level, "warning");
415+ assert!(!issue.properties.contains_key("file"));
416+ // An absolute path inside the workspace is made relative.
417+ let absolute = format!("{}(2,3): error TS9: Abs.", workspace.join("src/app.ts").display());
418+ assert_eq!(matchers.issue(&absolute).unwrap().properties["file"], "src/app.ts");
419+ // `info` is no annotation GitHub has.
420+ assert!(matchers.issue("src/app.ts(1,1): info TS1: Hint.").is_none());
421+ let _ = std::fs::remove_dir_all(&workspace);
422+ }
423+
424+ #[test]
425+ fn colour_codes_are_stripped() {
426+ assert_eq!(strip_colors("\x1b[1;31merror\x1b[0m: x"), "error: x");
427+ assert_eq!(strip_colors("plain"), "plain");
428+ assert_eq!(strip_colors("\x1b[Kodd"), "\x1b[Kodd");
429+ }
430+}
+273−30
1212 mod containers;
1313 mod files;
1414 mod glob;
15+mod matchers;
1516 mod paths;
1617 mod process;
1718 mod report;
5758 }
5859
5960 pub(crate) enum PostRun {
60− Node { action_dir: PathBuf, script: String },
61+ Node { action_dir: PathBuf, script: String, using: String },
6162 CacheSave { key: String, paths: Vec<String>, version: String },
6263 /// A Docker action's `post-entrypoint`.
6364 Docker(containers::DockerRun),
9192 /// What the last Node process left, for the step that ran it.
9293 pub(crate) last_node_outputs: BTreeMap<String, String>,
9394 pub(crate) last_node_state: BTreeMap<String, String>,
95+ /// What each action's `pre` saved, by its step's `GITHUB_ACTION`, for
96+ /// the step's main and post.
97+ pub(crate) pre_state: BTreeMap<String, BTreeMap<String, String>>,
9498 /// The names of the sandbox's own variables, which a container does
9599 /// not get.
96100 host_env: BTreeSet<String>,
115119 .unwrap_or_default()
116120 }
117121
122+/// Where `name_steps` puts a step's `GITHUB_ACTION`.
123+const ACTION_NAME: &str = "__g1t_action";
124+
125+/// Gives each step its `GITHUB_ACTION`, as GitHub's runner names them: its
126+/// `id`, else `__run` for a script and `__owner_repo` for an action
127+/// (`__self` for one in the repository), the second of a name `_2`, the
128+/// third `_3`.
129+pub(crate) fn name_steps(steps: &mut [Map<String, Value>]) {
130+ let mut taken: BTreeSet<String> = steps.iter().filter_map(|step| step.get("id").map(expr::to_text)).collect();
131+ for step in steps.iter_mut() {
132+ let name = match step.get("id").map(expr::to_text) {
133+ Some(id) => id,
134+ None => {
135+ let base = match step.get("uses").map(expr::to_text) {
136+ None => "run".to_owned(),
137+ Some(uses) if uses.starts_with("./") => "self".to_owned(),
138+ Some(uses) => {
139+ let name = uses.split('@').next().unwrap_or_default();
140+ let name = name.strip_prefix("docker://").unwrap_or(name);
141+ let repository: Vec<&str> = name.splitn(3, '/').take(2).collect();
142+ repository.join("_").chars().map(|c| if c.is_ascii_alphanumeric() || c == '_' || c == '-' { c } else { '_' }).collect()
143+ }
144+ };
145+ let base = format!("__{base}");
146+ let mut name = base.clone();
147+ let mut n = 2;
148+ while taken.contains(&name) {
149+ name = format!("{base}_{n}");
150+ n += 1;
151+ }
152+ taken.insert(name.clone());
153+ name
154+ }
155+ };
156+ step.insert(ACTION_NAME.into(), Value::String(name));
157+ }
158+}
159+
118160 /// A step's title when it has no name, as GitHub shows it.
119161 fn default_title(step: &Map<String, Value>) -> String {
120162 if let Some(uses) = step.get("uses").and_then(Value::as_str) {
285327 None => ("bash".into(), vec!["-e".into(), "{0}".into()], "sh"),
286328 Some("bash") => ("bash".into(), vec!["--noprofile".into(), "--norc".into(), "-eo".into(), "pipefail".into(), "{0}".into()], "sh"),
287329 Some("sh") => ("sh".into(), vec!["-e".into(), "{0}".into()], "sh"),
288− Some("python") => ("python3".into(), vec!["{0}".into()], "py"),
330+ Some("python") => (self.python_program(), vec!["{0}".into()], "py"),
289331 Some("pwsh") if cfg!(windows) || program_exists("pwsh") => ("pwsh".into(), powershell_args(), "ps1"),
290332 Some("powershell") if cfg!(windows) => ("powershell".into(), powershell_args(), "ps1"),
291333 Some("cmd") if cfg!(windows) => (
306348 if !args.iter().any(|a| a.contains("{0}")) {
307349 args.push("{0}".into());
308350 }
309− (program, args, "sh")
351+ let extension = script_extension(&program);
352+ (program, args, extension)
310353 }
311354 };
312− let script_path = self.temp.join(format!("{id}.{extension}"));
313− if let Err(error) = std::fs::write(&script_path, script) {
355+ let script_path = self.temp.join(if extension.is_empty() { id.clone() } else { format!("{id}.{extension}") });
356+ if let Err(error) = std::fs::write(&script_path, script_contents(&program, script)) {
314357 self.log.line(&format!("##[error]Could not write the script: {error}"));
315358 return (false, BTreeMap::new(), BTreeMap::new());
316359 }
420463 }
421464
422465 // The step's own env, read with the contexts before it.
423− let mut env = env_before.clone();
424− if let Some(Value::Object(step_env)) = step.get("env") {
425− for (name, value) in step_env {
426− let value = self.with_scope(&contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
427− env.insert(name.clone(), expr::to_text(&value));
428− }
429− }
466+ let env = self.step_env(step, &contexts, env_before.clone());
467+ let action_name = step.get(ACTION_NAME).map(expr::to_text).or_else(|| id.clone()).unwrap_or_else(|| "__run".into());
430468 let minutes = step
431469 .get("timeout-minutes")
432470 .and_then(|v| self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).ok())
470508 if let Some(path) = &frame.action_path {
471509 env.insert("GITHUB_ACTION_PATH".into(), path.clone());
472510 }
511+ env.insert("GITHUB_ACTION".into(), action_name.clone());
473512 let (ok, outputs, _) = self.run_script(&script, shell.as_deref(), working_directory.as_deref(), &env, timeout);
474513 (ok, outputs)
475514 }
476515 } else if let Some(uses) = step.get("uses").map(expr::to_text) {
477− let with: BTreeMap<String, String> = match step.get("with") {
478− Some(Value::Object(with)) => with
479− .iter()
480− .map(|(k, v)| {
481− let value = self.with_scope(&contexts, |scope| expr::interpolate_value(v, scope)).unwrap_or(Value::Null);
482− (k.clone(), expr::to_text(&value))
483− })
484− .collect(),
485− _ => BTreeMap::new(),
486− };
487− self.uses(&uses, &with, &env, frame, &title, id.as_deref(), timeout)
516+ let with = self.step_with(step, &contexts);
517+ self.uses(&uses, &with, &env, frame, &title, &action_name, timeout)
488518 } else {
489519 self.log.line("##[error]A step needs `run` or `uses`.");
490520 (false, BTreeMap::new())
531561 conclusion == "success"
532562 }
533563
564+ /// A step's own `env` over `base`, read with `contexts`.
565+ fn step_env(&self, step: &Map<String, Value>, contexts: &Map<String, Value>, mut base: BTreeMap<String, String>) -> BTreeMap<String, String> {
566+ if let Some(Value::Object(step_env)) = step.get("env") {
567+ for (name, value) in step_env {
568+ let value = self.with_scope(contexts, |scope| expr::interpolate_value(value, scope)).unwrap_or(Value::Null);
569+ base.insert(name.clone(), expr::to_text(&value));
570+ }
571+ }
572+ base
573+ }
574+
575+ /// A `uses:` step's `with`, read with `contexts`.
576+ fn step_with(&self, step: &Map<String, Value>, contexts: &Map<String, Value>) -> BTreeMap<String, String> {
577+ match step.get("with") {
578+ Some(Value::Object(with)) => with
579+ .iter()
580+ .map(|(k, v)| {
581+ let value = self.with_scope(contexts, |scope| expr::interpolate_value(v, scope)).unwrap_or(Value::Null);
582+ (k.clone(), expr::to_text(&value))
583+ })
584+ .collect(),
585+ _ => BTreeMap::new(),
586+ }
587+ }
588+
534589 fn report_steps(&self) {
535590 self.log.steps(&self.step_names);
536591 }
568623 vec!["-NoLogo".into(), "-NoProfile".into(), "-NonInteractive".into(), "-Command".into(), ". '{0}'".into()]
569624 }
570625
626+/// The name of a shell's program, without its folder or `.exe`.
627+fn program_name(program: &str) -> &str {
628+ let name = program.rsplit(['/', '\\']).next().unwrap_or(program);
629+ name.strip_suffix(".exe").or_else(|| name.strip_suffix(".EXE")).unwrap_or(name)
630+}
631+
632+/// The extension GitHub's runner gives a script, by its shell's program:
633+/// none for a program it does not know.
634+fn script_extension(program: &str) -> &'static str {
635+ match program_name(program) {
636+ "bash" | "sh" => "sh",
637+ "pwsh" | "powershell" => "ps1",
638+ "cmd" => "cmd",
639+ "python" => "py",
640+ _ => "",
641+ }
642+}
643+
644+/// A script as GitHub's runner writes it: for PowerShell, stopping at the
645+/// first error and exiting with the last program's exit code.
646+fn script_contents(program: &str, script: &str) -> String {
647+ let newline = if cfg!(windows) { "\r\n" } else { "\n" };
648+ match program_name(program) {
649+ "pwsh" | "powershell" => format!(
650+ "$ErrorActionPreference = 'stop'{newline}{script}{newline}if ((Test-Path -LiteralPath variable:\\LASTEXITCODE)) {{ exit $LASTEXITCODE }}"
651+ ),
652+ _ => script.to_owned(),
653+ }
654+}
655+
656+/// Whether `program` is a file in one of `path`'s folders (or, on
657+/// Windows, `program.exe`), without running it.
658+fn on_path(program: &str, path: &str) -> bool {
659+ let names = if cfg!(windows) { vec![program.to_owned(), format!("{program}.exe")] } else { vec![program.to_owned()] };
660+ path.split(paths::PATH_SEPARATOR).filter(|dir| !dir.is_empty()).any(|dir| names.iter().any(|name| Path::new(dir).join(name).is_file()))
661+}
662+
663+impl Job {
664+ /// `shell: python`'s program: `python`, as on GitHub, where there is
665+ /// one, else `python3` (Debian has no `python`).
666+ fn python_program(&self) -> String {
667+ if let Some(container) = &self.container {
668+ return container.python.into();
669+ }
670+ let path = std::iter::once(self.base_env.get("PATH").cloned().unwrap_or_default());
671+ let path: Vec<String> = self.path_prepend.iter().cloned().chain(path).collect();
672+ if on_path("python", &path.join(paths::PATH_SEPARATOR)) { "python".into() } else { "python3".into() }
673+ }
674+}
675+
571676 /// Whether `program` is on `PATH`.
572677 fn program_exists(program: &str) -> bool {
573678 Command::new(program)
615720
616721 fn setup(mut spec: Value, api: Api) -> Result<Job> {
617722 let masks: Vec<String> = spec["masks"].as_array().map(|m| m.iter().filter_map(|v| v.as_str().map(str::to_owned)).collect()).unwrap_or_default();
618− let log = Log::new(api, masks);
723+ let mut log = Log::new(api, masks);
724+ // Variables GitHub sets that a g1t from before them does not send.
725+ default_variables(&mut spec);
619726 // On a self-hosted runner's own machine, GitHub's layout lives in a
620727 // folder of the runner's (paths.rs).
621728 for part in ["variables", "github"] {
624731 paths::relocate(&mut spec["contexts"]["runner"]);
625732 let workspace = paths::under_home(WORKSPACE);
626733 let temp = paths::under_home(TEMP);
734+ log.matchers.workspace = workspace.clone();
627735 std::fs::create_dir_all(&workspace).context("could not make the workspace")?;
628736 std::fs::create_dir_all(&temp).context("could not make the temporary folder")?;
629737 std::fs::write(temp.join("event.json"), serde_json::to_string_pretty(&spec["event"])?)?;
633741 let docker_hosted = cfg!(target_os = "linux")
634742 && std::env::var("G1T_DOCKER").as_deref() == Ok("on")
635743 && spec["variables"]["RUNNER_ENVIRONMENT"].as_str() != Some("self-hosted");
636− // This process's environment, less what only it should see.
744+ // This process's environment, less what only it should see. `ImageOS`
745+ // and `ImageVersion` come with it when the machine's image sets them,
746+ // as GitHub's images do; nothing makes them up otherwise.
637747 let mut base_env: BTreeMap<String, String> =
638748 std::env::vars().filter(|(name, _)| !matches!(name.as_str(), "ACTIONS_TOKEN" | "ACTIONS_JOB" | "MODE") && !name.starts_with("G1T_")).collect();
639749 base_env.insert("HOME".into(), std::env::var("HOME").unwrap_or_else(|_| "/home/node".into()));
669779 debug,
670780 last_node_outputs: BTreeMap::new(),
671781 last_node_state: BTreeMap::new(),
782+ pre_state: BTreeMap::new(),
672783 host_env,
673784 docker_hosted,
674785 network: None,
688799 Ok(job)
689800 }
690801
802+/// `RUNNER_WORKSPACE` and `GITHUB_REPOSITORY_OWNER_ID` (with
803+/// `github.repository_owner_id`), when the spec does not have them: the
804+/// folder the workspace is in, and the owner's id from the event.
805+fn default_variables(spec: &mut Value) {
806+ if spec["variables"].is_object() && spec["variables"].get("RUNNER_WORKSPACE").is_none() {
807+ spec["variables"]["RUNNER_WORKSPACE"] = json!(g1t_actions::events::RUNNER_WORKSPACE);
808+ }
809+ let owner_id = g1t_actions::events::owner_id(&spec["event"]);
810+ if let Some(owner_id) = owner_id.filter(|id| !id.is_empty()) {
811+ if spec["variables"].is_object() && spec["variables"].get("GITHUB_REPOSITORY_OWNER_ID").and_then(Value::as_str).is_none_or(str::is_empty) {
812+ spec["variables"]["GITHUB_REPOSITORY_OWNER_ID"] = json!(owner_id);
813+ }
814+ if spec["github"].is_object() && spec["github"].get("repository_owner_id").and_then(Value::as_str).is_none_or(str::is_empty) {
815+ spec["github"]["repository_owner_id"] = json!(owner_id);
816+ }
817+ }
818+}
819+
691820 /// The job's `defaults.run`, its own over the workflow's.
692821 fn run_defaults(spec: &Value) -> Map<String, Value> {
693822 let mut defaults = spec["workflow"]["defaults"]["run"].as_object().cloned().unwrap_or_default();
698827 }
699828
700829 fn run_job(job: &mut Job) {
701− let steps: Vec<Map<String, Value>> = job.spec["spec"]["steps"]
830+ let mut steps: Vec<Map<String, Value>> = job.spec["spec"]["steps"]
702831 .as_array()
703832 .map(|steps| steps.iter().filter_map(|s| s.as_object().cloned()).collect())
704833 .unwrap_or_default();
834+ name_steps(&mut steps);
705835 let defaults = run_defaults(&job.spec);
706836
707837 // Step names as they read before anything has run.
751881 }
752882 }
753883 // Actions first, then containers, as GitHub's "Set up job" and
754− // "Initialize containers".
755− let containers_started = job.prepare_actions(&steps) && job.start_containers();
884+ // "Initialize containers". Each action's `pre` (read from the actions
885+ // just downloaded) is a step of its own, "Pre <step>", before the
886+ // job's first step.
887+ let prepared = job.prepare_actions(&steps);
888+ let pres = if prepared { job.pre_steps(&steps) } else { Vec::new() };
889+ if !pres.is_empty() {
890+ let mut names: Vec<String> = pres.iter().map(|(index, _)| format!("Pre {}", job.step_names[*index])).collect();
891+ names.extend(job.step_names.iter().cloned());
892+ job.step_names = names;
893+ job.report_steps();
894+ }
895+ let containers_started = prepared && job.start_containers();
756896 job.log.flush();
757897
758898 let mut frame = Frame::default();
762902 job.log.step_state(index as u32 + 1, name, "completed", Some("skipped"));
763903 }
764904 }
905+ for (index, (_, pre)) in pres.iter().enumerate().filter(|_| containers_started) {
906+ let number = index as u32 + 1;
907+ let name = job.step_names[index].clone();
908+ if report::cancelled() {
909+ report::take_cancel();
910+ }
911+ if !job.pre_runs(pre) {
912+ job.log.step_state(number, &name, "completed", Some("skipped"));
913+ continue;
914+ }
915+ job.log.step(number);
916+ job.log.step_state(number, &name, "in_progress", None);
917+ let ok = job.run_pre(pre);
918+ if !ok {
919+ job.failed = true;
920+ }
921+ let conclusion = if ok {
922+ "success"
923+ } else if report::cancelled() {
924+ "cancelled"
925+ } else {
926+ "failure"
927+ };
928+ job.log.step_state(number, &name, "completed", Some(conclusion));
929+ }
930+ let offset = pres.len() as u32;
765931 for (index, step) in steps.iter().enumerate().filter(|_| containers_started) {
766− job.step(&mut frame, step, index as u32 + 1, true, &defaults);
932+ job.step(&mut frame, step, offset + index as u32 + 1, true, &defaults);
767933 job.log_docker_notes();
768934 if job.job_remaining().is_zero() {
769935 job.log.line("##[error]The job ran past its time limit.");
787953 job.log.step(number);
788954 job.log.step_state(number, &post.name, "in_progress", None);
789955 let ok = match &post.run {
790− PostRun::Node { action_dir, script } => job.run_node(action_dir, script, &post.env),
956+ PostRun::Node { action_dir, script, using } => job.run_node(action_dir, script, &post.env, using),
791957 PostRun::CacheSave { key, paths, version } => job.cache_save(key, paths, version),
792958 PostRun::Docker(run) => job.run_docker(run).0,
793959 };
8831049 }
8841050
8851051 #[test]
1052+ fn scripts_get_githubs_extension_for_their_shell() {
1053+ assert_eq!(script_extension("bash"), "sh");
1054+ assert_eq!(script_extension("/usr/bin/sh"), "sh");
1055+ assert_eq!(script_extension("pwsh"), "ps1");
1056+ assert_eq!(script_extension("C:\\Windows\\powershell.exe"), "ps1");
1057+ assert_eq!(script_extension("cmd"), "cmd");
1058+ assert_eq!(script_extension("python"), "py");
1059+ // A program GitHub does not know: no extension.
1060+ assert_eq!(script_extension("perl"), "");
1061+ assert_eq!(script_extension("python3"), "");
1062+ }
1063+
1064+ #[test]
1065+ fn powershell_scripts_stop_at_errors_and_keep_the_exit_code() {
1066+ let newline = if cfg!(windows) { "\r\n" } else { "\n" };
1067+ assert_eq!(
1068+ script_contents("pwsh", "Write-Output hi"),
1069+ format!("$ErrorActionPreference = 'stop'{newline}Write-Output hi{newline}if ((Test-Path -LiteralPath variable:\\LASTEXITCODE)) {{ exit $LASTEXITCODE }}")
1070+ );
1071+ assert!(script_contents("powershell.exe", "x").starts_with("$ErrorActionPreference = 'stop'"));
1072+ assert_eq!(script_contents("bash", "echo hi"), "echo hi");
1073+ }
1074+
1075+ #[test]
1076+ fn programs_are_found_on_path_without_running_them() {
1077+ let dir = std::env::temp_dir().join(format!("g1t-on-path-{}", std::process::id()));
1078+ std::fs::create_dir_all(&dir).unwrap();
1079+ std::fs::write(dir.join("python"), "").unwrap();
1080+ let path = format!("/nowhere{}{}", paths::PATH_SEPARATOR, dir.display());
1081+ assert!(on_path("python", &path));
1082+ assert!(!on_path("python9", &path));
1083+ assert!(!on_path("python", ""));
1084+ let _ = std::fs::remove_dir_all(&dir);
1085+ }
1086+
1087+ #[test]
1088+ fn steps_are_named_for_github_action_as_github_names_them() {
1089+ let mut steps: Vec<Map<String, Value>> = serde_json::from_value(json!([
1090+ { "run": "a" },
1091+ { "run": "b" },
1092+ { "id": "build", "run": "c" },
1093+ { "uses": "actions/setup-node@v4" },
1094+ { "uses": "actions/cache/restore@v4" },
1095+ { "uses": "actions/setup-node@v4" },
1096+ { "uses": "./.github/actions/x" },
1097+ { "run": "d" },
1098+ { "id": "__run_4", "run": "e" },
1099+ { "run": "f" },
1100+ ]))
1101+ .unwrap();
1102+ name_steps(&mut steps);
1103+ let names: Vec<String> = steps.iter().map(|s| expr::to_text(&s[ACTION_NAME])).collect();
1104+ assert_eq!(
1105+ names,
1106+ ["__run", "__run_2", "build", "__actions_setup-node", "__actions_cache", "__actions_setup-node_2", "__self", "__run_3", "__run_4", "__run_5"]
1107+ );
1108+ }
1109+
1110+ #[test]
1111+ fn variables_a_spec_lacks_are_filled_in() {
1112+ let mut spec = json!({
1113+ "variables": { "CI": "true" },
1114+ "github": { "repository_owner_id": "" },
1115+ "event": { "repository": { "owner": { "id": 7 } } },
1116+ });
1117+ default_variables(&mut spec);
1118+ assert_eq!(spec["variables"]["RUNNER_WORKSPACE"], "/home/runner/work");
1119+ assert_eq!(spec["variables"]["GITHUB_REPOSITORY_OWNER_ID"], "7");
1120+ assert_eq!(spec["github"]["repository_owner_id"], "7");
1121+ // What g1t sent stands.
1122+ let mut spec = json!({ "variables": { "RUNNER_WORKSPACE": "/w", "GITHUB_REPOSITORY_OWNER_ID": "9" }, "github": {}, "event": {} });
1123+ default_variables(&mut spec);
1124+ assert_eq!(spec["variables"]["RUNNER_WORKSPACE"], "/w");
1125+ assert_eq!(spec["variables"]["GITHUB_REPOSITORY_OWNER_ID"], "9");
1126+ }
1127+
1128+ #[test]
8861129 fn a_steps_timeout_is_its_own_or_the_nearer_one_around_it() {
8871130 let now = Instant::now();
8881131 // No timeout-minutes anywhere: only the job's deadline applies.
+132−10
2323 pub(crate) stopped: Option<String>,
2424 /// Whether `::debug::` lines are shown (`ACTIONS_STEP_DEBUG`).
2525 pub(crate) debug: bool,
26+ /// `::echo::on` or `::echo::off`: whether command lines are shown as
27+ /// well as acted on. Until a step says, they are with debug logging.
28+ pub(crate) echo: Option<bool>,
2629 }
2730
2831 /// `%25`, `%0D`, `%0A`, and in properties `%3A` and `%2C`, as the
6568 self.stopped = None;
6669 return None;
6770 }
68− return Some(line.to_owned());
71+ return Some(output(line, log));
6972 }
7073 let Some((name, properties, data)) = parse_command(line) else {
71− return Some(line.to_owned());
74+ return Some(output(line, log));
7275 };
76+ // Echoed as it reads, before it is acted on; `::add-mask::` never is.
77+ let known = matches!(
78+ name.as_str(),
79+ "add-mask" | "error" | "warning" | "notice" | "group" | "endgroup" | "debug" | "set-output" | "save-state" | "stop-commands" | "echo" | "add-matcher" | "remove-matcher" | "add-path" | "set-env"
80+ );
81+ if known && name != "add-mask" && self.echo.unwrap_or(self.debug) {
82+ log.line(line);
83+ }
7384 match name.as_str() {
7485 "add-mask" => {
7586 if !data.trim().is_empty() {
7990 }
8091 "error" | "warning" | "notice" => {
8192 log.annotation(&name, &data, &properties);
82− let label = match name.as_str() {
83− "error" => "Error",
84− "warning" => "Warning",
85− _ => "Notice",
86− };
87− Some(format!("##[{name}]{label}: {data}"))
93+ Some(issue_line(&name, &data))
8894 }
8995 "group" => Some(format!("##[group]{data}")),
9096 "endgroup" => Some("##[endgroup]".to_owned()),
105111 self.stopped = Some(data);
106112 None
107113 }
108− "echo" => None,
114+ "echo" => {
115+ match data.trim().to_ascii_lowercase().as_str() {
116+ "on" => self.echo = Some(true),
117+ "off" => self.echo = Some(false),
118+ _ => return Some(format!("##[error]Invalid echo command value. Possible values can be: 'on', 'off'. Current value is: '{}'.", data.trim())),
119+ }
120+ None
121+ }
122+ "add-matcher" => {
123+ let path = matcher_path(data.trim(), &log.matchers.workspace);
124+ let read = std::fs::read_to_string(&path).map_err(|e| format!("{}: {e}", path.display())).and_then(|text| super::matchers::parse(&text));
125+ match read {
126+ Ok(matchers) => {
127+ log.matchers.add(matchers);
128+ None
129+ }
130+ Err(problem) => Some(format!("##[error]Unable to process command '{line}' successfully.\n##[error]{problem}")),
131+ }
132+ }
133+ "remove-matcher" => {
134+ match properties.get("owner").and_then(Value::as_str).filter(|o| !o.trim().is_empty()) {
135+ Some(owner) => log.matchers.remove(owner.trim()),
136+ // No owner: every matcher in the file named.
137+ None => {
138+ let path = matcher_path(data.trim(), &log.matchers.workspace);
139+ match std::fs::read_to_string(&path).map_err(|e| format!("{}: {e}", path.display())).and_then(|text| super::matchers::parse(&text)) {
140+ Ok(matchers) => matchers.iter().for_each(|m| log.matchers.remove(&m.owner)),
141+ Err(problem) => return Some(format!("##[error]Unable to process command '{line}' successfully.\n##[error]{problem}")),
142+ }
143+ }
144+ }
145+ None
146+ }
109147 "add-path" | "set-env" => Some(format!(
110148 "##[error]The `{name}` command is disabled, as on GitHub. Write to the file in $GITHUB_{} instead.",
111149 if name == "add-path" { "PATH" } else { "ENV" }
112150 )),
113− _ => Some(line.to_owned()),
151+ _ => Some(output(line, log)),
152+ }
153+ }
154+}
155+
156+/// How an annotation shows in the log.
157+fn issue_line(level: &str, message: &str) -> String {
158+ let label = match level {
159+ "error" => "Error",
160+ "warning" => "Warning",
161+ _ => "Notice",
162+ };
163+ format!("##[{level}]{label}: {message}")
164+}
165+
166+/// A line of output, as the log shows it: as it is, or, when a problem
167+/// matcher reads an issue in it, the issue, annotated.
168+fn output(line: &str, log: &mut Log) -> String {
169+ match log.matchers.issue(line) {
170+ Some(issue) => {
171+ log.annotation(issue.level, &issue.message, &issue.properties);
172+ issue_line(issue.level, &issue.message)
114173 }
174+ None => line.to_owned(),
115175 }
116176 }
117177
178+/// A matcher file's path: relative to the workspace, and a Docker action's
179+/// `/github/workspace` read as the workspace it is.
180+fn matcher_path(given: &str, workspace: &std::path::Path) -> std::path::PathBuf {
181+ if let Some(rest) = given.strip_prefix("/github/workspace") {
182+ return workspace.join(rest.trim_start_matches('/'));
183+ }
184+ let path = std::path::Path::new(given);
185+ if path.is_absolute() { path.to_path_buf() } else { workspace.join(path) }
186+}
187+
118188 /// How a process ended.
119189 pub(crate) enum Ended {
120190 Exited(i32),
316386 assert!(text.contains("cleaned up"), "{text}");
317387 }
318388
389+ /// `::echo::on` shows command lines until `::echo::off`, which is
390+ /// itself shown; `::add-mask::` never is.
391+ #[cfg(unix)]
392+ #[test]
393+ fn echo_shows_command_lines_while_on() {
394+ use super::Commands;
395+ use crate::actions::report::{Api, Log};
396+
397+ let mut log = Log::new(Api::offline(), Vec::new());
398+ let mut commands = Commands::default();
399+ for line in ["::set-output name=a::1", "::echo::on", "::save-state name=b::2", "::add-mask::s3cret", "::echo::off", "::save-state name=c::3"] {
400+ if let Some(shown) = commands.handle(line, &mut log) {
401+ log.line(&shown);
402+ }
403+ }
404+ let text = log.buffered();
405+ assert_eq!(text, "::save-state name=b::2\n::echo::off\n");
406+ assert_eq!((commands.outputs.len(), commands.state.len()), (1, 2));
407+ // Debug logging echoes until a step says otherwise.
408+ let mut log = Log::new(Api::offline(), Vec::new());
409+ let mut commands = Commands { debug: true, ..Commands::default() };
410+ commands.handle("::set-output name=a::1", &mut log);
411+ assert!(log.buffered().contains("::set-output name=a::1"));
412+ assert!(commands.handle("::echo::maybe", &mut log).unwrap().contains("Invalid echo command value"));
413+ }
414+
415+ /// A matcher added by one line reads the lines after it, until removed.
416+ #[cfg(unix)]
417+ #[test]
418+ fn matchers_are_added_and_removed_by_command() {
419+ use super::Commands;
420+ use crate::actions::report::{Api, Log};
421+
422+ let dir = std::env::temp_dir().join(format!("g1t-add-matcher-{}", std::process::id()));
423+ std::fs::create_dir_all(&dir).unwrap();
424+ std::fs::write(
425+ dir.join("go.json"),
426+ r#"{"problemMatcher": [{"owner": "go", "pattern": [{"regexp": "^\\s*(.+\\.go):(?:(\\d+):(\\d+):)? (.*)", "file": 1, "line": 2, "column": 3, "message": 4}]}]}"#,
427+ )
428+ .unwrap();
429+ let mut log = Log::new(Api::offline(), Vec::new());
430+ log.matchers.workspace = dir.clone();
431+ let mut commands = Commands::default();
432+ assert_eq!(commands.handle("main.go:3:1: undefined: x", &mut log).unwrap(), "main.go:3:1: undefined: x");
433+ assert!(commands.handle("::add-matcher::go.json", &mut log).is_none());
434+ assert_eq!(commands.handle("main.go:3:1: undefined: x", &mut log).unwrap(), "##[error]Error: undefined: x");
435+ assert!(commands.handle("::remove-matcher owner=go::", &mut log).is_none());
436+ assert_eq!(commands.handle("main.go:3:1: undefined: x", &mut log).unwrap(), "main.go:3:1: undefined: x");
437+ assert!(commands.handle("::add-matcher::missing.json", &mut log).unwrap().contains("Unable to process command"));
438+ let _ = std::fs::remove_dir_all(&dir);
439+ }
440+
319441 #[test]
320442 fn commands_are_read_with_their_properties() {
321443 let (name, properties, data) = parse_command("::error file=app.js,line=10,title=Bad%3A thing::Something%0Abroke").unwrap();
+35−3
167167 last: Instant,
168168 /// When anything was last sent, for the ping.
169169 sent: Instant,
170+ /// The problem matchers steps added, which last the whole job.
171+ pub(crate) matchers: super::matchers::Matchers,
170172 /// Everything sent so far, so a test reads the whole log however its
171173 /// batches fell (`buffered`).
172174 #[cfg(test)]
184186 buffer: String::new(),
185187 last: Instant::now(),
186188 sent: Instant::now(),
189+ matchers: super::matchers::Matchers::default(),
187190 #[cfg(test)]
188191 flushed: String::new(),
189192 }
282285 self.api.report(json!({ "kind": "summary", "step": self.step, "markdown": markdown }));
283286 }
284287
288+ /// An annotation, from `::error::` and its kin or a problem matcher:
289+ /// its `file`, `line`, `col`, `endLine` and `endColumn` as GitHub's
290+ /// commands spell them, sent as `file`, `line`, `column`, `end_line`
291+ /// and `end_column`.
285292 pub(crate) fn annotation(&mut self, level: &str, message: &str, properties: &serde_json::Map<String, Value>) {
293+ let report = self.annotation_report(level, message, properties);
294+ self.api.report(report);
295+ }
296+
297+ fn annotation_report(&self, level: &str, message: &str, properties: &serde_json::Map<String, Value>) -> Value {
286298 let message = self.mask(message);
287299 let title = properties.get("title").and_then(Value::as_str).map(|title| self.mask(title));
288− self.api.report(json!({
300+ let number = |keys: &[&str]| keys.iter().find_map(|key| properties.get(*key).and_then(Value::as_str)).and_then(|n| n.trim().parse::<u32>().ok());
301+ json!({
289302 "kind": "annotation",
290303 "level": level,
291304 "message": message,
292305 "title": title,
293306 "file": properties.get("file"),
294− "line": properties.get("line").and_then(|l| l.as_str()).and_then(|l| l.parse::<u32>().ok()),
295− }));
307+ "line": number(&["line"]),
308+ "column": number(&["col"]),
309+ "end_line": number(&["endLine"]),
310+ "end_column": number(&["endColumn"]),
311+ })
296312 }
297313
298314 pub(crate) fn done(&mut self, conclusion: &str, outputs: &serde_json::Map<String, Value>, reason: Option<&str>) {
367383 }
368384
369385 #[test]
386+ fn annotations_carry_their_columns_and_end() {
387+ let log = log(&["hidden"]);
388+ let properties: serde_json::Map<String, Value> =
389+ serde_json::from_value(json!({ "file": "a.rs", "line": "3", "col": "7", "endLine": "4", "endColumn": "2", "title": "t hidden" })).unwrap();
390+ let report = log.annotation_report("error", "bad hidden", &properties);
391+ assert_eq!(report["line"], 3);
392+ assert_eq!(report["column"], 7);
393+ assert_eq!(report["end_line"], 4);
394+ assert_eq!(report["end_column"], 2);
395+ assert_eq!(report["title"], "t ***");
396+ assert_eq!(report["message"], "bad ***");
397+ let bare = log.annotation_report("notice", "x", &serde_json::Map::new());
398+ assert!(bare["column"].is_null() && bare["file"].is_null());
399+ }
400+
401+ #[test]
370402 fn added_masks_cover_every_form() {
371403 let mut log = log(&[]);
372404 log.add_mask("line one\nline two");
+273−83
381381 ready
382382 }
383383
384− /// Runs a JavaScript file of an action with Node.
385− pub(crate) fn run_node(&mut self, action_dir: &Path, script: &str, env: &BTreeMap<String, String>) -> bool {
384+ /// Runs a JavaScript file of an action with Node: the one its
385+ /// `runs.using` names where the machine has it (`node_program`).
386+ pub(crate) fn run_node(&mut self, action_dir: &Path, script: &str, env: &BTreeMap<String, String>, using: &str) -> bool {
386387 let id = format!("node{}", super::rand_id());
387388 let Ok(files) = StepFiles::new(&self.temp, &id) else { return false };
388389 let full = self.process_env(env, &files);
396397 let command = match in_container {
397398 Some(command) => command,
398399 None => {
399− let mut command = Command::new("node");
400+ let node = node_program(using, |name| std::env::var(name).ok(), |path| path.is_file());
401+ let mut command = Command::new(node);
400402 command.arg(&script_path).current_dir(&self.workspace).env_clear().envs(full);
401403 command
402404 }
431433 env: &BTreeMap<String, String>,
432434 frame: &Frame,
433435 title: &str,
434− id: Option<&str>,
436+ action_name: &str,
435437 _timeout: Duration,
436438 ) -> (bool, BTreeMap<String, String>) {
437439 let uses = uses.trim();
439441 // `with.args` and `with.entrypoint` are the container's; every
440442 // input is also an `INPUT_` variable, as on GitHub.
441443 let mut step_env = env.clone();
444+ step_env.insert("GITHUB_ACTION".into(), action_name.to_owned());
442445 for (input, value) in with {
443446 step_env.insert(format!("INPUT_{}", input.replace(' ', "_").to_ascii_uppercase()), value.clone());
444447 }
466469 "actions/cache/save" => return self.cache_save_now(with),
467470 _ => {}
468471 }
469− let source = if let Some(local) = name.strip_prefix("./") {
470− // A repository moved from GitHub renamed `.github` to `.g1t`, but
471− // its workflows still say `./.github/actions/…`.
472− let mut dir = self.workspace.join(local);
473− if let Some(rest) = local.strip_prefix(".github/")
474− && !dir.exists()
475− {
476− dir = self.workspace.join(".g1t").join(rest);
477− }
478− Source::Local(dir)
479− } else {
480− let mut parts = name.splitn(3, '/');
481− let (Some(owner), Some(repo)) = (parts.next(), parts.next()) else {
482− self.log.line(&format!("##[error]`{uses}` is not an action: use owner/repo@ref, owner/repo/path@ref, or ./path."));
483− return (false, BTreeMap::new());
484− };
485− if git_ref.is_empty() {
486− self.log.line(&format!("##[error]`{uses}` needs a version, such as @v4."));
487− return (false, BTreeMap::new());
488− }
489− Source::GitHub {
490− owner: owner.to_owned(),
491− repo: repo.to_owned(),
492− path: parts.next().unwrap_or_default().to_owned(),
493− git_ref: git_ref.to_owned(),
494− }
495− };
496− let (dir, repository) = match &source {
497− Source::Local(dir) => (dir.clone(), String::new()),
498− Source::GitHub { owner, repo, path, git_ref } => match self.fetch_remote_action(owner, repo, git_ref) {
499− Some(root) => (if path.is_empty() { root } else { root.join(path) }, format!("{owner}/{repo}")),
500− None => return (false, BTreeMap::new()),
501− },
502− };
503− let manifest = ["action.yml", "action.yaml"].iter().map(|f| dir.join(f)).find(|p| p.exists());
504− let Some(manifest) = manifest else {
505− self.log.line(&format!("##[error]`{uses}` has no action.yml."));
472+ let Some(found) = self.resolve_action(uses) else {
506473 return (false, BTreeMap::new());
507474 };
508− let action = match std::fs::read_to_string(&manifest).ok().and_then(|text| serde_yaml::from_str::<serde_yaml::Value>(&text).ok()) {
509− Some(yaml) => yaml_to_json(&yaml),
510− None => {
511− self.log.line(&format!("##[error]`{uses}`: its action.yml does not read."));
512− return (false, BTreeMap::new());
513− }
514− };
475+ let Action { dir, local, manifest: action, .. } = &found;
515476
516477 // Inputs: what the step gives, else the action's defaults.
517478 let env_context = env.clone();
518479 let contexts = self.contexts_for(frame, &env_context);
519− let mut inputs: BTreeMap<String, String> = BTreeMap::new();
520− if let Some(Value::Object(declared)) = action.get("inputs") {
521− for (input, spec) in declared {
522− let given = with.iter().find(|(k, _)| k.eq_ignore_ascii_case(input)).map(|(_, v)| v.clone());
523− let value = match given {
524− Some(value) => value,
525− None => match spec.get("default") {
526− Some(default) => {
527− let default = self.with_scope(&contexts, |scope| expr::interpolate_value(default, scope)).unwrap_or(Value::Null);
528− expr::to_text(&default)
529− }
530− None => String::new(),
531− },
532− };
533− inputs.insert(input.clone(), value);
534− }
535− }
536− for (key, value) in with {
537− if !inputs.keys().any(|k| k.eq_ignore_ascii_case(key)) {
538− inputs.insert(key.clone(), value.clone());
539− }
540− }
480+ let inputs = self.action_inputs(action, with, &contexts, true);
541481
542482 let runs = action.get("runs").cloned().unwrap_or(Value::Null);
543483 let using = runs.get("using").map(expr::to_text).unwrap_or_default().to_ascii_lowercase();
544− let mut step_env = env.clone();
545− step_env.insert("GITHUB_ACTION".into(), id.map_or_else(|| format!("__{}", repository.replace('/', "_")), str::to_owned));
546− step_env.insert("GITHUB_ACTION_REPOSITORY".into(), repository.clone());
547− step_env.insert("GITHUB_ACTION_REF".into(), git_ref.to_owned());
548− step_env.insert("GITHUB_ACTION_PATH".into(), dir.display().to_string());
484+ let mut step_env = action_env(env, action_name, &found, git_ref);
549485
550486 if using.starts_with("node") {
551487 for (input, value) in &inputs {
552488 step_env.insert(format!("INPUT_{}", input.replace(' ', "_").to_ascii_uppercase()), value.clone());
553489 }
554490 let condition_of = |key: &str| runs.get(key).map(expr::to_text).unwrap_or_else(|| "always()".into());
491+ // State an action saves, `STATE_<name>` for its later stages.
492+ let mut state = BTreeMap::new();
555493 if let Some(pre) = runs.get("pre").map(expr::to_text) {
556− let run_pre = self.with_scope(&contexts, |scope| expr::condition(&condition_of("pre-if"), scope)).unwrap_or(true);
557− if run_pre && !self.run_node(&dir, &pre, &step_env) {
558− return (false, BTreeMap::new());
494+ if frame.action_path.is_some() {
495+ // An action inside a composite action: its `pre` right
496+ // before its `main`.
497+ let run_pre = self.with_scope(&contexts, |scope| expr::condition(&condition_of("pre-if"), scope)).unwrap_or(true);
498+ if run_pre {
499+ if !self.run_node(dir, &pre, &step_env, &using) {
500+ return (false, BTreeMap::new());
501+ }
502+ state = std::mem::take(&mut self.last_node_state);
503+ }
504+ } else if *local {
505+ self.log.line(&format!("##[warning]`pre` execution is not supported for local action from '{uses}'"));
506+ } else {
507+ // Run before the job's first step (`pre_steps`).
508+ state = self.pre_state.remove(action_name).unwrap_or_default();
559509 }
560510 }
561511 let Some(main) = runs.get("main").map(expr::to_text) else {
562512 self.log.line(&format!("##[error]`{uses}` has no `runs.main`."));
563513 return (false, BTreeMap::new());
564514 };
565− let ok = self.run_node(&dir, &main, &step_env);
515+ let mut main_env = step_env.clone();
516+ for (name, value) in &state {
517+ main_env.insert(format!("STATE_{name}"), value.clone());
518+ }
519+ let ok = self.run_node(dir, &main, &main_env, &using);
566520 let outputs = std::mem::take(&mut self.last_node_outputs);
567− let state = std::mem::take(&mut self.last_node_state);
521+ state.extend(std::mem::take(&mut self.last_node_state));
568522 if let Some(post) = runs.get("post").map(expr::to_text) {
569523 let mut post_env = step_env.clone();
570524 for (name, value) in state {
574528 name: format!("Post {title}"),
575529 condition: condition_of("post-if"),
576530 env: post_env,
577− run: PostRun::Node { action_dir: dir.clone(), script: post },
531+ run: PostRun::Node { action_dir: dir.clone(), script: post, using: using.clone() },
578532 });
579533 }
580534 return (ok, outputs);
586540 action_path: Some(dir.display().to_string()),
587541 env: env.clone(),
588542 };
589− let steps: Vec<Map<String, Value>> = runs.get("steps").and_then(Value::as_array).map(|s| s.iter().filter_map(|s| s.as_object().cloned()).collect()).unwrap_or_default();
543+ let mut steps: Vec<Map<String, Value>> = runs.get("steps").and_then(Value::as_array).map(|s| s.iter().filter_map(|s| s.as_object().cloned()).collect()).unwrap_or_default();
544+ super::name_steps(&mut steps);
590545 let was_failed = self.failed;
591546 // A composite's steps see their own success, not the job's.
592547 self.failed = false;
610565 return (ok, outputs);
611566 }
612567 if using == "docker" {
613− return self.docker_action(uses, &dir, &runs, &inputs, &step_env, frame, title);
568+ return self.docker_action(uses, dir, &runs, &inputs, &step_env, frame, title);
614569 }
615570 self.log.line(&format!("##[error]`{uses}` runs with `{using}`, which g1t does not know."));
616571 (false, BTreeMap::new())
692647 }
693648 }
694649
650+/// An action, read from its action.yml.
651+pub(crate) struct Action {
652+ pub(crate) dir: PathBuf,
653+ /// `owner/repo`, or empty for one in the repository.
654+ pub(crate) repository: String,
655+ pub(crate) local: bool,
656+ pub(crate) manifest: Value,
657+}
658+
659+/// An action's `pre`, run before the job's first step.
660+pub(crate) struct PreStep {
661+ /// Its step's `GITHUB_ACTION`, by which the step finds the state it saved.
662+ pub(crate) key: String,
663+ /// `pre-if`, `always()` when it has none.
664+ pub(crate) condition: String,
665+ pub(crate) action_dir: PathBuf,
666+ pub(crate) script: String,
667+ pub(crate) using: String,
668+ /// The `env` context its condition reads.
669+ pub(crate) context_env: BTreeMap<String, String>,
670+ /// What its process is given: the step's variables and inputs.
671+ pub(crate) env: BTreeMap<String, String>,
672+}
673+
674+impl Job {
675+ /// Finds `uses` (fetching it when it is another repository's) and reads
676+ /// its action.yml. A problem is told in the log.
677+ fn resolve_action(&mut self, uses: &str) -> Option<Action> {
678+ let (name, git_ref) = uses.split_once('@').unwrap_or((uses, ""));
679+ let source = if let Some(local) = name.strip_prefix("./") {
680+ // A repository moved from GitHub renamed `.github` to `.g1t`, but
681+ // its workflows still say `./.github/actions/…`.
682+ let mut dir = self.workspace.join(local);
683+ if let Some(rest) = local.strip_prefix(".github/")
684+ && !dir.exists()
685+ {
686+ dir = self.workspace.join(".g1t").join(rest);
687+ }
688+ Source::Local(dir)
689+ } else {
690+ let mut parts = name.splitn(3, '/');
691+ let (Some(owner), Some(repo)) = (parts.next(), parts.next()) else {
692+ self.log.line(&format!("##[error]`{uses}` is not an action: use owner/repo@ref, owner/repo/path@ref, or ./path."));
693+ return None;
694+ };
695+ if git_ref.is_empty() {
696+ self.log.line(&format!("##[error]`{uses}` needs a version, such as @v4."));
697+ return None;
698+ }
699+ Source::GitHub {
700+ owner: owner.to_owned(),
701+ repo: repo.to_owned(),
702+ path: parts.next().unwrap_or_default().to_owned(),
703+ git_ref: git_ref.to_owned(),
704+ }
705+ };
706+ let (dir, repository) = match &source {
707+ Source::Local(dir) => (dir.clone(), String::new()),
708+ Source::GitHub { owner, repo, path, git_ref } => {
709+ let root = self.fetch_remote_action(owner, repo, git_ref)?;
710+ (if path.is_empty() { root } else { root.join(path) }, format!("{owner}/{repo}"))
711+ }
712+ };
713+ let Some(manifest) = ["action.yml", "action.yaml"].iter().map(|f| dir.join(f)).find(|p| p.exists()) else {
714+ self.log.line(&format!("##[error]`{uses}` has no action.yml."));
715+ return None;
716+ };
717+ let Some(yaml) = std::fs::read_to_string(&manifest).ok().and_then(|text| serde_yaml::from_str::<serde_yaml::Value>(&text).ok()) else {
718+ self.log.line(&format!("##[error]`{uses}`: its action.yml does not read."));
719+ return None;
720+ };
721+ Some(Action { dir, repository, local: matches!(source, Source::Local(_)), manifest: yaml_to_json(&yaml) })
722+ }
723+
724+ /// An action's inputs: what the step gives, else the action's defaults,
725+ /// read with `contexts`. With `warn`, inputs the action does not declare
726+ /// are warned of, as GitHub's runner does.
727+ fn action_inputs(&mut self, action: &Value, with: &BTreeMap<String, String>, contexts: &Map<String, Value>, warn: bool) -> BTreeMap<String, String> {
728+ let mut inputs: BTreeMap<String, String> = BTreeMap::new();
729+ let mut declared_names = Vec::new();
730+ if let Some(Value::Object(declared)) = action.get("inputs") {
731+ for (input, spec) in declared {
732+ declared_names.push(input.clone());
733+ let given = with.iter().find(|(k, _)| k.eq_ignore_ascii_case(input)).map(|(_, v)| v.clone());
734+ let value = match given {
735+ Some(value) => value,
736+ None => match spec.get("default") {
737+ Some(default) => {
738+ let default = self.with_scope(contexts, |scope| expr::interpolate_value(default, scope)).unwrap_or(Value::Null);
739+ expr::to_text(&default)
740+ }
741+ None => String::new(),
742+ },
743+ };
744+ inputs.insert(input.clone(), value);
745+ }
746+ }
747+ for (key, value) in with {
748+ if !inputs.keys().any(|k| k.eq_ignore_ascii_case(key)) {
749+ inputs.insert(key.clone(), value.clone());
750+ }
751+ }
752+ let docker = action["runs"]["using"].as_str().is_some_and(|u| u.eq_ignore_ascii_case("docker"));
753+ if warn && let Some(warning) = unexpected_inputs(&declared_names, &with.keys().cloned().collect::<Vec<_>>(), docker) {
754+ self.log.line(&format!("##[warning]{warning}"));
755+ }
756+ inputs
757+ }
758+
759+ /// The `pre`s of the job's steps, in step order, each with its step's
760+ /// index: those of JavaScript actions from other repositories, fetched
761+ /// now, as GitHub's runner fetches every action while it sets up the
762+ /// job. An action in the repository is not there yet, so its `pre` is
763+ /// not run, as on GitHub.
764+ pub(crate) fn pre_steps(&mut self, steps: &[Map<String, Value>]) -> Vec<(usize, PreStep)> {
765+ let frame = Frame::default();
766+ let mut pres = Vec::new();
767+ for (index, step) in steps.iter().enumerate() {
768+ let Some(uses) = step.get("uses").map(|u| expr::to_text(u).trim().to_owned()) else { continue };
769+ // Only another repository's action, which `prepare_actions` has
770+ // already downloaded.
771+ if remote_action(&uses, self.docker_hosted).is_none() {
772+ continue;
773+ }
774+ let Some(found) = self.resolve_action(&uses) else { continue };
775+ let runs = found.manifest.get("runs").cloned().unwrap_or(Value::Null);
776+ let using = runs.get("using").map(expr::to_text).unwrap_or_default().to_ascii_lowercase();
777+ let Some(script) = runs.get("pre").map(expr::to_text).filter(|_| using.starts_with("node")) else { continue };
778+ let env_before = self.env_context(&frame);
779+ let contexts = self.contexts_for(&frame, &env_before);
780+ let with = self.step_with(step, &contexts);
781+ let context_env = self.step_env(step, &contexts, env_before);
782+ let contexts = self.contexts_for(&frame, &context_env);
783+ let inputs = self.action_inputs(&found.manifest, &with, &contexts, false);
784+ let key = step.get(super::ACTION_NAME).map(expr::to_text).unwrap_or_default();
785+ let git_ref = uses.split_once('@').map_or("", |(_, r)| r);
786+ let mut env = action_env(&context_env, &key, &found, git_ref);
787+ for (input, value) in &inputs {
788+ env.insert(format!("INPUT_{}", input.replace(' ', "_").to_ascii_uppercase()), value.clone());
789+ }
790+ let condition = runs.get("pre-if").map(expr::to_text).unwrap_or_else(|| "always()".into());
791+ pres.push((index, PreStep { key, condition, action_dir: found.dir, script, using, context_env, env }));
792+ }
793+ pres
794+ }
795+
796+ /// Whether a `pre`'s `pre-if` holds, read against the job's status.
797+ pub(crate) fn pre_runs(&self, pre: &PreStep) -> bool {
798+ let contexts = self.contexts_for(&Frame::default(), &pre.context_env);
799+ self.with_scope(&contexts, |scope| expr::condition(&pre.condition, scope)).unwrap_or(true)
800+ }
801+
802+ /// Runs a `pre`, keeping the state it saves for its step's main and post.
803+ pub(crate) fn run_pre(&mut self, pre: &PreStep) -> bool {
804+ let ok = self.run_node(&pre.action_dir, &pre.script, &pre.env, &pre.using);
805+ let state = std::mem::take(&mut self.last_node_state);
806+ self.last_node_outputs.clear();
807+ self.pre_state.insert(pre.key.clone(), state);
808+ ok
809+ }
810+}
811+
812+/// The variables a step running an action gets over its own: which
813+/// action it is and where.
814+fn action_env(env: &BTreeMap<String, String>, action_name: &str, action: &Action, git_ref: &str) -> BTreeMap<String, String> {
815+ let mut step_env = env.clone();
816+ step_env.insert("GITHUB_ACTION".into(), action_name.to_owned());
817+ step_env.insert("GITHUB_ACTION_REPOSITORY".into(), action.repository.clone());
818+ step_env.insert("GITHUB_ACTION_REF".into(), git_ref.to_owned());
819+ step_env.insert("GITHUB_ACTION_PATH".into(), action.dir.display().to_string());
820+ step_env
821+}
822+
823+/// GitHub's warning for inputs a step gives that its action does not
824+/// declare (a Docker action also takes `entryPoint` and `args`), if any.
825+fn unexpected_inputs(declared: &[String], given: &[String], docker: bool) -> Option<String> {
826+ let mut valid: Vec<String> = declared.to_vec();
827+ if docker {
828+ valid.extend(["entryPoint".to_owned(), "args".to_owned()]);
829+ }
830+ let unexpected: Vec<&str> = given.iter().filter(|g| !valid.iter().any(|v| v.eq_ignore_ascii_case(g))).map(String::as_str).collect();
831+ if unexpected.is_empty() {
832+ return None;
833+ }
834+ Some(format!("Unexpected input(s) '{}', valid inputs are ['{}']", unexpected.join("', '"), valid.join("', '")))
835+}
836+
837+/// The Node an action's `runs.using` asks for: for `node20` (or any
838+/// `nodeNN`), the binary `G1T_NODE20` names or `/opt/node20/bin/node`,
839+/// when there is one; otherwise, and for anything else, the `node` on
840+/// `PATH`, the runner's own (Node 24). Nothing is logged either way.
841+pub(crate) fn node_program(using: &str, var: impl Fn(&str) -> Option<String>, exists: impl Fn(&Path) -> bool) -> String {
842+ let using = using.trim().to_ascii_lowercase();
843+ let versioned = using.strip_prefix("node").is_some_and(|v| !v.is_empty() && v.chars().all(|c| c.is_ascii_digit()));
844+ if versioned {
845+ let named = var(&format!("G1T_{}", using.to_ascii_uppercase())).filter(|p| !p.is_empty() && exists(Path::new(p)));
846+ if let Some(path) = named {
847+ return path;
848+ }
849+ let standard = format!("/opt/{using}/bin/node");
850+ if exists(Path::new(&standard)) {
851+ return standard;
852+ }
853+ }
854+ "node".into()
855+}
856+
695857 #[cfg(test)]
696858 mod tests {
697859 use super::*;
711873 assert_eq!(found("actions/setup-node"), None);
712874 assert_eq!(found("setup-node@v4"), None);
713875 }
876+
877+ #[test]
878+ fn unexpected_inputs_are_named_with_the_valid_ones() {
879+ let declared = vec!["token".to_owned(), "path".to_owned()];
880+ assert_eq!(unexpected_inputs(&declared, &["Token".into()], false), None);
881+ assert_eq!(
882+ unexpected_inputs(&declared, &["token".into(), "foo".into(), "bar".into()], false).as_deref(),
883+ Some("Unexpected input(s) 'foo', 'bar', valid inputs are ['token', 'path']")
884+ );
885+ assert_eq!(unexpected_inputs(&[], &["x".into()], false).as_deref(), Some("Unexpected input(s) 'x', valid inputs are ['']"));
886+ // A Docker action takes `entrypoint` and `args` too.
887+ assert_eq!(unexpected_inputs(&declared, &["entrypoint".into(), "args".into()], true), None);
888+ assert!(unexpected_inputs(&declared, &["args".into()], false).is_some());
889+ }
890+
891+ #[test]
892+ fn node20_runs_with_a_node20_where_there_is_one() {
893+ let none = |_: &str| None;
894+ assert_eq!(node_program("node20", none, |_| false), "node");
895+ assert_eq!(node_program("node20", none, |p| p == Path::new("/opt/node20/bin/node")), "/opt/node20/bin/node");
896+ let named = |name: &str| (name == "G1T_NODE20").then(|| "/usr/local/n20/bin/node".to_owned());
897+ assert_eq!(node_program("Node20", named, |_| true), "/usr/local/n20/bin/node");
898+ // A named Node that is not there is passed over.
899+ assert_eq!(node_program("node20", named, |p| p == Path::new("/opt/node20/bin/node")), "/opt/node20/bin/node");
900+ assert_eq!(node_program("node24", none, |_| false), "node");
901+ assert_eq!(node_program("composite", none, |_| true), "node");
902+ assert_eq!(node_program("node", none, |_| true), "node");
903+ }
714904 }