Skip to content
Sign up

flagon-io/g1t

public
0
807 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R21//! Artifacts and the cache: `actions/upload-artifact`, its `merge`,
Cache hits are counted, not only their bytes2//! `actions/download-artifact` and `actions/cache`, done natively against
3//! g1t. Artifacts belong to the run; cache entries to the repository, found
4//! by exact key or by the newest under a `restore-keys` prefix.
5//!
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R26//! An artifact is a ZIP file, as GitHub's v4 actions make it (zip.rs), of
7//! the files its `path` finds (glob.rs): uploaded in parts with its SHA-256,
8//! and downloaded straight to a file before it is unpacked.
9//!
Cache hits are counted, not only their bytes10//! A cache entry is a tar archive, compressed with zstd where the machine
11//! has it (gzip otherwise), of up to 2 GB: uploaded in parts, and
12//! downloaded straight to a file. Its `path` takes globs (`**` included)
Merge branch 'worktree-agent-a3abfcce648e87dca'13//! and `!` patterns that leave paths out, as `actions/cache` does. Each is
14//! saved and found under a version, the hash of its `path` and compression,
15//! and g1t keeps it under the ref whose run saved it.
Cache hits are counted, not only their bytes16
17use std::collections::BTreeMap;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R218use std::io::{Read, Write};
Cache hits are counted, not only their bytes19use std::path::Path;
20use std::process::Command;
21use std::time::{Duration, Instant};
22
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R223use sha2::{Digest, Sha256};
24
Cache hits are counted, not only their bytes25use super::process::{self, Commands, Ended};
Merge remote-tracking branch 'origin/claude/cool-newton-7nm8r9'26use super::{Job, glob, zip};
Cache hits are counted, not only their bytes27
28
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.29/// How often a long transfer says how far it has got.
30const PROGRESS_EVERY: Duration = Duration::from_secs(5);
31
32/// A transfer's progress as GitHub's cache action words it:
33/// `Received 46137344 of 523190823 (8.8%), 43.9 MBs/sec`.
34fn progress(verb: &str, done: u64, total: Option<u64>, took: Duration) -> String {
35 let rate = done as f64 / 1_048_576.0 / took.as_secs_f64().max(0.001);
36 match total.filter(|total| *total > 0) {
37 Some(total) => format!("{verb} {done} of {total} ({:.1}%), {rate:.1} MBs/sec", done as f64 * 100.0 / total as f64),
38 None => format!("{verb} {done} bytes, {rate:.1} MBs/sec"),
39 }
40}
41
Cache hits are counted, not only their bytes42fn lines(text: &str) -> Vec<String> {
43 text.lines().map(str::trim).filter(|line| !line.is_empty() && !line.starts_with('#')).map(str::to_owned).collect()
44}
45
46fn quote(text: &str) -> String {
47 format!("'{}'", text.replace('\'', "'\\''"))
48}
49
50impl Job {
51 fn url(&self, rest: &str) -> String {
52 format!("{}/actions/jobs/{}/{rest}", self.log.api.base, self.log.api.job)
53 }
54
55 fn auth(&self) -> String {
56 format!("Bearer {}", self.log.api.token)
57 }
58
59 /// Runs a shell line, logging its output; whether it succeeded.
60 fn shell(&mut self, script: &str) -> bool {
61 let mut command = Command::new("bash");
62 command.args(["-c", script]).current_dir(&self.workspace);
63 let mut commands = Commands::default();
Merge Actions: cross-repo workflows and actions, release and deployment triggers, step timeouts64 matches!(process::run(command, Duration::from_secs(1800).min(self.remaining_time()), &mut self.log, &mut commands), Ok(Ended::Exited(0)))
Cache hits are counted, not only their bytes65 }
66
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.67 /// Reads a download to its end, handing `each` every piece, and says
68 /// how it is going every few seconds as GitHub's cache and artifact
69 /// actions do (`Received 46137344 of 523190823 (8.8%), 43.9 MBs/sec`),
70 /// so a long download is never a silent step. Its total in bytes.
71 fn receive(&mut self, reader: &mut dyn Read, total: Option<u64>, mut each: impl FnMut(&[u8]) -> std::io::Result<()>) -> Result<u64, String> {
72 let started = Instant::now();
73 let mut told = Instant::now();
74 let mut received = 0u64;
75 let mut buffer = vec![0u8; 256 * 1024];
76 loop {
77 let n = reader.read(&mut buffer).map_err(|e| e.to_string())?;
78 if n == 0 {
79 break;
80 }
81 each(&buffer[..n]).map_err(|e| e.to_string())?;
82 received += n as u64;
83 if told.elapsed() >= PROGRESS_EVERY {
84 told = Instant::now();
85 self.log.line(&progress("Received", received, total, started.elapsed()));
86 }
87 self.log.tick();
88 }
89 Ok(received)
Cache hits are counted, not only their bytes90 }
91
92 /// Sends `file` in parts of `part_bytes`; each part's number and etag.
93 fn send_parts(&mut self, base: &str, upload: &str, file: &Path, part_bytes: u64) -> Result<Vec<serde_json::Value>, String> {
94 let mut reader = std::fs::File::open(file).map_err(|e| e.to_string())?;
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.95 let size = reader.metadata().map(|m| m.len()).unwrap_or(0);
96 let (started, mut told, mut sent) = (Instant::now(), Instant::now(), 0u64);
Cache hits are counted, not only their bytes97 let mut parts = Vec::new();
98 let mut buffer = vec![0u8; part_bytes as usize];
99 for number in 1u32.. {
100 let mut filled = 0;
101 while filled < buffer.len() {
102 let read = reader.read(&mut buffer[filled..]).map_err(|e| e.to_string())?;
103 if read == 0 {
104 break;
105 }
106 filled += read;
107 }
108 if filled == 0 && number > 1 {
109 break;
110 }
111 let url = self.url(&format!("{base}/{number}?upload={}", urlencode(upload)));
112 // A part that fails is sent again, twice at most.
113 let mut tries = 0;
114 let answer = loop {
115 tries += 1;
116 match ureq::put(&url).set("authorization", &self.auth()).timeout(Duration::from_secs(600)).send_bytes(&buffer[..filled]) {
117 Ok(response) => break response.into_json::<serde_json::Value>().map_err(|e| e.to_string())?,
118 Err(ureq::Error::Status(status, response)) if status < 500 => return Err(refusal(response)),
119 Err(error) if tries >= 3 => return Err(error.to_string()),
120 Err(_) => std::thread::sleep(Duration::from_secs(2 * tries)),
121 }
122 };
123 parts.push(serde_json::json!({ "part": number, "etag": answer["etag"] }));
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.124 sent += filled as u64;
Cache hits are counted, not only their bytes125 if filled < buffer.len() {
126 break;
127 }
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.128 // More to send: say how far it has got, as a download does.
129 if told.elapsed() >= PROGRESS_EVERY {
130 told = Instant::now();
131 self.log.line(&progress("Sent", sent, Some(size), started.elapsed()));
132 }
133 self.log.tick();
Cache hits are counted, not only their bytes134 }
135 Ok(parts)
136 }
137
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2138 /// A value of the `github` context, as text.
139 fn github_value(&self, key: &str) -> String {
140 match self.contexts.get("github").and_then(|github| github.get(key)) {
141 Some(serde_json::Value::String(text)) => text.clone(),
142 Some(serde_json::Value::Null) | None => String::new(),
143 Some(other) => other.to_string(),
144 }
145 }
146
147 fn home(&self) -> String {
148 self.base_env_value("HOME").unwrap_or_else(|| "/home/node".into())
149 }
150
151 /// `actions/upload-artifact`: the files `path` names, packed into one
152 /// ZIP and uploaded in parts.
Cache hits are counted, not only their bytes153 pub(crate) fn upload_artifact(&mut self, with: &BTreeMap<String, String>) -> (bool, BTreeMap<String, String>) {
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2154 let name = input(with, "name").unwrap_or("artifact").to_owned();
155 let missing = input(with, "if-no-files-found").unwrap_or("warn").to_ascii_lowercase();
156 let checked = check_name(&name)
157 .and_then(|()| keep(with, true))
158 .and_then(|keep| if matches!(missing.as_str(), "warn" | "error" | "ignore") { Ok(keep) } else { Err(format!("`if-no-files-found` is `{missing}`; it takes warn, error or ignore.")) });
159 let keep = match checked {
160 Ok(keep) => keep,
161 Err(message) => {
162 self.log.line(&format!("##[error]{message}"));
163 return (false, BTreeMap::new());
Cache hits are counted, not only their bytes164 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2165 };
166 let paths = lines(input(with, "path").unwrap_or_default());
167 let found = glob::find(&paths, &self.workspace.display().to_string(), &self.home(), keep.hidden);
168 if found.files.is_empty() {
169 let message = format!("No files were found with the provided path: {}. No artifacts will be uploaded.", paths.join(", "));
170 return match missing.as_str() {
171 "error" => {
172 self.log.line(&format!("##[error]{message}"));
173 (false, BTreeMap::new())
174 }
175 "ignore" => {
176 self.log.line(&message);
177 (true, BTreeMap::new())
178 }
179 _ => {
180 self.log.line(&format!("##[warning]{message}"));
181 (true, BTreeMap::new())
182 }
183 };
184 }
185 self.log.line(&format!("Found {} with the provided path, stored relative to {}.", count(found.files.len(), "file"), found.root));
186 self.send_artifact(&name, &found.files, &keep)
187 }
188
189 /// Packs `files` into a ZIP and uploads it as the artifact `name`;
190 /// whether it worked, and the outputs `upload-artifact` sets.
191 fn send_artifact(&mut self, name: &str, files: &[(String, std::path::PathBuf)], keep: &Keep) -> (bool, BTreeMap<String, String>) {
192 let archive = self.temp.join(format!("artifact-{}.zip", super::rand_id()));
193 let sent = zip::write(&archive, files, keep.level)
194 .map_err(|e| format!("The files could not be packed: {e}"))
195 .and_then(|packed| self.post_artifact(name, &archive, keep).map(|done| (packed, done)));
196 let _ = std::fs::remove_file(&archive);
197 let (packed, done) = match sent {
198 Ok(sent) => sent,
199 Err(error) => {
200 self.log.line(&format!("##[error]{error}"));
Cache hits are counted, not only their bytes201 return (false, BTreeMap::new());
202 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2203 };
204 let kept = done.retention.map(|days| format!(" It is kept for {}.", count(days as usize, "day"))).unwrap_or_default();
205 self.log.line(&format!("Uploaded artifact {name} ({}, {}).{kept}", megabytes(packed.size), count(packed.files, "file")));
206 let url = format!(
207 "{}/{}/actions/runs/{}/artifacts/{}",
208 self.github_value("server_url").trim_end_matches('/'),
209 self.github_value("repository"),
210 self.github_value("run_id"),
211 done.id
212 );
213 self.log.line(&format!("Artifact download URL: {url}"));
214 let mut outputs = BTreeMap::new();
215 outputs.insert("artifact-id".into(), done.id.to_string());
216 outputs.insert("artifact-url".into(), url);
217 outputs.insert("artifact-digest".into(), done.digest);
218 (true, outputs)
219 }
220
221 /// Uploads the ZIP file `archive` as the artifact `name`, in parts,
222 /// giving the upload up when a part or the end fails.
223 fn post_artifact(&mut self, name: &str, archive: &Path, keep: &Keep) -> Result<Sent, String> {
224 let size = std::fs::metadata(archive).map_err(|e| e.to_string())?.len();
225 let digest = sha256_file(archive).map_err(|e| e.to_string())?;
226 let query = format!(
227 "artifacts/uploads?name={}&size={size}&retention_days={}&overwrite={}&format=zip",
228 urlencode(name),
229 keep.retention,
230 keep.overwrite
231 );
232 let failed = |e: ureq::Error| match e {
233 ureq::Error::Status(_, response) => format!("The artifact could not be uploaded: {}", refusal(response)),
234 other => format!("The artifact could not be uploaded: {other}"),
235 };
236 let started = ureq::post(&self.url(&query))
237 .set("authorization", &self.auth())
238 .timeout(Duration::from_secs(60))
239 .call()
240 .map_err(failed)?
241 .into_json::<serde_json::Value>()
242 .map_err(|e| e.to_string())?;
243 let id = number(&started["id"]).ok_or("g1t did not say which artifact the upload is")?;
244 let upload = started["upload"].as_str().unwrap_or_default().to_owned();
245 let part_bytes = started["part_bytes"].as_u64().filter(|n| *n > 0).unwrap_or(32 * 1024 * 1024);
246 let retention = started["retention_days"].as_u64();
247 if let Some(applied) = retention
248 && keep.retention != 0
249 && applied != u64::from(keep.retention)
250 {
251 self.log.line(&format!(
252 "##[notice]The artifact is kept for {}, not the {} asked for: the most this repository keeps artifacts.",
253 count(applied as usize, "day"),
254 keep.retention
255 ));
Cache hits are counted, not only their bytes256 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2257 let base = format!("artifacts/uploads/{id}");
258 let give_up = |job: &Job| {
259 let _ = ureq::delete(&job.url(&format!("{base}?upload={}", urlencode(&upload)))).set("authorization", &job.auth()).call();
260 };
261 let parts = match self.send_parts(&base, &upload, archive, part_bytes) {
262 Ok(parts) => parts,
263 Err(error) => {
264 give_up(self);
265 return Err(format!("The artifact could not be uploaded: {error}"));
Cache hits are counted, not only their bytes266 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2267 };
268 let done = ureq::post(&self.url(&format!("{base}/complete?upload={}", urlencode(&upload))))
269 .set("authorization", &self.auth())
270 .timeout(Duration::from_secs(120))
271 .send_json(serde_json::json!({ "size": size, "parts": parts, "digest": format!("sha256:{digest}") }));
272 let done = match done {
273 Ok(response) => response.into_json::<serde_json::Value>().unwrap_or_default(),
Cache hits are counted, not only their bytes274 Err(error) => {
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2275 give_up(self);
276 return Err(failed(error));
277 }
278 };
279 Ok(Sent { id: number(&done["id"]).unwrap_or(id), digest, retention: retention.map(|d| d as u32).or((keep.retention != 0).then_some(keep.retention)) })
280 }
281
282 /// The artifacts of this run, or of the run `run_id` of this
283 /// repository; one per name, the newest.
284 fn list_artifacts(&mut self, run_id: Option<&str>) -> Result<Vec<Listed>, String> {
285 let rest = match run_id {
286 Some(run) => format!("artifacts?run_id={}", urlencode(run)),
287 None => "artifacts".to_owned(),
288 };
289 let listed = ureq::get(&self.url(&rest))
290 .set("authorization", &self.auth())
291 .timeout(Duration::from_secs(60))
292 .call()
293 .map_err(|e| match e {
294 ureq::Error::Status(_, response) => refusal(response),
295 other => other.to_string(),
296 })?
297 .into_json::<Vec<serde_json::Value>>()
298 .map_err(|e| e.to_string())?;
299 Ok(newest(listed.iter().filter_map(Listed::from_json).collect()))
300 }
301
302 /// Downloads an artifact to a file, as it comes, and unpacks it into
303 /// `into`.
304 fn fetch_artifact(&mut self, artifact: &Listed, into: &Path) -> Result<(), String> {
305 let file = self.temp.join(format!("download-{}.zip", super::rand_id()));
306 let fetched = self.fetch_to(artifact, &file).and_then(|format| {
307 if format == "tgz" {
308 std::fs::create_dir_all(into).map_err(|e| e.to_string())?;
Merge remote-tracking branch 'origin/claude/cool-newton-7nm8r9' into claude/cool-newton-7nm8r9309 let script = format!("tar{} -xzf {} -C {}", tar_local(), quote(&file.display().to_string()), quote(&into.display().to_string()));
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2310 if self.shell(&script) { Ok(()) } else { Err("it could not be unpacked".into()) }
311 } else {
312 zip::extract(&file, into).map(|_| ()).map_err(|e| format!("it could not be unpacked: {e}"))
313 }
314 });
315 let _ = std::fs::remove_file(&file);
316 fetched
317 }
318
319 /// Downloads an artifact into `file`, checking its digest; how it is
320 /// packed, `zip` or `tgz`.
321 fn fetch_to(&mut self, artifact: &Listed, file: &Path) -> Result<String, String> {
322 let response = match ureq::get(&self.url(&format!("artifacts/{}/download", artifact.id)))
323 .set("authorization", &self.auth())
324 .timeout(Duration::from_secs(4 * 3600))
325 .call()
326 {
327 Ok(response) => response,
328 Err(ureq::Error::Status(404, _)) => return Err("it is gone: it expired or was deleted".into()),
329 Err(ureq::Error::Status(_, response)) => return Err(refusal(response)),
330 Err(error) => return Err(error.to_string()),
331 };
332 let format = response.header("x-g1t-format").map(str::to_owned).unwrap_or_else(|| artifact.format.clone());
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.333 let total = response.header("content-length").and_then(|n| n.parse::<u64>().ok()).or(Some(artifact.size));
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2334 let mut reader = response.into_reader();
335 let mut out = std::io::BufWriter::new(std::fs::File::create(file).map_err(|e| e.to_string())?);
336 let mut hasher = Sha256::new();
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.337 self.receive(&mut reader, total, |bytes| {
338 hasher.update(bytes);
339 out.write_all(bytes)
340 })?;
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2341 out.flush().map_err(|e| e.to_string())?;
342 let got = hex::encode(hasher.finalize());
343 if let Some(expected) = artifact.digest.as_deref().map(|d| d.trim_start_matches("sha256:"))
344 && !expected.is_empty()
345 && !expected.eq_ignore_ascii_case(&got)
346 {
347 self.log.line(&format!("##[warning]Artifact {} does not match its digest (sha256:{got}, not sha256:{expected}): it may have been changed since it was uploaded.", artifact.name));
348 }
349 Ok(format)
Cache hits are counted, not only their bytes350 }
351
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2352 /// Where artifacts are downloaded: `path`, under the workspace unless
353 /// absolute, `~` the home folder.
354 fn download_dir(&self, path: Option<&str>) -> std::path::PathBuf {
355 match path {
356 None => self.workspace.clone(),
357 Some(path) => {
358 let home = self.home();
359 let path = if path == "~" {
360 home
361 } else if let Some(rest) = path.strip_prefix("~/") {
362 format!("{}/{rest}", home.trim_end_matches('/'))
363 } else {
364 path.to_owned()
365 };
366 self.workspace.join(path)
Cache hits are counted, not only their bytes367 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2368 }
369 }
370
371 /// `actions/download-artifact`: one artifact by name straight into
372 /// `path`, or the run's artifacts (by `pattern` or `artifact-ids`) each
373 /// into a folder of its name, or all into `path` with `merge-multiple`.
374 pub(crate) fn download_artifact(&mut self, with: &BTreeMap<String, String>) -> (bool, BTreeMap<String, String>) {
375 let fail = |job: &mut Job, message: &str| {
376 job.log.line(&format!("##[error]{message}"));
377 (false, BTreeMap::new())
378 };
379 let name = input(with, "name");
380 let ids = match artifact_ids(input(with, "artifact-ids").unwrap_or_default()) {
381 Ok(ids) => ids,
382 Err(message) => return fail(self, &message),
383 };
384 if name.is_some() && !ids.is_empty() {
385 return fail(self, "Inputs 'name' and 'artifact-ids' cannot be used together. Please specify only one.");
386 }
387 let merge = match flag(with, "merge-multiple", false) {
388 Ok(merge) => merge,
389 Err(message) => return fail(self, &message),
Cache hits are counted, not only their bytes390 };
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2391 let dest = self.download_dir(input(with, "path"));
392 // Another run's artifacts, as v4 has it: with a token and a run id.
393 let run_id = match (input(with, "github-token"), input(with, "run-id")) {
394 (Some(_), Some(run)) => {
395 let own = self.github_value("repository");
396 let repository = input(with, "repository").unwrap_or(&own);
397 if !repository.eq_ignore_ascii_case(&own) {
398 return fail(self, &format!("g1t downloads artifacts from other runs of the same repository only; {repository} is not this run's repository, {own}."));
Cache hits are counted, not only their bytes399 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2400 Some(run.to_owned())
Cache hits are counted, not only their bytes401 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2402 _ => None,
403 };
404 let listed = match self.list_artifacts(run_id.as_deref()) {
405 Ok(listed) => listed,
406 Err(error) => return fail(self, &format!("The artifacts could not be listed: {error}")),
407 };
408 let chosen: Vec<Listed> = if let Some(name) = name {
409 match listed.into_iter().find(|a| a.name == name) {
410 Some(artifact) => vec![artifact],
411 None => return fail(self, &format!("Unable to download artifact(s): Artifact not found for name: {name}")),
Cache hits are counted, not only their bytes412 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2413 } else if !ids.is_empty() {
414 let chosen: Vec<Listed> = listed.into_iter().filter(|a| ids.contains(&a.id)).collect();
415 if chosen.is_empty() {
416 return fail(self, "Unable to download artifact(s): None of the provided artifact IDs were found.");
417 }
418 if chosen.len() < ids.len() {
419 self.log.line(&format!("##[warning]Could only find {} of {} artifact IDs.", chosen.len(), ids.len()));
420 }
421 chosen
422 } else {
423 match input(with, "pattern") {
424 Some(pattern) => listed.into_iter().filter(|a| glob::matches_part(pattern, &a.name)).collect(),
425 None => listed,
426 }
427 };
Cache hits are counted, not only their bytes428 let mut outputs = BTreeMap::new();
429 outputs.insert("download-path".into(), dest.display().to_string());
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2430 if chosen.is_empty() {
431 match input(with, "pattern") {
432 Some(pattern) => self.log.line(&format!("No artifacts matched the pattern {pattern}; nothing was downloaded.")),
433 None => self.log.line("The run has no artifacts; nothing was downloaded."),
434 }
435 return (true, outputs);
436 }
437 let targets = download_targets(&dest, &chosen, name.is_some(), !ids.is_empty(), merge);
438 for (artifact, target) in chosen.iter().zip(targets) {
439 let Some(target) = target else {
440 return fail(self, &format!("The artifact {} cannot be downloaded into a folder of its name.", artifact.name));
441 };
442 if let Err(error) = self.fetch_artifact(artifact, &target) {
443 return fail(self, &format!("The artifact {} could not be downloaded: {error}", artifact.name));
444 }
445 self.log.line(&format!("Downloaded artifact {} ({}) into {}", artifact.name, megabytes(artifact.size), target.display()));
446 }
447 if chosen.len() > 1 {
448 self.log.line(&format!("Downloaded {}.", count(chosen.len(), "artifact")));
449 }
Cache hits are counted, not only their bytes450 (true, outputs)
451 }
452
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2453 /// `actions/upload-artifact/merge`: the run's artifacts matching
454 /// `pattern`, downloaded together and uploaded again as one.
455 pub(crate) fn merge_artifacts(&mut self, with: &BTreeMap<String, String>) -> (bool, BTreeMap<String, String>) {
456 let fail = |job: &mut Job, message: &str| {
457 job.log.line(&format!("##[error]{message}"));
458 (false, BTreeMap::new())
459 };
460 let name = input(with, "name").unwrap_or("merged-artifacts").to_owned();
461 let pattern = input(with, "pattern").unwrap_or("*").to_owned();
462 let options = check_name(&name).and_then(|()| {
463 let keep = keep(with, false)?;
464 Ok((keep, flag(with, "separate-directories", false)?, flag(with, "delete-merged", false)?))
465 });
466 let (keep, separate, delete) = match options {
467 Ok(options) => options,
468 Err(message) => return fail(self, &message),
469 };
470 let listed = match self.list_artifacts(None) {
471 Ok(listed) => listed,
472 Err(error) => return fail(self, &format!("The artifacts could not be listed: {error}")),
473 };
474 let chosen: Vec<Listed> = listed.into_iter().filter(|a| glob::matches_part(&pattern, &a.name)).collect();
475 if chosen.is_empty() {
476 return fail(self, &format!("No artifacts found matching pattern '{pattern}'."));
477 }
478 self.log.line(&format!("Merging {}: {}", count(chosen.len(), "artifact"), chosen.iter().map(|a| a.name.as_str()).collect::<Vec<_>>().join(", ")));
479 let folder = self.temp.join(format!("merge-{}", super::rand_id()));
480 let targets = download_targets(&folder, &chosen, false, false, !separate);
481 let mut result = None;
482 for (artifact, target) in chosen.iter().zip(targets) {
483 let Some(target) = target else {
484 result = Some(fail(self, &format!("The artifact {} cannot be merged into a folder of its name.", artifact.name)));
485 break;
486 };
487 if let Err(error) = self.fetch_artifact(artifact, &target) {
488 result = Some(fail(self, &format!("The artifact {} could not be downloaded: {error}", artifact.name)));
489 break;
490 }
491 }
492 let result = result.unwrap_or_else(|| {
493 let root = folder.display().to_string();
494 let found = glob::find(std::slice::from_ref(&root), &root, &self.home(), keep.hidden);
495 if found.files.is_empty() {
496 fail(self, "The artifacts matched hold no files to merge.")
497 } else {
498 self.send_artifact(&name, &found.files, &keep)
499 }
500 });
501 let _ = std::fs::remove_dir_all(&folder);
502 if result.0 && delete {
503 for artifact in &chosen {
504 match ureq::delete(&self.url(&format!("artifacts/{}", artifact.id))).set("authorization", &self.auth()).timeout(Duration::from_secs(60)).call() {
505 Ok(_) => self.log.line(&format!("Deleted artifact {}.", artifact.name)),
506 Err(ureq::Error::Status(_, response)) => self.log.line(&format!("##[warning]Artifact {} could not be deleted: {}", artifact.name, refusal(response))),
507 Err(error) => self.log.line(&format!("##[warning]Artifact {} could not be deleted: {error}", artifact.name)),
508 }
509 }
510 }
511 result
Cache hits are counted, not only their bytes512 }
513
514}
515
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2516/// An input, trimmed; `None` when empty.
517fn input<'a>(with: &'a BTreeMap<String, String>, key: &str) -> Option<&'a str> {
518 with.get(key).map(|v| v.trim()).filter(|v| !v.is_empty())
519}
520
521/// A true-or-false input, as `core.getBooleanInput` reads it.
522fn flag(with: &BTreeMap<String, String>, key: &str, default: bool) -> Result<bool, String> {
523 match input(with, key) {
524 None => Ok(default),
525 Some("true" | "True" | "TRUE") => Ok(true),
526 Some("false" | "False" | "FALSE") => Ok(false),
527 Some(other) => Err(format!("`{key}` is `{other}`; it takes true or false.")),
528 }
529}
530
531/// `1 file`, `3 files`.
532fn count(n: usize, what: &str) -> String {
533 if n == 1 { format!("1 {what}") } else { format!("{n} {what}s") }
534}
535
536/// An id that came as a number or as text.
537fn number(value: &serde_json::Value) -> Option<u64> {
538 value.as_u64().or_else(|| value.as_str().and_then(|s| s.parse().ok()))
539}
540
541/// Whether a name is one GitHub takes for an artifact: not empty, at most
542/// 256 characters, none of `" : < > | * ? \ /` or a line break.
543fn check_name(name: &str) -> Result<(), String> {
544 if name.is_empty() {
545 return Err("The artifact needs a name.".into());
546 }
547 if name.chars().count() > 256 {
548 return Err(format!("The artifact name `{name}` is longer than 256 characters."));
549 }
550 if let Some(bad) = name.chars().find(|c| matches!(c, '"' | ':' | '<' | '>' | '|' | '*' | '?' | '\r' | '\n' | '\\' | '/')) {
551 let shown = match bad {
552 '\r' => "a carriage return".to_owned(),
553 '\n' => "a line break".to_owned(),
554 c => format!("`{c}`"),
555 };
556 return Err(format!("The artifact name `{name}` is not valid: it contains {shown}. A name may not contain \" : < > | * ? \\ / or line breaks."));
557 }
558 Ok(())
559}
560
561/// How an artifact is packed and kept, from the inputs `upload-artifact`
562/// and its merge share.
563#[derive(Debug, PartialEq)]
564struct Keep {
565 /// Days; 0 for the repository's own setting.
566 retention: u32,
567 /// 0 (stored) to 9.
568 level: u32,
569 overwrite: bool,
570 /// Whether files and folders whose names start with `.` are taken.
571 hidden: bool,
572}
573
574fn keep(with: &BTreeMap<String, String>, takes_overwrite: bool) -> Result<Keep, String> {
575 let retention = match input(with, "retention-days") {
576 None => 0,
577 Some(text) => match text.parse::<u32>() {
578 Ok(days @ 0..=90) => days,
579 _ => return Err(format!("`retention-days` is `{text}`; it takes a number of days from 1 to 90, or nothing for the repository's setting.")),
580 },
581 };
582 let level = match input(with, "compression-level") {
583 None => 6,
584 Some(text) => match text.parse::<u32>() {
585 Ok(level @ 0..=9) => level,
586 _ => return Err(format!("`compression-level` is `{text}`; it takes 0 (no compression) to 9.")),
587 },
588 };
589 Ok(Keep {
590 retention,
591 level,
592 overwrite: takes_overwrite && flag(with, "overwrite", false)?,
593 hidden: flag(with, "include-hidden-files", false)?,
594 })
595}
596
597/// What finishing an upload gave: the artifact's id, the ZIP's SHA-256 in
598/// hex, and the days it is kept, when known.
599struct Sent {
600 id: u64,
601 digest: String,
602 retention: Option<u32>,
603}
604
605/// An artifact as g1t lists it.
606#[derive(Debug, Clone, PartialEq)]
607struct Listed {
608 id: u64,
609 name: String,
610 size: u64,
611 digest: Option<String>,
612 /// `zip`, or `tgz` for an artifact an older runner stored.
613 format: String,
614}
615
616impl Listed {
617 fn from_json(value: &serde_json::Value) -> Option<Listed> {
618 Some(Listed {
619 id: number(&value["id"])?,
620 name: value["name"].as_str()?.to_owned(),
621 size: value["size"].as_u64().unwrap_or(0),
622 digest: value["digest"].as_str().map(str::to_owned),
623 format: value["format"].as_str().unwrap_or("zip").to_owned(),
624 })
625 }
626}
627
628/// One artifact per name, the newest (highest id), in name order.
629fn newest(listed: Vec<Listed>) -> Vec<Listed> {
630 let mut by_name: BTreeMap<String, Listed> = BTreeMap::new();
631 for artifact in listed {
632 if by_name.get(&artifact.name).is_none_or(|kept| kept.id < artifact.id) {
633 by_name.insert(artifact.name.clone(), artifact);
634 }
635 }
636 by_name.into_values().collect()
637}
638
639/// `artifact-ids`: numbers, separated by commas.
640fn artifact_ids(text: &str) -> Result<Vec<u64>, String> {
641 text.split(',')
642 .map(str::trim)
643 .filter(|id| !id.is_empty())
644 .map(|id| id.parse::<u64>().map_err(|_| format!("`artifact-ids` takes artifact ids, numbers separated by commas; `{id}` is not one.")))
645 .collect()
646}
647
648/// The folder each artifact is unpacked into: `dest` itself for one
649/// artifact by name, for `merge-multiple`, and for a single artifact by
650/// id; otherwise a folder of the artifact's name in `dest`. `None` for a
651/// name that cannot be a folder.
652fn download_targets(dest: &Path, chosen: &[Listed], by_name: bool, by_ids: bool, merge: bool) -> Vec<Option<std::path::PathBuf>> {
653 let straight = by_name || merge || (by_ids && chosen.len() == 1);
654 chosen
655 .iter()
656 .map(|artifact| {
657 if straight {
658 Some(dest.to_path_buf())
659 } else if artifact.name.is_empty() || artifact.name == "." || artifact.name == ".." || artifact.name.contains(['/', '\\']) {
660 None
661 } else {
662 Some(dest.join(&artifact.name))
663 }
664 })
665 .collect()
666}
667
668/// The SHA-256 of a file, in hex, read as it comes.
669fn sha256_file(path: &Path) -> std::io::Result<String> {
670 let mut file = std::fs::File::open(path)?;
671 let mut hasher = Sha256::new();
672 let mut buffer = vec![0u8; 256 * 1024];
673 loop {
674 let n = file.read(&mut buffer)?;
675 if n == 0 {
676 break;
677 }
678 hasher.update(&buffer[..n]);
679 }
680 Ok(hex::encode(hasher.finalize()))
681}
682
Cache hits are counted, not only their bytes683/// The message of a refused request, from its JSON body.
684fn refusal(response: ureq::Response) -> String {
685 let status = response.status();
686 let body: serde_json::Value = response.into_json().unwrap_or_default();
687 body["error"]["message"].as_str().map_or_else(|| format!("g1t answered {status}"), str::to_owned)
688}
689
690fn megabytes(bytes: u64) -> String {
691 if bytes < 1_048_576 { format!("{} KB", bytes.div_ceil(1024)) } else { format!("{:.1} MB", bytes as f64 / 1_048_576.0) }
692}
693
Merge remote-tracking branch 'origin/claude/cool-newton-7nm8r9' into claude/cool-newton-7nm8r9694/// GNU tar on Windows reads `C:...` as a host and a file; `--force-local`
695/// makes every path a local one. Elsewhere the flag is left off, since
696/// bsdtar (macOS) doesn't know it.
697pub(crate) fn tar_local() -> &'static str {
698 if cfg!(windows) {
699 " --force-local"
700 } else {
701 ""
702 }
703}
704
Cache hits are counted, not only their bytes705fn urlencode(text: &str) -> String {
706 let mut out = String::new();
707 for byte in text.bytes() {
708 if byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.' | b'~') {
709 out.push(byte as char);
710 } else {
711 out.push_str(&format!("%{byte:02X}"));
712 }
713 }
714 out
715}
716
717#[cfg(test)]
718mod tests {
719 use super::*;
720
721 #[test]
A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so.722 fn transfers_say_how_far_they_have_got_as_githubs_do() {
723 assert_eq!(progress("Received", 46_137_344, Some(523_190_823), Duration::from_secs(1)), "Received 46137344 of 523190823 (8.8%), 44.0 MBs/sec");
724 assert_eq!(progress("Sent", 1_048_576, None, Duration::from_secs(2)), "Sent 1048576 bytes, 0.5 MBs/sec");
725 assert_eq!(progress("Received", 10, Some(0), Duration::ZERO), "Received 10 bytes, 0.0 MBs/sec");
726 }
727
728 #[test]
Cache hits are counted, not only their bytes729 fn keys_are_encoded_and_names_checked() {
730 assert_eq!(urlencode("Linux-node-abc/1 2"), "Linux-node-abc%2F1%202");
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2731 assert!(check_name("coverage report").is_ok());
732 assert!(check_name("dist-linux_x64.1 (debug)").is_ok());
733 assert!(check_name("ünïcødé").is_ok());
734 assert!(check_name(&"a".repeat(256)).is_ok());
735 assert!(check_name(&"a".repeat(257)).is_err());
736 assert!(check_name("").is_err());
737 for bad in ["a/b", "a\\b", "a:b", "a*b", "a?b", "a\"b", "a<b", "a>b", "a|b", "a\nb", "a\rb"] {
738 assert!(check_name(bad).is_err(), "{bad}");
739 }
Cache hits are counted, not only their bytes740 assert_eq!(lines("dist/\n\n# note\n coverage \n"), ["dist/", "coverage"]);
741 }
Actions: OIDC tokens, the toolkit's cache and artifact services, and artifacts in R2742
743 fn with(pairs: &[(&str, &str)]) -> BTreeMap<String, String> {
744 pairs.iter().map(|(k, v)| (k.to_string(), v.to_string())).collect()
745 }
746
747 #[test]
748 fn upload_inputs_are_read_as_v4_reads_them() {
749 assert_eq!(keep(&with(&[]), true).unwrap(), Keep { retention: 0, level: 6, overwrite: false, hidden: false });
750 let set = with(&[("retention-days", "14"), ("compression-level", "0"), ("overwrite", "true"), ("include-hidden-files", "True")]);
751 assert_eq!(keep(&set, true).unwrap(), Keep { retention: 14, level: 0, overwrite: true, hidden: true });
752 // The merge takes no `overwrite`.
753 assert!(!keep(&set, false).unwrap().overwrite);
754 assert_eq!(keep(&with(&[("retention-days", "0")]), true).unwrap().retention, 0);
755 assert!(keep(&with(&[("retention-days", "91")]), true).is_err());
756 assert!(keep(&with(&[("retention-days", "-1")]), true).is_err());
757 assert!(keep(&with(&[("retention-days", "two")]), true).is_err());
758 assert!(keep(&with(&[("compression-level", "10")]), true).is_err());
759 assert!(keep(&with(&[("overwrite", "yes")]), true).is_err());
760 assert_eq!(artifact_ids(" 12, 34 ,,").unwrap(), [12, 34]);
761 assert!(artifact_ids("12,abc").is_err());
762 assert_eq!(count(1, "file"), "1 file");
763 assert_eq!(count(37, "file"), "37 files");
764 }
765
766 fn listed(id: u64, name: &str) -> Listed {
767 Listed { id, name: name.into(), size: 0, digest: None, format: "zip".into() }
768 }
769
770 #[test]
771 fn listings_read_and_keep_the_newest_of_a_name() {
772 let json = serde_json::json!([
773 { "id": 1, "name": "dist", "size": 10, "digest": null, "format": "tgz", "created_at": "x", "expires_at": "y" },
774 { "id": "3", "name": "dist", "size": 30, "digest": "sha256:ab", "format": "zip" },
775 { "id": 2, "name": "logs", "size": 5 },
776 { "name": "no id" }
777 ]);
778 let all: Vec<Listed> = json.as_array().unwrap().iter().filter_map(Listed::from_json).collect();
779 assert_eq!(all.len(), 3);
780 assert_eq!(all[0].format, "tgz");
781 let kept = newest(all);
782 assert_eq!(kept.iter().map(|a| (a.id, a.name.as_str())).collect::<Vec<_>>(), [(3, "dist"), (2, "logs")]);
783 assert_eq!(kept[0].digest.as_deref(), Some("sha256:ab"));
784 }
785
786 #[test]
787 fn downloads_land_where_v4_puts_them() {
788 let dest = Path::new("/w/out");
789 let one = [listed(1, "dist")];
790 let two = [listed(1, "dist"), listed(2, "logs")];
791 let at = |targets: Vec<Option<std::path::PathBuf>>| targets.into_iter().map(|t| t.unwrap()).collect::<Vec<_>>();
792 // By name: straight into `path`.
793 assert_eq!(at(download_targets(dest, &one, true, false, false)), [dest.to_path_buf()]);
794 // Every artifact, or by pattern: a folder each.
795 assert_eq!(at(download_targets(dest, &two, false, false, false)), [dest.join("dist"), dest.join("logs")]);
796 assert_eq!(at(download_targets(dest, &one, false, false, false)), [dest.join("dist")]);
797 // merge-multiple: all into `path`.
798 assert_eq!(at(download_targets(dest, &two, false, false, true)), [dest.to_path_buf(), dest.to_path_buf()]);
799 // By id: one straight into `path`, several a folder each.
800 assert_eq!(at(download_targets(dest, &one, false, true, false)), [dest.to_path_buf()]);
801 assert_eq!(at(download_targets(dest, &two, false, true, false)), [dest.join("dist"), dest.join("logs")]);
802 // A name that is no folder is refused.
803 assert!(download_targets(dest, &[listed(1, ".."), listed(2, "x")], false, false, false)[0].is_none());
804 assert!(glob::matches_part("dist-*", "dist-linux"));
805 assert!(!glob::matches_part("dist-*", "logs"));
806 }
Cache hits are counted, not only their bytes807}

This file's history is long; its oldest lines are credited to the oldest commit read.