flagon-io/g1t

public

Merge branch 'worktree-agent-ac79a505eca749d4d' into claude/cool-newton-7nm8r9

# Conflicts: # Cargo.lock # apps/docs/src/content/docs/about/limitations.md # crates/runner/Cargo.toml # crates/runner/src/actions/mod.rs

Claudeagentcommitted 12h agoParentse1e1a96 a5cc983Browse files

Merged inReal GitHub Actions on g1t, and managed deploy workflows #36Why it was written

5	5
6	6		A self-hosted runner is a machine of yours that runs your workflow jobs
7	7		instead of g1t's sandboxes. It can be a server, a Mac in a cupboard, a
8		−	Windows box with a GPU, or a pod in your cluster. You install `g1t-runner`
9		−	on it, register it once, and it picks up jobs that ask for it with
10		−	`runs-on: self-hosted`.
	8	+	Windows box with a GPU, or a pod in your cluster. On Linux you run
	9	+	`g1t-runner` as a container under [Sysbox](#run-it-as-a-container), so
	10	+	neither the runner nor its jobs can reach the machine; it registers once
	11	+	and picks up jobs that ask for it with `runs-on: self-hosted`.
11	12
12	13		- Time on your runners is $0, on every plan, including the free one.
13	14		It shows on [usage](/guides/usage-and-billing/) as self-hosted minutes.
14	15		- It only connects out. The runner polls `api.g1t.sh` over HTTPS for
15	16		work. Nothing needs to reach the machine: no open port, no tunnel.
16		−	- Any OS. Linux, macOS and Windows, on x64 and arm64. Jobs run in a
17		−	Docker container by default, or directly on the machine.
	17	+	- Each job in a fresh sandbox. Every job gets a container of its own,
	18	+	made for it and removed after it, inside the runner's own container.
	19	+	See [isolation](#isolation).
	20	+	- Any OS. Linux, macOS and Windows, on x64 and arm64. On macOS and
	21	+	Windows, sandboxed jobs are [not built yet](#windows-and-macos).
18	22
19	23		\| Where \| Page \| Who manages it \|
20	24		\| --- \| --- \| --- \|
⋯
24	28		A workspace's runners serve the repositories their [group](#groups) allows.
25	29		A project's own runners serve only that project.
26	30
	31	+	## Run it as a container
	32	+
	33	+	On a Linux machine, run the runner's image with the Sysbox runtime. The
	34	+	runner starts a Docker Engine of its own inside its container and runs
	35	+	each job in a container on that engine, so jobs are containers inside the
	36	+	runner's container. Sysbox gives the runner's container a user namespace
	37	+	of its own, so its root is an unprivileged user on the machine, and lets
	38	+	it run Docker without `--privileged` and without the machine's Docker
	39	+	socket.
	40	+
	41	+	1. Install [Sysbox](https://github.com/nestybox/sysbox) on the machine,
	42	+	following its install guide. It needs Linux; Docker Desktop on macOS
	43	+	and Windows cannot use it.
	44	+	2. Make a registration token: open Runners and click New runner.
	45	+	3. Run the runner:
	46	+
	47	+	```sh
	48	+	docker run -d --name g1t-runner --restart unless-stopped \
	49	+	--runtime=sysbox-runc \
	50	+	-v g1t-runner:/data \
	51	+	-e G1T_RUNNER_URL=https://g1t.sh \
	52	+	-e G1T_RUNNER_TOKEN=g1trt_… \
	53	+	g1t.sh/flagon-io/g1t-runner
	54	+	```
	55	+
	56	+	The image runs `register-and-run`, which registers once and then runs.
	57	+	Its configuration and credential are kept on the `g1t-runner` volume, so
	58	+	the container can be restarted and replaced without registering again.
	59	+	Every [register option](#register-options) can come from
	60	+	`G1T_RUNNER_<OPTION>` in the environment, such as `G1T_RUNNER_LABELS=gpu`
	61	+	or `G1T_RUNNER_MEMORY=8g`, so a secret can hold the token.
	62	+
	63	+	Its log says which isolation it has:
	64	+
	65	+	```text
	66	+	… Isolation: nested-container: each job runs in a fresh container on this runner's own Docker Engine, inside the runner's sandboxed container; neither reaches the machine.
	67	+	```
	68	+
	69	+	Do not give the container the machine's Docker socket
	70	+	(`-v /var/run/docker.sock:/var/run/docker.sock`) or `--privileged`. The
	71	+	runner looks for both when it starts, and refuses to start if it finds
	72	+	them:
	73	+
	74	+	\| It finds \| What it says \|
	75	+	\| --- \| --- \|
	76	+	\| An engine's socket mounted in from the machine (`docker.sock`, `containerd.sock`, `podman.sock`, under any name) \| … is a container engine's socket mounted in from outside this container \|
	77	+	\| An engine's socket it did not make, such as from a mounted `/var/run` \| … is a container engine's socket that this runner did not make \|
	78	+	\| `DOCKER_HOST` set \| DOCKER_HOST is set … \|
	79	+	\| Root in the container is root on the machine (no Sysbox, or `--privileged`) \| root in this container is root on the machine \|
	80	+
	81	+	To start anyway, with jobs able to reach the machine, set
	82	+	`G1T_RUNNER_UNSAFE_ALLOW_HOST_ACCESS=1`. The runner then logs each
	83	+	problem with `UNSAFE:` in front, and registers as `container` rather than
	84	+	`nested-container`.
	85	+
	86	+	The engine's images and build cache are kept in the container, so they
	87	+	last until the container is removed. Sysbox keeps them on the machine's
	88	+	disk for that container alone; to keep them when you replace the
	89	+	container, add `-v g1t-runner-docker:/var/lib/docker`, a volume for one
	90	+	runner container at a time.
	91	+
	92	+	## Isolation
	93	+
	94	+	How well a runner keeps work from its machine. A runner tells g1t its
	95	+	level when it registers.
	96	+
	97	+	\| Isolation \| How you get it \| What a job can reach \|
	98	+	\| --- \| --- \| --- \|
	99	+	\| `nested-container` \| The runner's image under Sysbox, as above. \| Its own container, and its own Docker Engine if it uses Docker. Not the machine, and not other jobs. \|
	100	+	\| `container` \| `g1t-runner` installed on the machine with Docker, or the image given the machine's Docker socket with the override. \| Its own container. The runner itself drives the machine's Docker, which can do anything on the machine, so a job that escapes its container, or anything that takes over the runner, controls the machine. \|
	101	+	\| `host` \| `--no-docker`. Deprecated. \| Everything the runner's user can: the machine, its files, and what earlier jobs left. \|
	102	+
	103	+	### What a job's container gets
	104	+
	105	+	With `nested-container` and `container` isolation, each job runs in a
	106	+	container made for it and removed after it:
	107	+
	108	+	\| \| \|
	109	+	\| --- \| --- \|
	110	+	\| Image \| The job's `container:`, else the runner's `--image`, else `node:24-bookworm`. \|
	111	+	\| Mounts \| The harness (`g1t-runner`), read-only, and a volume of its own at `/home/runner`, removed after the job. Nothing else of the runner's or the machine's. \|
	112	+	\| Network \| A Docker network of its own, removed after the job. Never the machine's network. \|
	113	+	\| Capabilities \| Every one dropped but `CHOWN`, `DAC_OVERRIDE`, `FOWNER`, `FSETID`, `SETUID`, `SETGID`, `KILL` and `NET_BIND_SERVICE`, which installing packages and running test servers need. Raw sockets (`ping`) are among those dropped. \|
	114	+	\| Privileges \| `no-new-privileges`: a setuid program such as `sudo` gains nothing. Run steps as the image's root user rather than with `sudo`. Agent work keeps `sudo`, which its image uses once to put its guardrails out of the agent's reach. \|
	115	+	\| Limits \| `--memory` (three quarters of the runner's memory unless you set it, with no swap beyond it), `--cpus` (every CPU unless you set it) and `--pids-limit` (4096 unless you set it). \|
	116	+	\| Credentials \| Only the job's own short-lived token, passed by name. Never the runner's credential. \|
	117	+
	118	+	### Docker in jobs
	119	+
	120	+	With `nested-container` isolation, a job that uses Docker (its
	121	+	`services:`, a Docker action, a `docker://` step, or `docker` in a step)
	122	+	gets a Docker Engine of its own. It starts the first time the job uses
	123	+	Docker, from `docker:28-dind`, beside the job's container and on the job's
	124	+	network, so a service's published ports are on the job's `localhost`. It
	125	+	is removed, with its images and containers, when the job ends. A job that
	126	+	does not use Docker never starts one.
	127	+
	128	+	The job's engine is privileged inside the runner's container. Sysbox keeps
	129	+	it off the machine, but a job that uses Docker can reach the runner's own
	130	+	container, the runner's credential included. For code you do not trust,
	131	+	use [ephemeral runners](#autoscaling), so each job has a runner container
	132	+	of its own, or register with `--no-job-docker`, which gives jobs no Docker.
	133	+
	134	+	With `container` isolation, jobs have no Docker of their own, since it
	135	+	would be the machine's: their `services:` are not started, and the log
	136	+	says so.
	137	+
	138	+	### Host mode is deprecated
	139	+
	140	+	`--no-docker` still works, and runs each job directly on the machine in a
	141	+	folder of its own. It will be removed. Until then:
	142	+
	143	+	- The runner's log says so when it starts: *Deprecated: this runner runs
	144	+	jobs directly on this machine (--no-docker), unsandboxed.*
	145	+	- Every job's log starts with a warning: *This job ran on the host,
	146	+	unsandboxed …*
	147	+	- It refuses a public repository's work, which anyone can propose code
	148	+	to: *acme/web is public, and this self-hosted runner runs jobs directly
	149	+	on its machine (--no-docker), unsandboxed, so it does not take a public
	150	+	repository's work.* The job fails with that message.
	151	+
	152	+	Move a Linux runner to [the container](#run-it-as-a-container). A macOS
	153	+	or Windows runner stays in host mode until sandboxes for them are built.
	154	+
	155	+	### Windows and macOS
	156	+
	157	+	Sandboxed jobs on Windows and macOS are not built yet; jobs native to
	158	+	those systems run in host mode. What is planned:
	159	+
	160	+	\| \| How each job will run \|
	161	+	\| --- \| --- \|
	162	+	\| Windows \| A Windows container with Hyper-V isolation: a small virtual machine with its own kernel, made from a signed image for the job and destroyed after it. \|
	163	+	\| macOS \| An ephemeral macOS virtual machine, using Apple's Virtualization framework, cloned from a signed image for the job and deleted after it. Apple's licence allows two macOS virtual machines per Mac at once, so a Mac will run at most two jobs at a time. \|
	164	+
	165	+	On both, the only part on the machine itself will be a small launcher that
	166	+	makes and destroys those sandboxes from signed images; the runner and the
	167	+	job run inside them.
	168	+
	169	+	On a Mac or Windows machine with Docker Desktop, a runner installed there
	170	+	without `--no-docker` already runs jobs in Linux containers, with
	171	+	`container` isolation.
	172	+
27	173		## The Runners page
28	174
29	175		A workspace's Runners page shows g1t cloud and your own runners side
⋯
51	197
52	198		## Add a runner
53	199
	200	+	On Linux, [run it as a container](#run-it-as-a-container). To install
	201	+	`g1t-runner` on a machine instead, as on macOS and Windows:
	202	+
54	203		1. Open Runners (a project's: Settings → Runners) and click New runner. g1t makes a
55	204		registration token and shows the commands for Linux, macOS, Windows and
56	205		Docker with it filled in. The token lasts an hour, can register any
⋯
90	239
91	240		The runner shows up in the list within a few seconds, with a green dot.
92	241
	242	+	Installed this way, it has `container` isolation with Docker, or `host`
	243	+	isolation with `--no-docker`. See [isolation](#isolation).
	244	+
93	245		### Register options
94	246
95	247		\| Option \| \|
⋯
100	252		\| `--labels` \| Extra labels, comma-separated: `gpu,cuda-12`. See [labels](#labels). \|
101	253		\| `--group` \| A workspace runner's [group](#groups). The default group, or the one the token was made for, if left out. \|
102	254		\| `--ephemeral` \| Take one job, then remove itself. For [autoscaling](#autoscaling). \|
103		−	\| `--no-docker` \| Run jobs directly on the machine instead of in containers. \|
	255	+	\| `--no-docker` \| Deprecated. Run jobs directly on the machine instead of in containers ([host mode](#host-mode-is-deprecated)). \|
104	256		\| `--image` \| The image workflow jobs run in when they name no `container:`. `node:24-bookworm` if left out. \|
105	257		\| `--agent-image` \| The image [agent work](#agents-on-your-runners) runs in. \|
	258	+	\| `--memory` \| Each job container's memory, such as `8g`. Three quarters of the runner's memory if left out. \|
	259	+	\| `--cpus` \| Each job container's CPUs, such as `4` or `1.5`. Every CPU if left out. \|
	260	+	\| `--pids-limit` \| The most processes in a job container. 4096 if left out. \|
	261	+	\| `--no-job-docker` \| Give jobs no [Docker of their own](#docker-in-jobs). \|
	262	+	\| `--job-docker-image` \| The image a job's own Docker Engine runs. `docker:28-dind` if left out. \|
106	263		\| `--work-dir` \| Where jobs run with `--no-docker` keep their files. `~/.g1t-runner/work` if left out. \|
107	264		\| `--dir` \| Where the runner keeps its configuration and credential. `~/.g1t-runner`, or `G1T_RUNNER_DIR`, if left out. Every command takes it. \|
108	265		\| `--replace` \| Take the place of a runner with the same name. \|
⋯
162	319		and `runner.arch` are the machine's, and `RUNNER_ENVIRONMENT` is
163	320		`self-hosted`.
164	321
165		−	- In Docker (the default), each job gets a fresh container from its
166		−	`container:` image or the runner's `--image`, removed when it ends. The
167		−	runner needs Docker, and its user needs to be allowed to use it. Its
168		−	`services:` are not started, since the job's container has no Docker of
169		−	its own; the log says so.
170		−	- With `--no-docker`, each job gets a fresh folder under the work
171		−	folder, removed when it ends, and runs with whatever the machine has
172		−	installed. A step's default shell is `bash` on Linux and macOS and
173		−	PowerShell on Windows; `shell: pwsh`, `powershell`, `cmd`, `bash` and
174		−	`python` work where installed. On a machine with Docker, the job's
175		−	`services:`, `container:`, `docker://` steps and Docker actions use
176		−	the machine's Docker, as on GitHub's runners.
	322	+	- In a container (the default), each job gets a fresh container from
	323	+	its `container:` image or the runner's `--image`, removed when it ends,
	324	+	with the walls in [what a job's container gets](#what-a-jobs-container-gets).
	325	+	In the runner's image under Sysbox, its `services:` and Docker actions
	326	+	get [a Docker Engine of its own](#docker-in-jobs).
	327	+	- With `--no-docker` (deprecated), each job gets a fresh folder under
	328	+	the work folder, removed when it ends, and runs with whatever the
	329	+	machine has installed. A step's default shell is `bash` on Linux and
	330	+	macOS and PowerShell on Windows; `shell: pwsh`, `powershell`, `cmd`,
	331	+	`bash` and `python` work where installed. On a machine with Docker, the
	332	+	job's `services:`, `container:`, `docker://` steps and Docker actions
	333	+	use the machine's Docker, as on GitHub's runners.
177	334
178	335		A self-hosted job stops at 360 minutes unless its `timeout-minutes` says
179	336		otherwise, as on GitHub, and can ask for up to 24 hours (1440). Jobs on
⋯
205	362		your own [model provider](/guides/models/), g1t charges nothing for the
206	363		run; otherwise the model is paid as usual and the machine is free.
207	364		- Agent work needs an image with git, Node and the agent's CLI: register
208		−	the runner with `--agent-image` and an image of yours that has them, or
209		−	run it with `--no-docker` on a Linux machine set aside for it, where
210		−	`/work` can be written. g1t does not publish an agent image yet.
	365	+	the runner with `--agent-image` and an image of yours that has them.
	366	+	Host mode on a Linux machine set aside for it, where `/work` can be
	367	+	written, also works, and is deprecated. g1t does not publish an agent
	368	+	image yet.
211	369		- [Guardrails](/guides/guardrails/) still apply to what the agent does, but
212	370		their network list cannot be enforced on your machine. The run's session
213	371		says so when it starts.
⋯
240	398		it. It is kept only as a hash on g1t's side, rotates every day, and stops
241	399		working the moment the runner is removed. It is never given to a job:
242	400		each job gets only its own short-lived token.
243		−	- Isolation: a job in Docker gets a container of its own, removed after.
244		−	A job with `--no-docker` runs as the runner's user, in a folder of its
245		−	own; it can reach whatever that user can. Use a dedicated account, and
246		−	ephemeral runners for untrusted code.
	401	+	- Isolation: run the runner's image under Sysbox. Each job then gets
	402	+	a container of its own inside the runner's container, removed after,
	403	+	and neither reaches the machine. A job with `--no-docker` runs as the
	404	+	runner's user and can reach whatever that user can; host mode is
	405	+	deprecated and refuses public repositories' work. Use ephemeral runners
	406	+	for code you do not trust. See [isolation](#isolation).
247	407		- Network: g1t's network guardrails are not enforced on your machines.
248	408		A job reaches whatever the machine can.
249	409		- Audit: making a registration token, registering, removing, and every
⋯
256	416
257	417		## Docker and Kubernetes
258	418
259		−	The runner's image runs `register-and-run`, which registers once and then
260		−	runs. Each option can also come from `G1T_RUNNER_<OPTION>` in the
261		−	environment, such as `G1T_RUNNER_TOKEN` and `G1T_RUNNER_LABELS`, so a
262		−	secret can hold the token:
	419	+	[Run it as a container](#run-it-as-a-container) covers Docker. The image's
	420	+	settings, besides the register options:
263	421
264		−	```sh
265		−	docker run -d --name g1t-runner --restart unless-stopped \
266		−	-v /var/run/docker.sock:/var/run/docker.sock \
267		−	-v g1t-runner:/data -e G1T_RUNNER_DIR=/data \
268		−	g1t.sh/flagon-io/g1t-runner register-and-run --url https://g1t.sh --token g1trt_…
269		−	```
	422	+	\| Variable \| \|
	423	+	\| --- \| --- \|
	424	+	\| `G1T_RUNNER_SANDBOX` \| `nested` in the image: start a Docker Engine inside the container and run jobs on it. \|
	425	+	\| `G1T_RUNNER_DIR` \| `/data` in the image: where the configuration and credential are kept. \|
	426	+	\| `G1T_RUNNER_UNSAFE_ALLOW_HOST_ACCESS` \| `1` starts the runner even when it can reach the machine. See [above](#run-it-as-a-container). \|
270	427
271		−	With the host's Docker socket, each job runs in a sibling container.
	428	+	A runner that ran the image with the machine's Docker socket, as these
	429	+	docs said before, refuses to start with the new image. Remove the
	430	+	`-v /var/run/docker.sock:/var/run/docker.sock` and add
	431	+	`--runtime=sysbox-runc`; the `/data` volume keeps its registration.
272	432
273	433		### Autoscaling
274	434
275		−	Ephemeral runners take one job and remove themselves, so each job starts on
276		−	a clean machine. In Kubernetes, run them with `--no-docker` in pods that
277		−	make their own registration token from an owner's personal
278		−	[access token](/guides/authentication/#access-tokens) with only
	435	+	Ephemeral runners take one job and remove themselves, so each job starts in
	436	+	a fresh runner container. In Kubernetes with Sysbox installed on the
	437	+	nodes (its Kubernetes install adds the `sysbox-runc` runtime class), run
	438	+	them in pods that make their own registration token from an owner's
	439	+	personal [access token](/guides/authentication/#access-tokens) with only
279	440		`runners:admin`, and let the Deployment replace each pod that finishes:
280	441
281	442		```yaml
⋯
287	448		replicas: 3
288	449		selector: { matchLabels: { app: g1t-runner } }
289	450		template:
290		−	metadata: { labels: { app: g1t-runner } }
	451	+	metadata:
	452	+	labels: { app: g1t-runner }
	453	+	annotations: { io.kubernetes.cri-o.userns-mode: "auto:size=65536" }
291	454		spec:
	455	+	runtimeClassName: sysbox-runc
292	456		containers:
293	457		- name: runner
294	458		image: g1t.sh/flagon-io/g1t-runner
⋯
304	468		- { name: G1T_RUNNER_URL, value: https://g1t.sh }
305	469		- { name: G1T_RUNNER_LABELS, value: k8s }
306	470		- { name: G1T_RUNNER_EPHEMERAL, value: "1" }
307		−	- { name: G1T_RUNNER_NO_DOCKER, value: "1" }
308	471		- name: G1T_ADMIN_TOKEN
309	472		valueFrom: { secretKeyRef: { name: g1t, key: runners-admin-token } }
310	473		```
⋯
350	513		\| `register` says the token is not valid \| It expired after an hour, or was mistyped. Make a new one. \|
351	514		\| `register` says a runner with that name is registered \| Choose another `--name`, or add `--replace`. \|
352	515		\| `run` says g1t no longer knows this runner \| It was removed, or its credential was. Register it again. \|
353		−	\| could not run docker \| Install Docker and start it, give the runner's user access to it, or register with `--no-docker`. \|
354		−	\| Agent work fails at once \| Give the runner `--agent-image`, or run it on Linux with `--no-docker`. \|
	516	+	\| this runner is not sandboxed from its machine, so it will not take work \| The runner's container can reach the machine: it lists why. Remove the machine's Docker socket and `--privileged`, and run it with `--runtime=sysbox-runc`. See [run it as a container](#run-it-as-a-container). \|
	517	+	\| the runner's own Docker Engine could not start \| The container is not running under Sysbox. Add `--runtime=sysbox-runc`, and check that Sysbox is installed (`docker info` lists `sysbox-runc` under runtimes). \|
	518	+	\| … is public, and this self-hosted runner runs jobs directly on its machine \| A runner in host mode does not take public repositories' work. Run the runner's image under Sysbox. \|
	519	+	\| A step's `sudo` fails \| Job containers have `no-new-privileges`. Steps run as the image's user, root in most CI images, so drop the `sudo`. \|
	520	+	\| could not run docker \| Install Docker and start it, and give the runner's user access to it. \|
	521	+	\| Agent work fails at once \| Give the runner `--agent-image`. \|
355	522		\| The runner is offline in the list \| It has not polled for 90 seconds. Check its log (`journalctl -u g1t-runner-<name>`, `~/.g1t-runner/runner.log`, or the terminal) and that the machine can reach `https://api.g1t.sh`. \|

1	+	//! Docker Engines of the runner's own, for `nested-container` isolation
2	+	//! (sandbox.rs):
3	+	//!
4	+	//! - The runner's engine. In the runner's image under Sysbox
5	+	//! (`G1T_RUNNER_SANDBOX=nested`), the runner starts `dockerd` inside its
6	+	//! own container and runs every job's container on it. It is a child of
7	+	//! the runner, which `docker-init` (tini) supervises as PID 1, and is
8	+	//! started again before a job if it stopped. Started the way g1t's own
9	+	//! sandboxes start a job's engine (crate::docker::engine): the cgroup
10	+	//! controllers handed down first, as moby's `hack/dind` does, and plain
11	+	//! copies when the disk takes no overlays. Unlike there, Sysbox lets it
12	+	//! keep its packet filter, so each job gets a bridge network of its own.
13	+	//! - A job's engine. A job that uses Docker (`services:`, a Docker
14	+	//! action, `docker` in a step) gets an engine of its own, started the
15	+	//! first time the job's Docker socket is used: a `docker:dind` container
16	+	//! beside the job's, in the job's network namespace, so what it
17	+	//! publishes is on the job's `localhost`. The job sees only a socket
18	+	//! that the runner passes on. The engine is privileged within the
19	+	//! runner's container, never on the machine (Sysbox maps its root to an
20	+	//! unprivileged user there), and is removed with its data after the job.
21	+
22	+	use std::io::{self, Read, Write};
23	+	use std::os::unix::net::{UnixListener, UnixStream};
24	+	use std::path::{Path, PathBuf};
25	+	use std::process::{Child, Command, Stdio};
26	+	use std::sync::atomic::{AtomicBool, Ordering};
27	+	use std::sync::{Arc, Mutex, OnceLock};
28	+	use std::time::{Duration, Instant};
29	+
30	+	use anyhow::{Context, Result, bail};
31	+	use serde_json::{Value, json};
32	+
33	+	use super::log;
34	+
35	+	/// Everything of the runner's engine that is not its data.
36	+	pub const DIR: &str = "/run/g1t-runner";
37	+	/// The runner's engine's socket: not `/var/run/docker.sock`, so a socket
38	+	/// there is never the runner's own (sandbox.rs looks for one).
39	+	pub const SOCKET: &str = "/run/g1t-runner/docker.sock";
40	+	/// Where a job's sockets are kept, a folder each.
41	+	pub const JOBS_DIR: &str = "/run/g1t-runner/jobs";
42	+	/// Where the job sees its Docker socket.
43	+	pub const JOB_SOCKET_DIR: &str = "/run/g1t-docker";
44	+	/// The Docker CLI in the runner's image, lent to a job read-only.
45	+	pub const DOCKER_CLI: &str = "/usr/local/bin/docker";
46	+	/// The image a job's engine runs.
47	+	pub const DEFAULT_JOB_ENGINE_IMAGE: &str = "docker:28-dind";
48	+
49	+	const LOG: &str = "/run/g1t-runner/dockerd.log";
50	+	/// From moby's `hack/dind`, as crate::docker::engine does it.
51	+	const CGROUP_NESTING: &str = "if [ -f /sys/fs/cgroup/cgroup.controllers ] && [ -z \"$(cat /sys/fs/cgroup/cgroup.subtree_control)\" ]; then \
52	+	mkdir -p /sys/fs/cgroup/init && xargs -rn1 < /sys/fs/cgroup/cgroup.procs > /sys/fs/cgroup/init/cgroup.procs 2>/dev/null; \
53	+	sed -e 's/ / +/g' -e 's/^/+/' < /sys/fs/cgroup/cgroup.controllers > /sys/fs/cgroup/cgroup.subtree_control; fi";
54	+
55	+	/// The engine the runner's `docker` commands use: its own, once started.
56	+	static HOST: OnceLock<String> = OnceLock::new();
57	+	/// The runner's engine, while it runs.
58	+	static ENGINE: Mutex<Option<Child>> = Mutex::new(None);
59	+
60	+	/// Whether the runner uses an engine of its own.
61	+	pub fn in_use() -> bool {
62	+	HOST.get().is_some()
63	+	}
64	+
65	+	/// `docker`, pointed at the runner's own engine when it has one.
66	+	pub fn docker() -> Command {
67	+	let mut command = Command::new("docker");
68	+	if let Some(host) = HOST.get() {
69	+	command.env("DOCKER_HOST", host);
70	+	}
71	+	command
72	+	}
73	+
74	+	/// The runner's engine's `daemon.json`. `copies`: the disk takes no
75	+	/// overlays. Its own address ranges, away from Docker's defaults, which
76	+	/// the machine's engine (and so the runner's own address) already uses.
77	+	pub fn daemon_config(copies: bool) -> Value {
78	+	let mut config = json!({
79	+	"hosts": [format!("unix://{SOCKET}")],
80	+	"pidfile": format!("{DIR}/dockerd.pid"),
81	+	"bip": "172.31.254.1/24",
82	+	"default-address-pools": [{ "base": "172.30.0.0/16", "size": 24 }],
83	+	"registry-mirrors": [crate::docker::engine::MIRROR],
84	+	"log-driver": "json-file",
85	+	"log-opts": { "max-size": "20m", "max-file": "2" },
86	+	});
87	+	if copies {
88	+	config["storage-driver"] = json!("vfs");
89	+	}
90	+	config
91	+	}
92	+
93	+	/// A request to an engine's socket; its status.
94	+	fn ping(socket: &Path) -> Option<u16> {
95	+	let mut stream = UnixStream::connect(socket).ok()?;
96	+	stream.set_read_timeout(Some(Duration::from_secs(5))).ok()?;
97	+	stream.write_all(b"GET /_ping HTTP/1.1\r\nHost: docker\r\nConnection: close\r\n\r\n").ok()?;
98	+	let mut head = [0u8; 12];
99	+	stream.read_exact(&mut head).ok()?;
100	+	std::str::from_utf8(&head[9..12]).ok()?.parse().ok()
101	+	}
102	+
103	+	fn tail(path: &Path, lines: usize) -> String {
104	+	let text = std::fs::read_to_string(path).unwrap_or_default();
105	+	let all: Vec<&str> = text.lines().filter(\|l\| !l.trim().is_empty()).collect();
106	+	all[all.len().saturating_sub(lines)..].join("\n")
107	+	}
108	+
109	+	/// Starts the runner's own engine, or finds it running (the runner
110	+	/// restarted after an update); its version.
111	+	pub fn start() -> Result<String> {
112	+	let _ = HOST.set(format!("unix://{SOCKET}"));
113	+	let mut engine = ENGINE.lock().unwrap_or_else(\|p\| p.into_inner());
114	+	if let Some(child) = engine.as_mut()
115	+	&& matches!(child.try_wait(), Ok(None))
116	+	&& ping(Path::new(SOCKET)) == Some(200)
117	+	{
118	+	return Ok(version());
119	+	}
120	+	if engine.is_none() && ping(Path::new(SOCKET)) == Some(200) {
121	+	return Ok(version());
122	+	}
123	+	std::fs::create_dir_all(DIR).with_context(\|\| format!("could not make {DIR}"))?;
124	+	let _ = Command::new("sh").args(["-c", CGROUP_NESTING]).status();
125	+	let mut last = String::new();
126	+	for copies in [false, true] {
127	+	std::fs::write(format!("{DIR}/daemon.json"), serde_json::to_vec_pretty(&daemon_config(copies))?)?;
128	+	let _ = std::fs::remove_file(SOCKET);
129	+	let out = std::fs::File::create(LOG).with_context(\|\| format!("could not write {LOG}"))?;
130	+	let err = out.try_clone()?;
131	+	let mut child = Command::new("dockerd")
132	+	.args(["--config-file", &format!("{DIR}/daemon.json")])
133	+	.env_remove("DOCKER_HOST")
134	+	.stdin(Stdio::null())
135	+	.stdout(out)
136	+	.stderr(err)
137	+	.spawn()
138	+	.context("could not run dockerd: the runner's image has it; is this the runner's image?")?;
139	+	let until = Instant::now() + Duration::from_secs(90);
140	+	loop {
141	+	if ping(Path::new(SOCKET)) == Some(200) {
142	+	*engine = Some(child);
143	+	let version = version();
144	+	return Ok(if copies { format!("{version} (plain-copy storage: this disk takes no overlays)") } else { version });
145	+	}
146	+	if let Ok(Some(status)) = child.try_wait() {
147	+	last = format!("dockerd stopped ({status}):\n{}", tail(Path::new(LOG), 15));
148	+	break;
149	+	}
150	+	if Instant::now() >= until {
151	+	let _ = child.kill();
152	+	let _ = child.wait();
153	+	last = format!("dockerd did not answer in 90 s:\n{}", tail(Path::new(LOG), 15));
154	+	break;
155	+	}
156	+	std::thread::sleep(Duration::from_millis(200));
157	+	}
158	+	}
159	+	bail!(
160	+	"the runner's own Docker Engine could not start. Under Sysbox it needs no more than `docker run --runtime=sysbox-runc`; without Sysbox it cannot start without --privileged, which this runner does not use.\n{last}"
161	+	)
162	+	}
163	+
164	+	/// Before a job: the runner's engine is running, started again if not.
165	+	pub fn ensure_running() -> Result<()> {
166	+	let stopped = {
167	+	let mut engine = ENGINE.lock().unwrap_or_else(\|p\| p.into_inner());
168	+	match engine.as_mut() {
169	+	Some(child) => !matches!(child.try_wait(), Ok(None)),
170	+	None => ping(Path::new(SOCKET)) != Some(200),
171	+	}
172	+	};
173	+	if stopped {
174	+	log("The runner's Docker Engine stopped; starting it again.");
175	+	start()?;
176	+	}
177	+	Ok(())
178	+	}
179	+
180	+	fn version() -> String {
181	+	docker()
182	+	.args(["version", "--format", "{{.Server.Version}}"])
183	+	.output()
184	+	.ok()
185	+	.filter(\|o\| o.status.success())
186	+	.map(\|o\| String::from_utf8_lossy(&o.stdout).trim().to_owned())
187	+	.filter(\|v\| !v.is_empty())
188	+	.unwrap_or_else(\|\| "?".into())
189	+	}
190	+
191	+	// --- A job's own engine -------------------------------------------------
192	+
193	+	/// A job's Docker: a socket now, an engine the first time it is used.
194	+	pub struct JobEngine {
195	+	/// `JOBS_DIR/<job container>`: `proxy/docker.sock` (the job's) and
196	+	/// `engine/docker.sock` (its engine's).
197	+	pub dir: PathBuf,
198	+	/// The engine's container and its data's volume.
199	+	pub container: String,
200	+	pub volume: String,
201	+	stop: Arc<AtomicBool>,
202	+	}
203	+
204	+	/// What starting it needs.
205	+	#[derive(Clone)]
206	+	struct EngineSpec {
207	+	image: String,
208	+	container: String,
209	+	volume: String,
210	+	/// The job's container, whose network the engine joins.
211	+	job: String,
212	+	label: String,
213	+	engine_dir: PathBuf,
214	+	}
215	+
216	+	/// The `docker run` for a job's engine. Privileged within the runner's
217	+	/// container: Sysbox maps its root to an unprivileged user on the
218	+	/// machine, and gives the runner's container no devices of the machine's.
219	+	fn engine_args(spec: &EngineSpec) -> Vec<String> {
220	+	let mut args: Vec<String> = ["run", "-d", "--rm", "--name", &spec.container, "--label", &spec.label, "--privileged", "--pull", "missing"]
221	+	.iter()
222	+	.map(\|s\| s.to_string())
223	+	.collect();
224	+	args.extend(["--network".into(), format!("container:{}", spec.job)]);
225	+	// No TLS certificates and no TCP port: only the socket, which the
226	+	// runner passes on.
227	+	args.extend(["-e".into(), "DOCKER_TLS_CERTDIR=".into()]);
228	+	args.extend(["-v".into(), format!("{}:/run/g1t-engine", spec.engine_dir.display())]);
229	+	args.extend(["-v".into(), format!("{}:/var/lib/docker", spec.volume)]);
230	+	args.extend([spec.image.clone(), "dockerd".into(), "--host=unix:///run/g1t-engine/docker.sock".into()]);
231	+	args
232	+	}
233	+
234	+	fn start_job_engine(spec: &EngineSpec) -> Result<(), String> {
235	+	let begun = Instant::now();
236	+	let output = docker().args(engine_args(spec)).output().map_err(\|e\| format!("could not run docker: {e}"))?;
237	+	if !output.status.success() {
238	+	return Err(format!("could not start the job's Docker Engine: {}", String::from_utf8_lossy(&output.stderr).trim()));
239	+	}
240	+	let socket = spec.engine_dir.join("docker.sock");
241	+	while begun.elapsed() < Duration::from_secs(120) {
242	+	if ping(&socket) == Some(200) {
243	+	log(&format!("Started the job's own Docker Engine in {:.1}s.", begun.elapsed().as_secs_f64()));
244	+	return Ok(());
245	+	}
246	+	std::thread::sleep(Duration::from_millis(200));
247	+	}
248	+	Err("the job's Docker Engine did not answer in 120 s".into())
249	+	}
250	+
251	+	/// Passes one connection on to the job's engine, starting it first.
252	+	fn serve(mut client: UnixStream, spec: &EngineSpec, started: &Mutex<Option<Result<(), String>>>) {
253	+	let ready = {
254	+	let mut started = started.lock().unwrap_or_else(\|p\| p.into_inner());
255	+	if started.is_none() {
256	+	*started = Some(start_job_engine(spec));
257	+	}
258	+	started.clone().unwrap_or(Ok(()))
259	+	};
260	+	if let Err(problem) = ready {
261	+	let body = serde_json::to_string(&json!({ "message": problem })).unwrap_or_default();
262	+	let _ = write!(client, "HTTP/1.1 503 Service Unavailable\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", body.len());
263	+	return;
264	+	}
265	+	let Ok(engine) = UnixStream::connect(spec.engine_dir.join("docker.sock")) else { return };
266	+	let (Ok(mut client_read), Ok(mut engine_write)) = (client.try_clone(), engine.try_clone()) else { return };
267	+	let upstream = std::thread::spawn(move \|\| {
268	+	let _ = io::copy(&mut client_read, &mut engine_write);
269	+	let _ = engine_write.shutdown(std::net::Shutdown::Write);
270	+	});
271	+	let mut engine_read = engine;
272	+	let _ = io::copy(&mut engine_read, &mut client);
273	+	let _ = client.shutdown(std::net::Shutdown::Both);
274	+	let _ = upstream.join();
275	+	}
276	+
277	+	impl JobEngine {
278	+	/// Sets a job's Docker socket up, with no engine behind it yet.
279	+	pub fn prepare(job: &str, runner: &str, image: &str) -> Result<JobEngine> {
280	+	let dir = Path::new(JOBS_DIR).join(job);
281	+	let _ = std::fs::remove_dir_all(&dir);
282	+	let (proxy, engine_dir) = (dir.join("proxy"), dir.join("engine"));
283	+	std::fs::create_dir_all(&proxy).and_then(\|()\| std::fs::create_dir_all(&engine_dir)).with_context(\|\| format!("could not make {}", dir.display()))?;
284	+	let socket = proxy.join("docker.sock");
285	+	let listener = UnixListener::bind(&socket).with_context(\|\| format!("could not listen on {}", socket.display()))?;
286	+	{
287	+	use std::os::unix::fs::PermissionsExt;
288	+	// Whoever the job's image runs as.
289	+	std::fs::set_permissions(&socket, std::fs::Permissions::from_mode(0o666))?;
290	+	}
291	+	listener.set_nonblocking(true)?;
292	+	let spec = EngineSpec {
293	+	image: image.to_owned(),
294	+	container: format!("{job}-docker"),
295	+	volume: format!("{job}-docker"),
296	+	job: job.to_owned(),
297	+	label: format!("sh.g1t.runner={runner}"),
298	+	engine_dir,
299	+	};
300	+	let stop = Arc::new(AtomicBool::new(false));
301	+	let (stopping, thread_spec) = (stop.clone(), spec.clone());
302	+	std::thread::spawn(move \|\| {
303	+	let started: Arc<Mutex<Option<Result<(), String>>>> = Arc::new(Mutex::new(None));
304	+	while !stopping.load(Ordering::SeqCst) {
305	+	match listener.accept() {
306	+	Ok((client, _)) => {
307	+	let _ = client.set_nonblocking(false);
308	+	let (spec, started) = (thread_spec.clone(), started.clone());
309	+	std::thread::spawn(move \|\| serve(client, &spec, &started));
310	+	}
311	+	Err(error) if error.kind() == io::ErrorKind::WouldBlock => std::thread::sleep(Duration::from_millis(100)),
312	+	Err(_) => std::thread::sleep(Duration::from_millis(100)),
313	+	}
314	+	}
315	+	});
316	+	Ok(JobEngine { dir, container: spec.container, volume: spec.volume, stop })
317	+	}
318	+
319	+	/// What the job's `docker run` adds: its socket's folder, read-only
320	+	/// (a socket on a read-only mount still connects), the Docker CLI, and
321	+	/// `DOCKER_HOST`.
322	+	pub fn job_args(&self) -> Vec<String> {
323	+	let mut args = vec!["-v".into(), format!("{}:{JOB_SOCKET_DIR}:ro", self.dir.join("proxy").display())];
324	+	if Path::new(DOCKER_CLI).exists() {
325	+	args.extend(["-v".into(), format!("{DOCKER_CLI}:/usr/local/bin/docker:ro")]);
326	+	}
327	+	args.extend(["-e".into(), format!("DOCKER_HOST=unix://{JOB_SOCKET_DIR}/docker.sock")]);
328	+	args
329	+	}
330	+
331	+	/// Stops the engine and removes it, its data and its sockets.
332	+	pub fn remove(&self) {
333	+	self.stop.store(true, Ordering::SeqCst);
334	+	let quiet = \|args: &[&str]\| {
335	+	let _ = docker().args(args).stdout(Stdio::null()).stderr(Stdio::null()).status();
336	+	};
337	+	quiet(&["rm", "-f", "-v", &self.container]);
338	+	quiet(&["volume", "rm", "-f", &self.volume]);
339	+	let _ = std::fs::remove_dir_all(&self.dir);
340	+	}
341	+	}
342	+
343	+	#[cfg(test)]
344	+	mod tests {
345	+	use super::*;
346	+
347	+	#[test]
348	+	fn the_runners_engine_keeps_its_own_socket_and_addresses() {
349	+	let config = daemon_config(false);
350	+	assert_eq!(config["hosts"][0], "unix:///run/g1t-runner/docker.sock");
351	+	assert_ne!(SOCKET, "/var/run/docker.sock");
352	+	assert!(config.get("storage-driver").is_none());
353	+	assert_eq!(daemon_config(true)["storage-driver"], "vfs");
354	+	// Not Docker's default range, which the machine's engine has.
355	+	assert!(!config["bip"].as_str().unwrap().starts_with("172.17."));
356	+	}
357	+
358	+	#[test]
359	+	fn a_jobs_engine_is_in_the_jobs_network_and_mounts_nothing_of_the_machine() {
360	+	let spec = EngineSpec {
361	+	image: DEFAULT_JOB_ENGINE_IMAGE.into(),
362	+	container: "g1t-job_1-docker".into(),
363	+	volume: "g1t-job_1-docker".into(),
364	+	job: "g1t-job_1".into(),
365	+	label: "sh.g1t.runner=rnr_1".into(),
366	+	engine_dir: PathBuf::from("/run/g1t-runner/jobs/g1t-job_1/engine"),
367	+	};
368	+	let args = engine_args(&spec);
369	+	let joined = args.join(" ");
370	+	assert!(joined.contains("--network container:g1t-job_1"));
371	+	assert!(joined.contains("--rm"));
372	+	assert!(!joined.contains("docker.sock:/"), "no engine socket is mounted in: {joined}");
373	+	// Its volumes: the runner's own folder for its socket, and a volume.
374	+	let mounts: Vec<&String> = args.windows(2).filter(\|w\| w[0] == "-v").map(\|w\| &w[1]).collect();
375	+	assert_eq!(mounts.len(), 2);
376	+	assert!(mounts[0].starts_with("/run/g1t-runner/jobs/"));
377	+	assert!(mounts[1].starts_with("g1t-job_1-docker:"));
378	+	assert!(args.iter().any(\|a\| a == "DOCKER_TLS_CERTDIR="));
379	+	}
380	+
381	+	/// Against a real Docker Engine, as root, with `docker:28-dind` at
382	+	/// hand: `G1T_TEST_DOCKER=1 cargo test -- --ignored job_engine`.
383	+	#[test]
384	+	#[ignore]
385	+	fn a_jobs_engine_starts_when_the_job_first_uses_it() {
386	+	if std::env::var("G1T_TEST_DOCKER").as_deref() != Ok("1") {
387	+	return;
388	+	}
389	+	let job = "g1t-engine-test";
390	+	let quiet = \|args: &[&str]\| {
391	+	let _ = docker().args(args).stdout(Stdio::null()).stderr(Stdio::null()).status();
392	+	};
393	+	quiet(&["rm", "-f", job]);
394	+	let engine = JobEngine::prepare(job, "test", DEFAULT_JOB_ENGINE_IMAGE).unwrap();
395	+	// A job's container with the job's Docker: the dind image has a CLI.
396	+	let mut run = docker();
397	+	run.args(["run", "-d", "--rm", "--name", job, "--cap-drop", "ALL", "--security-opt", "no-new-privileges"]);
398	+	let mut job_args = engine.job_args();
399	+	// This machine's CLI may not run in Alpine; the image has its own.
400	+	if let Some(at) = job_args.iter().position(\|a\| a.starts_with(DOCKER_CLI)) {
401	+	job_args.drain(at - 1..=at);
402	+	}
403	+	run.args(&job_args).args(["--entrypoint", "sleep", DEFAULT_JOB_ENGINE_IMAGE, "300"]);
404	+	assert!(run.status().unwrap().success());
405	+	// Nothing runs behind the socket until it is used.
406	+	assert!(!docker().args(["inspect", &engine.container]).stdout(Stdio::null()).stderr(Stdio::null()).status().unwrap().success());
407	+	let version = docker().args(["exec", job, "docker", "version", "--format", "{{.Server.Version}}"]).output().unwrap();
408	+	let shown = String::from_utf8_lossy(&version.stdout).trim().to_owned();
409	+	let errors = String::from_utf8_lossy(&version.stderr).trim().to_owned();
410	+	// The job cannot change its socket's folder.
411	+	let wrote = docker().args(["exec", job, "touch", "/run/g1t-docker/x"]).stderr(Stdio::null()).status().unwrap();
412	+	quiet(&["rm", "-f", job]);
413	+	engine.remove();
414	+	assert!(version.status.success(), "docker version in the job: {errors}");
415	+	assert!(shown.starts_with("28."), "{shown}");
416	+	assert!(!wrote.success());
417	+	assert!(!engine.dir.exists());
418	+	}
419	+	}

3	3		//!
4	4		//! - In Docker (the default): one container per job, removed when it
5	5		//! ends, from the job's `container:` image or the runner's `--image`. A
6		−	//! Linux build of the harness is mounted into it read-only. Its
7		−	//! credentials are passed by name (`-e NAME`), never on the command line.
8		−	//! - Directly (`--no-docker`): the harness runs as a child process, in a
9		−	//! fresh folder under the runner's work folder that is removed after.
	6	+	//! Linux build of the harness is mounted into it read-only; that and the
	7	+	//! job's own work volume, removed after, are its only mounts. It runs
	8	+	//! with every capability dropped but the few a build needs, no new
	9	+	//! privileges, a process, memory and CPU limit, and a network of its
	10	+	//! own. With nested-container isolation (sandbox.rs), all of this is on
	11	+	//! the runner's own Docker Engine, inside the runner's container, and a
	12	+	//! job that uses Docker gets an engine of its own (engine.rs).
	13	+	//! - Directly (`--no-docker`, deprecated): the harness runs as a child
	14	+	//! process, in a fresh folder under the runner's work folder that is
	15	+	//! removed after. Unsandboxed: the job's log says so, and a public
	16	+	//! repository's work is refused.
10	17		//!
11	18		//! The runner's own credential never reaches either: a job only ever has
12		−	//! its own short-lived token, as it would in g1t's sandbox.
	19	+	//! its own short-lived token, as it would in g1t's sandbox. Credentials are
	20	+	//! passed by name (`-e NAME`), never on the command line.
13	21
14	22		use std::collections::BTreeMap;
15	23		use std::path::{Path, PathBuf};
⋯
20	28
21	29		use super::api::Assignment;
22	30		use super::config::Config;
	31	+	use super::sandbox::{self, Isolation};
23	32		use super::{log, update};
24	33
25	34		/// The image workflow jobs run in when they name none: Debian with Node,
26	35		/// git and Python, as g1t's sandbox has.
27	36		pub const DEFAULT_IMAGE: &str = "node:24-bookworm";
	37	+	/// The most processes in a job's container, unless the runner says.
	38	+	pub const DEFAULT_PIDS_LIMIT: u32 = 4096;
	39	+
	40	+	/// The capabilities a job's container keeps; Docker's others, and
	41	+	/// everything else, are dropped. A workflow job runs as its image's user,
	42	+	/// root for most CI images, and these are what installing packages and
	43	+	/// building need:
	44	+	///
	45	+	/// - `CHOWN`, `FOWNER`, `FSETID`, `DAC_OVERRIDE`: a package manager (apt,
	46	+	/// npm as root) writing files it then hands to another owner, keeping
	47	+	/// setgid bits, and root writing files it does not own.
	48	+	/// - `SETUID`, `SETGID`: apt and others drop to their own user (`_apt`) to
	49	+	/// download; with no new privileges, nothing can gain a user back.
	50	+	/// - `KILL`: a step stopping processes another user started.
	51	+	/// - `NET_BIND_SERVICE`: a test server on port 80 or 443.
	52	+	///
	53	+	/// Not kept, among others: `NET_RAW` (raw sockets, `ping`), `MKNOD`,
	54	+	/// `SYS_CHROOT`, `SETFCAP`, `AUDIT_WRITE`, and every `SYS_*` and `NET_ADMIN`.
	55	+	pub const JOB_CAPS: &[&str] = &["CHOWN", "DAC_OVERRIDE", "FOWNER", "FSETID", "SETUID", "SETGID", "KILL", "NET_BIND_SERVICE"];
28	56
	57	+	/// Agent work's image runs as `node` and uses `sudo` once, to put its
	58	+	/// guardrails' policy where the agent cannot change it, and then removes
	59	+	/// its own sudo (guard.rs). sudo needs a setuid program to work and
	60	+	/// `AUDIT_WRITE` to log, so agent work keeps those and has no
	61	+	/// `no-new-privileges`; it still has every other limit.
	62	+	pub const AGENT_EXTRA_CAPS: &[&str] = &["AUDIT_WRITE"];
	63	+
29	64		/// Work in progress.
30	65		pub struct Running {
31	66		pub id: String,
⋯
33	68		pub name: String,
34	69		child: Child,
35	70		container: Option<String>,
	71	+	/// The job's network and work volume, removed after.
	72	+	network: Option<String>,
	73	+	volume: Option<String>,
	74	+	/// The job's own Docker Engine, when it may have one.
	75	+	engine: Option<JobEngine>,
36	76		folder: Option<PathBuf>,
37	77		/// Past its time limit and a grace for g1t to say so: stopped here.
38	78		deadline: Instant,
39	79		}
40	80
	81	+	#[cfg(unix)]
	82	+	use super::engine::JobEngine;
	83	+
	84	+	/// No job has an engine of its own off Unix.
	85	+	#[cfg(not(unix))]
	86	+	pub struct JobEngine;
	87	+
	88	+	#[cfg(not(unix))]
	89	+	impl JobEngine {
	90	+	fn remove(&self) {}
	91	+	fn job_args(&self) -> Vec<String> {
	92	+	Vec::new()
	93	+	}
	94	+	}
	95	+
	96	+	/// `docker`, at the runner's own engine when it has one.
	97	+	fn docker() -> Command {
	98	+	#[cfg(unix)]
	99	+	return super::engine::docker();
	100	+	#[cfg(not(unix))]
	101	+	Command::new("docker")
	102	+	}
	103	+
	104	+	fn quietly(command: &mut Command) {
	105	+	let _ = command.stdout(Stdio::null()).stderr(Stdio::null()).status();
	106	+	}
	107	+
41	108		impl Running {
42	109		pub fn overdue(&self) -> bool {
43	110		Instant::now() > self.deadline
⋯
55	122		/// Stops it: the container, then the process and everything it started.
56	123		pub fn stop(&mut self) {
57	124		if let Some(container) = &self.container {
58		−	let _ = Command::new("docker").args(["kill", container]).stdout(Stdio::null()).stderr(Stdio::null()).status();
	125	+	quietly(docker().args(["kill", container]));
	126	+	}
	127	+	if let Some(engine) = &self.engine {
	128	+	quietly(docker().args(["kill", &engine.container]));
59	129		}
60	130		let pid = self.child.id().to_string();
61	131		if cfg!(windows) {
62		−	let _ = Command::new("taskkill").args(["/T", "/F", "/PID", &pid]).stdout(Stdio::null()).stderr(Stdio::null()).status();
	132	+	quietly(Command::new("taskkill").args(["/T", "/F", "/PID", &pid]));
63	133		} else {
64	134		// Its own process group (see `directly`): the steps' processes too.
65		−	let _ = Command::new("kill").args(["-TERM", &format!("-{pid}")]).stdout(Stdio::null()).stderr(Stdio::null()).status();
	135	+	quietly(Command::new("kill").args(["-TERM", &format!("-{pid}")]));
66	136		}
67	137		let _ = self.child.kill();
68	138		let _ = self.child.wait();
69	139		}
70	140
71		−	/// Removes what it left on the machine.
	141	+	/// Removes what it left: its container, its engine and that engine's
	142	+	/// data, its network, its work volume, its folder.
72	143		pub fn clean(&mut self) {
73	144		if let Some(container) = &self.container {
74		−	let _ = Command::new("docker").args(["rm", "-f", container]).stdout(Stdio::null()).stderr(Stdio::null()).status();
	145	+	quietly(docker().args(["rm", "-f", "-v", container]));
	146	+	}
	147	+	if let Some(engine) = self.engine.take() {
	148	+	engine.remove();
75	149		}
	150	+	if let Some(network) = &self.network {
	151	+	quietly(docker().args(["network", "rm", network]));
	152	+	}
	153	+	if let Some(volume) = &self.volume {
	154	+	quietly(docker().args(["volume", "rm", "-f", volume]));
	155	+	}
76	156		if let Some(folder) = &self.folder {
77	157		let _ = std::fs::remove_dir_all(folder);
78	158		}
⋯
119	199
120	200		/// Starts work. The harness reports a workflow job's steps and log itself;
121	201		/// the runner tells g1t how the process ended.
122		−	pub fn start(config: &Config, folder: &Path, work: &Assignment) -> Result<Running> {
	202	+	pub fn start(config: &Config, isolation: Isolation, folder: &Path, work: &Assignment) -> Result<Running> {
123	203		let env = harness_env(config, work)?;
124		−	if config.docker { in_docker(config, folder, work, env) } else { directly(config, work, env) }
	204	+	if isolation == Isolation::Host {
	205	+	refuse_public_on_host(work)?;
	206	+	return directly(config, work, env);
	207	+	}
	208	+	in_docker(config, isolation, folder, work, env)
	209	+	}
	210	+
	211	+	/// Host mode does not take a public repository's work: anyone can propose
	212	+	/// code to one, and here it would run on the machine itself.
	213	+	fn refuse_public_on_host(work: &Assignment) -> Result<()> {
	214	+	match work.visibility.as_deref() {
	215	+	Some("public") => bail!("{}", sandbox::host_mode_refuses(&work.repo)),
	216	+	Some(_) => Ok(()),
	217	+	None => {
	218	+	log(&format!("g1t did not say whether {} is public (an older g1t), so it runs; host mode refuses public repositories' work once it does.", work.repo));
	219	+	Ok(())
	220	+	}
	221	+	}
	222	+	}
	223	+
	224	+	/// Each job container's limits.
	225	+	#[derive(Debug, Clone, PartialEq)]
	226	+	pub struct Limits {
	227	+	pub memory: String,
	228	+	pub cpus: String,
	229	+	pub pids: u32,
	230	+	}
	231	+
	232	+	/// Three quarters of `MemTotal` in `/proc/meminfo`, in MiB, which under
	233	+	/// Sysbox is the runner container's own memory.
	234	+	fn default_memory(meminfo: &str) -> Option<String> {
	235	+	let kb: u64 = meminfo.lines().find_map(\|l\| l.strip_prefix("MemTotal:"))?.trim().trim_end_matches("kB").trim().parse().ok()?;
	236	+	let mib = kb / 1024 * 3 / 4;
	237	+	(mib >= 256).then(\|\| format!("{mib}m"))
	238	+	}
	239	+
	240	+	pub fn limits(config: &Config) -> Limits {
	241	+	let memory = config
	242	+	.memory
	243	+	.clone()
	244	+	.or_else(\|\| default_memory(&std::fs::read_to_string("/proc/meminfo").unwrap_or_default()))
	245	+	.unwrap_or_else(\|\| "4g".into());
	246	+	let cpus = config.cpus.clone().unwrap_or_else(\|\| std::thread::available_parallelism().map(\|n\| n.get()).unwrap_or(2).to_string());
	247	+	Limits { memory, cpus, pids: config.pids_limit.unwrap_or(DEFAULT_PIDS_LIMIT) }
	248	+	}
	249	+
	250	+	/// Everything a job's `docker run` is made from.
	251	+	pub struct Plan<'a> {
	252	+	pub kind: &'a str,
	253	+	pub name: &'a str,
	254	+	pub runner: &'a str,
	255	+	pub image: &'a str,
	256	+	pub harness: &'a Path,
	257	+	pub network: &'a str,
	258	+	/// The job's work volume, for a workflow job's `/home/runner`.
	259	+	pub volume: Option<&'a str>,
	260	+	pub limits: &'a Limits,
	261	+	/// The names of the harness's variables, whose values come from the
	262	+	/// runner's process, never its command line.
	263	+	pub env_names: Vec<&'a str>,
	264	+	/// What the job's own engine adds (`JobEngine::job_args`).
	265	+	pub engine_args: Vec<String>,
	266	+	}
	267	+
	268	+	/// The arguments of `docker` that start a job's container.
	269	+	pub fn run_args(plan: &Plan) -> Vec<String> {
	270	+	let mut args: Vec<String> = Vec::new();
	271	+	fn add(args: &mut Vec<String>, items: &[&str]) {
	272	+	args.extend(items.iter().map(\|s\| s.to_string()));
	273	+	}
	274	+	add(&mut args, &["run", "--rm", "--name", plan.name, "--label", &format!("sh.g1t.runner={}", plan.runner)]);
	275	+	add(&mut args, &["--pull", "missing", "--init"]);
	276	+	// The walls.
	277	+	add(&mut args, &["--cap-drop", "ALL"]);
	278	+	let mut caps: Vec<&str> = JOB_CAPS.to_vec();
	279	+	if plan.kind == "agent" {
	280	+	caps.extend(AGENT_EXTRA_CAPS);
	281	+	} else {
	282	+	add(&mut args, &["--security-opt", "no-new-privileges"]);
	283	+	}
	284	+	for cap in caps {
	285	+	add(&mut args, &["--cap-add", cap]);
	286	+	}
	287	+	let pids = plan.limits.pids.to_string();
	288	+	add(&mut args, &["--pids-limit", &pids, "--memory", &plan.limits.memory, "--memory-swap", &plan.limits.memory, "--cpus", &plan.limits.cpus]);
	289	+	add(&mut args, &["--network", plan.network, "--ipc", "private"]);
	290	+	// The job is in its `container:` image already: the harness inside
	291	+	// does not start it again (actions/containers.rs).
	292	+	add(&mut args, &["-e", "G1T_JOB_CONTAINER=1"]);
	293	+	for name in &plan.env_names {
	294	+	add(&mut args, &["-e", name]);
	295	+	}
	296	+	// Its only mounts: the harness, read-only, and its own volume.
	297	+	add(&mut args, &["-v", &format!("{}:/opt/g1t/g1t-runner:ro", plan.harness.display())]);
	298	+	if let Some(volume) = plan.volume {
	299	+	add(&mut args, &["-v", &format!("{volume}:/home/runner")]);
	300	+	}
	301	+	args.extend(plan.engine_args.iter().cloned());
	302	+	add(&mut args, &["--entrypoint", "/opt/g1t/g1t-runner", plan.image]);
	303	+	args
125	304		}
126	305
127		−	fn in_docker(config: &Config, folder: &Path, work: &Assignment, env: BTreeMap<String, String>) -> Result<Running> {
	306	+	fn in_docker(config: &Config, isolation: Isolation, folder: &Path, work: &Assignment, env: BTreeMap<String, String>) -> Result<Running> {
128	307		let image = match work.kind.as_str() {
129	308		"agent" => match config.agent_image.clone().or_else(\|\| update::agent_image(config)) {
130	309		Some(image) => image,
131	310		None => bail!(
132		−	"agent work needs an image with git, Node and the agent's CLI: register this runner with --agent-image, or run it with --no-docker on Linux"
	311	+	"agent work needs an image with git, Node and the agent's CLI: register this runner with --agent-image IMAGE"
133	312		),
134	313		},
135	314		_ => work.image.clone().or_else(\|\| config.image.clone()).unwrap_or_else(\|\| DEFAULT_IMAGE.to_owned()),
136	315		};
	316	+	#[cfg(unix)]
	317	+	if super::engine::in_use() {
	318	+	super::engine::ensure_running()?;
	319	+	}
137	320		let harness = update::linux_harness(config, folder)?;
138	321		let name = container_name(&work.id);
139		−	let _ = Command::new("docker").args(["rm", "-f", &name]).stdout(Stdio::null()).stderr(Stdio::null()).status();
140		−	let mut command = Command::new("docker");
141		−	command.args(["run", "--rm", "--name", &name, "--label", &format!("sh.g1t.runner={}", config.runner)]);
142		−	command.args(["--pull", "missing", "--init"]);
143		−	// The job is in its `container:` image already: the harness inside
144		−	// does not start it again (actions/containers.rs).
145		−	command.args(["-e", "G1T_JOB_CONTAINER=1"]);
146		−	for name in env.keys() {
147		−	command.args(["-e", name]);
	322	+	// What an earlier run of this job may have left.
	323	+	quietly(docker().args(["rm", "-f", "-v", &name]));
	324	+	quietly(docker().args(["network", "rm", &name]));
	325	+	let output = docker()
	326	+	.args(["network", "create", "--label", &format!("sh.g1t.runner={}", config.runner), &name])
	327	+	.output()
	328	+	.context("could not run docker: is Docker installed and running?")?;
	329	+	if !output.status.success() {
	330	+	bail!("could not make the job's network: {}", String::from_utf8_lossy(&output.stderr).trim());
148	331		}
	332	+	let volume = (work.kind == "workflow").then(\|\| format!("{name}-work"));
	333	+	if let Some(volume) = &volume {
	334	+	quietly(docker().args(["volume", "rm", "-f", volume]));
	335	+	quietly(docker().args(["volume", "create", "--label", &format!("sh.g1t.runner={}", config.runner), volume]));
	336	+	}
	337	+	// A job's own Docker, only inside the runner's own sandbox: on a
	338	+	// shared engine it would be the machine's.
	339	+	let engine: Option<JobEngine> = job_engine(config, isolation, &name);
	340	+	let limits = limits(config);
	341	+	let plan = Plan {
	342	+	kind: &work.kind,
	343	+	name: &name,
	344	+	runner: &config.runner,
	345	+	image: &image,
	346	+	harness: &harness,
	347	+	network: &name,
	348	+	volume: volume.as_deref(),
	349	+	limits: &limits,
	350	+	env_names: env.keys().map(String::as_str).collect(),
	351	+	engine_args: engine.as_ref().map(JobEngine::job_args).unwrap_or_default(),
	352	+	};
	353	+	let mut command = docker();
	354	+	command.args(run_args(&plan));
149	355		command.envs(&env);
150		−	command.args(["-v", &format!("{}:/opt/g1t/g1t-runner:ro", harness.display())]);
151		−	command.args(["--entrypoint", "/opt/g1t/g1t-runner", &image]);
152		−	log(&format!("Running {} {} in {image}", work.kind, work.name));
153		−	let child = command.spawn().context("could not run docker: is Docker installed and running? (or register with --no-docker)")?;
154		−	Ok(Running { id: work.id.clone(), kind: work.kind.clone(), name: work.name.clone(), child, container: Some(name), folder: None, deadline: deadline(work) })
	356	+	log(&format!(
	357	+	"Running {} {} in {image}: its own container and network, capabilities dropped, {} memory, {} CPUs, {} processes{}",
	358	+	work.kind,
	359	+	work.name,
	360	+	limits.memory,
	361	+	limits.cpus,
	362	+	limits.pids,
	363	+	if engine.is_some() { ", Docker of its own when it uses it" } else { "" }
	364	+	));
	365	+	let child = match command.spawn() {
	366	+	Ok(child) => child,
	367	+	Err(error) => {
	368	+	if let Some(engine) = &engine {
	369	+	engine.remove();
	370	+	}
	371	+	quietly(docker().args(["network", "rm", &name]));
	372	+	return Err(error).context("could not run docker: is Docker installed and running?");
	373	+	}
	374	+	};
	375	+	Ok(Running {
	376	+	id: work.id.clone(),
	377	+	kind: work.kind.clone(),
	378	+	name: work.name.clone(),
	379	+	child,
	380	+	container: Some(name.clone()),
	381	+	network: Some(name),
	382	+	volume,
	383	+	engine,
	384	+	folder: None,
	385	+	deadline: deadline(work),
	386	+	})
	387	+	}
	388	+
	389	+	#[cfg(unix)]
	390	+	fn job_engine(config: &Config, isolation: Isolation, name: &str) -> Option<JobEngine> {
	391	+	if isolation != Isolation::NestedContainer \|\| !config.job_docker {
	392	+	return None;
	393	+	}
	394	+	let image = config.job_docker_image.as_deref().unwrap_or(super::engine::DEFAULT_JOB_ENGINE_IMAGE);
	395	+	match JobEngine::prepare(name, &config.runner, image) {
	396	+	Ok(engine) => Some(engine),
	397	+	Err(error) => {
	398	+	log(&format!("This job has no Docker of its own: {error:#}"));
	399	+	None
	400	+	}
	401	+	}
155	402		}
156	403
	404	+	#[cfg(not(unix))]
	405	+	fn job_engine(_config: &Config, _isolation: Isolation, _name: &str) -> Option<JobEngine> {
	406	+	None
	407	+	}
	408	+
157	409		fn directly(config: &Config, work: &Assignment, env: BTreeMap<String, String>) -> Result<Running> {
158	410		let me = std::env::current_exe().context("could not find this program to run the job")?;
159	411		let mut command = Command::new(me);
⋯
177	429		}
178	430		}
179	431		command.envs(&env);
	432	+	// The job's log says it ran unsandboxed (actions/mod.rs).
	433	+	command.env(JOB_NOTICE, sandbox::HOST_MODE_IN_JOB);
180	434		// A group of its own, so stopping it stops what its steps started.
181	435		#[cfg(unix)]
182	436		std::os::unix::process::CommandExt::process_group(&mut command, 0);
183		−	log(&format!("Running {} {} on this machine", work.kind, work.name));
	437	+	log(&format!("Running {} {} on this machine, unsandboxed (host mode, deprecated)", work.kind, work.name));
184	438		let child = command.spawn().context("could not start the job's process")?;
185		−	Ok(Running { id: work.id.clone(), kind: work.kind.clone(), name: work.name.clone(), child, container: None, folder, deadline: deadline(work) })
	439	+	Ok(Running {
	440	+	id: work.id.clone(),
	441	+	kind: work.kind.clone(),
	442	+	name: work.name.clone(),
	443	+	child,
	444	+	container: None,
	445	+	network: None,
	446	+	volume: None,
	447	+	engine: None,
	448	+	folder,
	449	+	deadline: deadline(work),
	450	+	})
186	451		}
187	452
	453	+	/// A warning the harness puts at the top of the job's log.
	454	+	pub const JOB_NOTICE: &str = "G1T_JOB_NOTICE";
	455	+
188	456		fn deadline(work: &Assignment) -> Instant {
189	457		Instant::now() + Duration::from_secs(u64::from(work.timeout_minutes.max(1)) * 60 + 10 * 60)
190	458		}
191	459
192	460		/// Whether Docker answers.
193	461		pub fn docker_ready() -> bool {
194		−	Command::new("docker")
	462	+	docker()
195	463		.args(["version", "--format", "{{.Server.Version}}"])
196	464		.stdout(Stdio::null())
197	465		.stderr(Stdio::null())
⋯
199	467		.is_ok_and(\|status\| status.success())
200	468		}
201	469
	470	+	/// Removes what jobs of this runner's left behind, if it stopped while one ran.
	471	+	pub fn sweep(config: &Config) {
	472	+	let label = format!("label=sh.g1t.runner={}", config.runner);
	473	+	for (list, remove) in [
	474	+	(vec!["ps", "-aq", "--filter", &label], vec!["rm", "-f", "-v"]),
	475	+	(vec!["network", "ls", "-q", "--filter", &label], vec!["network", "rm"]),
	476	+	(vec!["volume", "ls", "-q", "--filter", &label], vec!["volume", "rm", "-f"]),
	477	+	] {
	478	+	let Ok(output) = docker().args(&list).output() else { return };
	479	+	let ids: Vec<String> = String::from_utf8_lossy(&output.stdout).split_whitespace().map(str::to_owned).collect();
	480	+	if !ids.is_empty() {
	481	+	quietly(docker().args(&remove).args(&ids));
	482	+	}
	483	+	}
	484	+	}
	485	+
202	486		#[cfg(test)]
203	487		mod tests {
204	488		use super::*;
⋯
222	506		agent_image: None,
223	507		harness: None,
224	508		auto_update: false,
	509	+	memory: None,
	510	+	cpus: None,
	511	+	pids_limit: None,
	512	+	job_docker: true,
	513	+	job_docker_image: None,
225	514		}
226	515		}
227	516
⋯
262	551		assert_eq!(container_name("job_01ab"), "g1t-job_01ab");
263	552		assert_eq!(container_name("a/b c"), "g1t-a-b-c");
264	553		}
	554	+
	555	+	fn args_for(kind: &str, engine_args: Vec<String>) -> (Vec<String>, BTreeMap<String, String>) {
	556	+	let config = config();
	557	+	let env = harness_env(&config, &assignment(kind)).unwrap();
	558	+	let limits = Limits { memory: "6144m".into(), cpus: "4".into(), pids: 4096 };
	559	+	let harness = PathBuf::from("/usr/local/bin/g1t-runner");
	560	+	let plan = Plan {
	561	+	kind,
	562	+	name: "g1t-job_1",
	563	+	runner: "rnr_1",
	564	+	image: DEFAULT_IMAGE,
	565	+	harness: &harness,
	566	+	network: "g1t-job_1",
	567	+	volume: (kind == "workflow").then_some("g1t-job_1-work"),
	568	+	limits: &limits,
	569	+	env_names: env.keys().map(String::as_str).collect(),
	570	+	engine_args,
	571	+	};
	572	+	(run_args(&plan), env.clone())
	573	+	}
	574	+
	575	+	fn value_of<'a>(args: &'a [String], flag: &str) -> Vec<&'a str> {
	576	+	args.windows(2).filter(\|w\| w[0] == flag).map(\|w\| w[1].as_str()).collect()
	577	+	}
	578	+
	579	+	#[test]
	580	+	fn a_jobs_container_is_walled_in() {
	581	+	let (args, _) = args_for("workflow", Vec::new());
	582	+	assert_eq!(args[0], "run");
	583	+	assert!(args.contains(&"--rm".to_owned()));
	584	+	assert_eq!(value_of(&args, "--cap-drop"), vec!["ALL"]);
	585	+	assert_eq!(value_of(&args, "--cap-add"), JOB_CAPS.to_vec());
	586	+	assert_eq!(value_of(&args, "--security-opt"), vec!["no-new-privileges"]);
	587	+	assert_eq!(value_of(&args, "--pids-limit"), vec!["4096"]);
	588	+	assert_eq!(value_of(&args, "--memory"), vec!["6144m"]);
	589	+	assert_eq!(value_of(&args, "--memory-swap"), vec!["6144m"]);
	590	+	assert_eq!(value_of(&args, "--cpus"), vec!["4"]);
	591	+	assert_eq!(value_of(&args, "--network"), vec!["g1t-job_1"]);
	592	+	for forbidden in ["--privileged", "--pid", "--userns", "--device", "host"] {
	593	+	assert!(!args.iter().any(\|a\| a == forbidden), "{forbidden} in {args:?}");
	594	+	}
	595	+	assert!(!JOB_CAPS.iter().any(\|c\| c.starts_with("SYS_") \|\| c == "NET_ADMIN" \|\| c == "NET_RAW"));
	596	+	}
	597	+
	598	+	#[test]
	599	+	fn a_jobs_only_mounts_are_the_harness_and_its_own_volume() {
	600	+	let (args, _) = args_for("workflow", Vec::new());
	601	+	let mounts = value_of(&args, "-v");
	602	+	assert_eq!(mounts, vec!["/usr/local/bin/g1t-runner:/opt/g1t/g1t-runner:ro", "g1t-job_1-work:/home/runner"]);
	603	+	assert!(!args.iter().any(\|a\| a.contains("docker.sock")), "no engine socket: {args:?}");
	604	+	assert!(!args.iter().any(\|a\| a == "--mount"));
	605	+	}
	606	+
	607	+	#[test]
	608	+	fn credentials_go_by_name_and_the_runners_never() {
	609	+	let (args, env) = args_for("workflow", Vec::new());
	610	+	let passed = value_of(&args, "-e");
	611	+	assert!(passed.contains(&"ACTIONS_TOKEN"));
	612	+	// Never a value on the command line, and never the runner's credential.
	613	+	assert!(!args.iter().any(\|a\| a.contains("jobtoken")));
	614	+	assert!(!args.iter().any(\|a\| a.contains("g1tr_")));
	615	+	assert!(env.values().all(\|v\| !v.contains("g1tr_secret")));
	616	+	assert!(!env.contains_key("G1T_RUNNER_TOKEN"));
	617	+	}
	618	+
	619	+	#[test]
	620	+	fn agent_work_keeps_sudo_for_its_guardrails_and_every_other_wall() {
	621	+	let (args, _) = args_for("agent", Vec::new());
	622	+	assert!(value_of(&args, "--security-opt").is_empty());
	623	+	assert!(value_of(&args, "--cap-add").contains(&"AUDIT_WRITE"));
	624	+	assert_eq!(value_of(&args, "--cap-drop"), vec!["ALL"]);
	625	+	assert_eq!(value_of(&args, "-v"), vec!["/usr/local/bin/g1t-runner:/opt/g1t/g1t-runner:ro"]);
	626	+	}
	627	+
	628	+	#[test]
	629	+	fn a_job_with_docker_gets_a_socket_of_its_own_not_the_runners() {
	630	+	let engine_args = vec!["-v".into(), "/run/g1t-runner/jobs/g1t-job_1/proxy:/run/g1t-docker:ro".into(), "-e".into(), "DOCKER_HOST=unix:///run/g1t-docker/docker.sock".into()];
	631	+	let (args, _) = args_for("workflow", engine_args);
	632	+	let mounts = value_of(&args, "-v");
	633	+	assert!(mounts.contains(&"/run/g1t-runner/jobs/g1t-job_1/proxy:/run/g1t-docker:ro"));
	634	+	assert!(!mounts.iter().any(\|m\| m.starts_with("/var/run/docker.sock") \|\| m.starts_with("/run/g1t-runner/docker.sock")));
	635	+	// The entrypoint and image stay last.
	636	+	assert_eq!(&args[args.len() - 3..], ["--entrypoint", "/opt/g1t/g1t-runner", DEFAULT_IMAGE]);
	637	+	}
	638	+
	639	+	#[test]
	640	+	fn limits_default_sensibly() {
	641	+	assert_eq!(default_memory("MemTotal: 16384000 kB\nMemFree: 1 kB\n"), Some("12000m".into()));
	642	+	assert_eq!(default_memory(""), None);
	643	+	let mut config = config();
	644	+	config.memory = Some("2g".into());
	645	+	config.pids_limit = Some(100);
	646	+	let limits = limits(&config);
	647	+	assert_eq!(limits.memory, "2g");
	648	+	assert_eq!(limits.pids, 100);
	649	+	assert!(limits.cpus.parse::<f64>().unwrap() >= 1.0);
	650	+	}
	651	+
	652	+	#[test]
	653	+	fn host_mode_refuses_a_public_repositorys_work() {
	654	+	let mut work = assignment("workflow");
	655	+	work.visibility = Some("public".into());
	656	+	let refused = refuse_public_on_host(&work).unwrap_err().to_string();
	657	+	assert!(refused.contains("acme/web is public"), "{refused}");
	658	+	work.visibility = Some("private".into());
	659	+	assert!(refuse_public_on_host(&work).is_ok());
	660	+	work.visibility = None;
	661	+	assert!(refuse_public_on_host(&work).is_ok());
	662	+	// And refuses it before anything runs.
	663	+	work.visibility = Some("public".into());
	664	+	assert!(start(&config(), Isolation::Host, Path::new("/tmp"), &work).is_err());
	665	+	}
265	666		}

4	4		//! g1t-runner register --url https://g1t.sh --token g1trt_… [--name build-01]
5	5		//! [--labels gpu,cuda] [--group Default] [--ephemeral] [--work-dir DIR]
6	6		//! [--no-docker] [--image IMAGE] [--agent-image IMAGE] [--replace]
	7	+	//! [--memory 8g] [--cpus 4] [--pids-limit 4096] [--no-job-docker]
7	8		//! g1t-runner run [--once]
8	9		//! g1t-runner service install\|start\|stop\|status\|uninstall
9	10		//! g1t-runner remove
⋯
17	18		//! program in another mode, see `exec`), and says how it ended. Nothing
18	19		//! listens on the machine. Every command takes `--dir` (default
19	20		//! `~/.g1t-runner`, or `G1T_RUNNER_DIR`), where its configuration is kept.
	21	+	//!
	22	+	//! How it keeps jobs from the machine is `sandbox`: in its image under
	23	+	//! Sysbox (`G1T_RUNNER_SANDBOX=nested`) it runs its own Docker Engine
	24	+	//! (`engine`) and refuses to start if it can reach the machine's.
20	25
21	26		mod api;
22	27		mod config;
	28	+	#[cfg(unix)]
	29	+	mod engine;
23	30		mod exec;
	31	+	mod sandbox;
24	32		mod service;
25	33		mod update;
26	34
⋯
119	127		/// `G1T_RUNNER_EPHEMERAL=1`), so a Kubernetes secret can hold the token.
120	128		fn with_environment(mut self) -> Options {
121	129		let env_name = \|name: &str\| format!("G1T_RUNNER_{}", name.to_ascii_uppercase().replace('-', "_"));
122		−	for name in ["url", "api", "token", "name", "labels", "group", "work-dir", "image", "agent-image", "harness"] {
	130	+	for name in VALUE_OPTIONS {
123	131		if self.value(name).is_none()
124	132		&& let Ok(value) = std::env::var(env_name(name))
125	133		&& !value.is_empty()
⋯
127	135		self.values.push((name.to_owned(), value));
128	136		}
129	137		}
130		−	for name in ["ephemeral", "no-docker", "replace", "no-auto-update"] {
	138	+	for name in ["ephemeral", "no-docker", "replace", "no-auto-update", "no-job-docker"] {
131	139		if !self.flag(name) && std::env::var(env_name(name)).is_ok_and(\|v\| matches!(v.as_str(), "1" \| "true" \| "yes")) {
132	140		self.flags.push(name.to_owned());
133	141		}
⋯
136	144		}
137	145		}
138	146
	147	+	/// The options that take a value.
	148	+	const VALUE_OPTIONS: [&str; 14] =
	149	+	["url", "api", "token", "name", "labels", "group", "work-dir", "image", "agent-image", "harness", "memory", "cpus", "pids-limit", "job-docker-image"];
	150	+
139	151		const HELP: &str = "g1t-runner: a self-hosted runner for g1t.
140	152
141	153		g1t-runner register --url https://g1t.sh --token <registration token>
142	154		[--name NAME] [--labels a,b] [--group GROUP] [--ephemeral]
143	155		[--work-dir DIR] [--no-docker] [--image IMAGE] [--agent-image IMAGE]
144	156		[--api URL] [--harness PATH] [--replace] [--no-auto-update]
	157	+	[--memory 8g] [--cpus 4] [--pids-limit 4096] [--no-job-docker]
	158	+	[--job-docker-image IMAGE]
145	159		g1t-runner run [--once] poll for work and run it
146	160		g1t-runner register-and-run ... both, for containers; options may be
147	161		G1T_RUNNER_URL, _TOKEN, _LABELS, _EPHEMERAL…
⋯
155	169		g1t-runner detect [--root-dir DIR] what that build would find
156	170
157	171		Every command takes --dir DIR (default ~/.g1t-runner, or G1T_RUNNER_DIR).
	172	+	Run it as a container under Sysbox so neither it nor its jobs reach the
	173	+	machine: docker run --runtime=sysbox-runc … g1t.sh/flagon-io/g1t-runner.
	174	+	--no-docker (host mode) is deprecated.
158	175		Make a registration token under Settings, Runners on g1t.sh.
159	176		Docs: https://docs.g1t.sh/guides/self-hosted-runners/";
160	177
161	178		/// Runs a command. Returns the process's exit code.
162	179		pub fn main(args: Vec<String>) -> i32 {
163		−	let takes = ["url", "api", "token", "name", "labels", "group", "work-dir", "image", "agent-image", "harness", "dir"];
	180	+	let mut takes = VALUE_OPTIONS.to_vec();
	181	+	takes.push("dir");
164	182		let options = match Options::parse(&args[1..], &takes) {
165	183		Ok(options) => options,
166	184		Err(error) => {
⋯
251	269		.filter(\|l\| !l.is_empty())
252	270		.collect();
253	271		let docker = !options.flag("no-docker");
254		−	if docker && !exec::docker_ready() {
255		−	log("Docker does not answer here. Jobs will fail until it does; register with --no-docker to run them on this machine instead.");
	272	+	let isolation = isolation(docker)?;
	273	+	if isolation == sandbox::Isolation::Host {
	274	+	log(sandbox::HOST_MODE_AT_START);
	275	+	} else if !sandbox::nested_requested() && !exec::docker_ready() {
	276	+	log("Docker does not answer here. Jobs will fail until it does.");
256	277		}
	278	+	let pids_limit = match options.value("pids-limit") {
	279	+	Some(text) => Some(text.parse::<u32>().ok().filter(\|n\| *n > 0).with_context(\|\| format!("--pids-limit takes a number of processes, not {text}"))?),
	280	+	None => None,
	281	+	};
257	282		let existing = config::load(folder).ok();
258	283		if existing.is_some() && !options.flag("replace") {
259	284		bail!("{} already holds a registered runner: run `g1t-runner remove` first, or register with --replace or another --dir", folder.display());
⋯
279	304		"ephemeral": options.flag("ephemeral"),
280	305		"group": options.value("group"),
281	306		"replace": options.flag("replace"),
	307	+	"isolation": isolation.as_str(),
282	308		}),
283	309		)?;
284	310		let work_dir = options.value("work-dir").map(PathBuf::from).unwrap_or_else(\|\| folder.join("work"));
⋯
299	325		agent_image: options.value("agent-image").map(str::to_owned),
300	326		harness: options.value("harness").map(PathBuf::from),
301	327		auto_update: !options.flag("no-auto-update"),
	328	+	memory: options.value("memory").map(str::to_owned),
	329	+	cpus: options.value("cpus").map(str::to_owned),
	330	+	pids_limit,
	331	+	job_docker: !options.flag("no-job-docker"),
	332	+	job_docker_image: options.value("job-docker-image").map(str::to_owned),
302	333		};
303	334		config::save(folder, &config)?;
304	335		let place = match &config.repo {
⋯
310	341		Ok(())
311	342		}
312	343
	344	+	/// How this runner keeps work from its machine; refuses a nested runner
	345	+	/// that can reach the machine (sandbox.rs).
	346	+	fn isolation(docker: bool) -> Result<sandbox::Isolation> {
	347	+	use sandbox::{Isolation, SandboxDriver};
	348	+	if !docker {
	349	+	// What is coming for jobs native to this OS.
	350	+	if let Some(driver) = sandbox::native_driver()
	351	+	&& let Err(why) = driver.preflight()
	352	+	{
	353	+	log(&format!("Sandboxed {} jobs, in {}: {why}.", std::env::consts::OS, driver.name()));
	354	+	}
	355	+	return Ok(Isolation::Host);
	356	+	}
	357	+	if !sandbox::nested_requested() {
	358	+	return sandbox::SharedDocker.preflight();
	359	+	}
	360	+	if !cfg!(target_os = "linux") {
	361	+	bail!("{}=nested is for the runner's Linux image", sandbox::SANDBOX_ENV);
	362	+	}
	363	+	let isolation = sandbox::NestedDocker { allow_unsafe: sandbox::unsafe_allowed() }.preflight()?;
	364	+	if isolation != Isolation::NestedContainer {
	365	+	for problem in sandbox::Findings::read().problems() {
	366	+	log(&format!("UNSAFE: {problem}"));
	367	+	}
	368	+	log(&format!("UNSAFE: {}=1 is set, so the runner starts anyway, and its jobs can reach this machine.", sandbox::UNSAFE_ENV));
	369	+	}
	370	+	Ok(isolation)
	371	+	}
	372	+
	373	+	/// A nested runner's own Docker Engine, which it uses whatever else it
	374	+	/// can see (with the override, it starts despite that).
	375	+	fn prepare(docker: bool) -> Result<()> {
	376	+	if !docker \|\| !sandbox::nested_requested() {
	377	+	return Ok(());
	378	+	}
	379	+	#[cfg(unix)]
	380	+	{
	381	+	let version = engine::start()?;
	382	+	log(&format!("Started this runner's own Docker Engine {version}, inside its container."));
	383	+	}
	384	+	Ok(())
	385	+	}
	386	+
313	387		fn remove(folder: &Path) -> Result<()> {
314	388		let config = config::load(folder)?;
315	389		let api = Api { base: config.api.clone(), runner: config.runner.clone(), credential: config.credential.clone() };
⋯
356	430		let mut config = config::load(folder)?;
357	431		let mut api = Api { base: config.api.clone(), runner: config.runner.clone(), credential: config.credential.clone() };
358	432		std::fs::create_dir_all(&config.work_dir).with_context(\|\| format!("could not make {}", config.work_dir.display()))?;
	433	+	let isolation = isolation(config.docker)?;
	434	+	prepare(config.docker)?;
	435	+	if isolation != sandbox::Isolation::Host {
	436	+	exec::sweep(&config);
	437	+	}
359	438		log(&format!(
360		−	"g1t-runner {VERSION}: {} ({}) is listening for work from {}, {}.",
	439	+	"g1t-runner {VERSION}: {} ({}) is listening for work from {}. Isolation: {}.",
361	440		config.name,
362	441		config.labels.join(", "),
363	442		config.url,
364		−	if config.docker { "in Docker" } else { "on this machine" }
	443	+	isolation.describe()
365	444		));
	445	+	if isolation == sandbox::Isolation::Host {
	446	+	log(sandbox::HOST_MODE_AT_START);
	447	+	}
366	448		let mut current: Option<exec::Running> = None;
367	449		let mut ran_one = false;
368	450		let mut backoff = Duration::from_secs(2);
⋯
460	542		}
461	543		if let Some(work) = poll.assignment {
462	544		log(&format!("Took {} {} ({}).", work.kind, work.name, work.repo));
463		−	match exec::start(&config, folder, &work) {
	545	+	match exec::start(&config, isolation, folder, &work) {
464	546		Ok(running) => current = Some(running),
465	547		Err(error) => {
466	548		log(&format!("Could not start it: {error:#}"));

1	+	//! How the runner keeps work from the machine it is on, and what it checks
2	+	//! before it takes any.
3	+	//!
4	+	//! The goal is two walls: the runner in a sandbox of its own, away from
5	+	//! its machine, and each job in a fresh sandbox of its own inside that,
6	+	//! made for the job and destroyed after. Where the runner stands today:
7	+	//!
8	+	//! \| Isolation \| How \| Wall from the machine \|
9	+	//! \| --- \| --- \| --- \|
10	+	//! \| `nested-container` \| The runner is a container under Sysbox (`docker run --runtime=sysbox-runc`), with its own Docker Engine inside; each job is a container on that engine. \| Two: the job's container, then the runner's, whose root is no one on the machine. \|
11	+	//! \| `container` \| The runner drives a Docker Engine it shares with the machine (installed on it, or a container with the machine's Docker socket); each job is a container on it. \| One: the job's container. The runner itself can do anything the machine's Docker can, which is everything. \|
12	+	//! \| `host` \| `--no-docker`: each job is a process on the machine, in a folder of its own. Deprecated. \| None. \|
13	+	//!
14	+	//! [`SandboxDriver`] is what makes a job's sandbox. [`NestedDocker`] (and
15	+	//! [`SharedDocker`], the same on a shared engine) is built: exec.rs runs
16	+	//! jobs through it. [`HyperVContainer`] and [`MacVm`] are the designs for
17	+	//! Windows and macOS, declared so the shape is fixed, and not built.
18	+
19	+	use std::path::Path;
20	+
21	+	use anyhow::{Result, bail};
22	+
23	+	/// How well a runner keeps work from its machine, as it tells g1t when it
24	+	/// registers (`isolation`).
25	+	#[derive(Clone, Copy, Debug, PartialEq, Eq)]
26	+	pub enum Isolation {
27	+	/// A container per job, inside the runner's own sandboxed container.
28	+	NestedContainer,
29	+	/// A container per job, on a Docker Engine shared with the machine.
30	+	Container,
31	+	/// Directly on the machine.
32	+	Host,
33	+	}
34	+
35	+	impl Isolation {
36	+	pub fn as_str(self) -> &'static str {
37	+	match self {
38	+	Isolation::NestedContainer => "nested-container",
39	+	Isolation::Container => "container",
40	+	Isolation::Host => "host",
41	+	}
42	+	}
43	+
44	+	/// For the log at start.
45	+	pub fn describe(self) -> &'static str {
46	+	match self {
47	+	Isolation::NestedContainer => {
48	+	"nested-container: each job runs in a fresh container on this runner's own Docker Engine, inside the runner's sandboxed container; neither reaches the machine"
49	+	}
50	+	Isolation::Container => {
51	+	"container: each job runs in a fresh container, but the runner is not sandboxed from the machine (it shares the machine's Docker Engine, or its container's root is the machine's), so the runner, and anything that takes it over, controls the machine. Run the runner's image under Sysbox for nested-container isolation"
52	+	}
53	+	Isolation::Host => "host: jobs run directly on this machine, unsandboxed",
54	+	}
55	+	}
56	+	}
57	+
58	+	/// What `G1T_RUNNER_SANDBOX` asks for: `nested`, the runner's image's
59	+	/// setting, starts a Docker Engine inside the runner's container.
60	+	pub const SANDBOX_ENV: &str = "G1T_RUNNER_SANDBOX";
61	+
62	+	/// The override for the checks below, loudly named: with it, a runner
63	+	/// whose jobs can reach the machine starts anyway.
64	+	pub const UNSAFE_ENV: &str = "G1T_RUNNER_UNSAFE_ALLOW_HOST_ACCESS";
65	+
66	+	pub fn nested_requested() -> bool {
67	+	std::env::var(SANDBOX_ENV).is_ok_and(\|v\| v.eq_ignore_ascii_case("nested"))
68	+	}
69	+
70	+	pub fn unsafe_allowed() -> bool {
71	+	std::env::var(UNSAFE_ENV).is_ok_and(\|v\| matches!(v.as_str(), "1" \| "true" \| "yes"))
72	+	}
73	+
74	+	// --- Host mode's deprecation ---------------------------------------------
75	+
76	+	/// What the runner's log says when it starts in host mode.
77	+	pub const HOST_MODE_AT_START: &str = "Deprecated: this runner runs jobs directly on this machine (--no-docker), unsandboxed. Every job can read, change and leave behind anything this user can. Host mode will be removed: run the runner's image under Sysbox instead, see https://docs.g1t.sh/guides/self-hosted-runners/#isolation";
78	+
79	+	/// What every job's log says when it ran in host mode.
80	+	pub const HOST_MODE_IN_JOB: &str = "This job ran on the host, unsandboxed: the self-hosted runner runs jobs directly on its machine (--no-docker). Host mode will be removed; see https://docs.g1t.sh/guides/self-hosted-runners/#isolation";
81	+
82	+	/// Why a public repository's job is refused in host mode.
83	+	pub fn host_mode_refuses(repo: &str) -> String {
84	+	format!(
85	+	"{repo} is public, and this self-hosted runner runs jobs directly on its machine (--no-docker), unsandboxed, so it does not take a public repository's work. Run the runner's image under Sysbox, or register it without --no-docker. See https://docs.g1t.sh/guides/self-hosted-runners/#isolation"
86	+	)
87	+	}
88	+
89	+	// --- What the runner's container can see ----------------------------------
90	+
91	+	/// Socket names that are a container engine's: whoever has one controls
92	+	/// that engine, and through it the machine it runs on.
93	+	const ENGINE_SOCKETS: &[&str] = &["docker.sock", "containerd.sock", "podman.sock", "crio.sock", "dockershim.sock", "cri-dockerd.sock"];
94	+
95	+	fn is_engine_socket(path: &str) -> bool {
96	+	let name = path.rsplit('/').next().unwrap_or(path);
97	+	ENGINE_SOCKETS.contains(&name)
98	+	}
99	+
100	+	/// mountinfo escapes spaces and the like as `\040`; undoes it.
101	+	fn unescape(field: &str) -> String {
102	+	let bytes = field.as_bytes();
103	+	let mut out = Vec::with_capacity(bytes.len());
104	+	let mut i = 0;
105	+	while i < bytes.len() {
106	+	if bytes[i] == b'\\'
107	+	&& let Some(code) = bytes.get(i + 1..i + 4).and_then(\|d\| std::str::from_utf8(d).ok()).and_then(\|d\| u8::from_str_radix(d, 8).ok())
108	+	{
109	+	out.push(code);
110	+	i += 4;
111	+	continue;
112	+	}
113	+	out.push(bytes[i]);
114	+	i += 1;
115	+	}
116	+	String::from_utf8_lossy(&out).into_owned()
117	+	}
118	+
119	+	/// The engine sockets mounted into this container from outside it, from
120	+	/// `/proc/self/mountinfo`: a mount point, or the file a mount came from,
121	+	/// named like an engine's socket (`-v /var/run/docker.sock:/var/run/docker.sock`,
122	+	/// or the same socket under another name).
123	+	pub fn mounted_engine_sockets(mountinfo: &str) -> Vec<String> {
124	+	let mut found = Vec::new();
125	+	for line in mountinfo.lines() {
126	+	// `id parent major:minor root mount-point options [optional…] - fstype source super-options`
127	+	let fields: Vec<&str> = line.split_whitespace().collect();
128	+	if fields.len() < 5 {
129	+	continue;
130	+	}
131	+	let (root, mount_point) = (unescape(fields[3]), unescape(fields[4]));
132	+	if is_engine_socket(&mount_point) \|\| is_engine_socket(&root) {
133	+	found.push(mount_point);
134	+	}
135	+	}
136	+	found
137	+	}
138	+
139	+	/// Whether this container runs under Sysbox, whose virtual files
140	+	/// (`/proc/uptime`, `/sys/kernel`, …) come from `sysboxfs`.
141	+	pub fn under_sysbox(mountinfo: &str) -> bool {
142	+	mountinfo.lines().any(\|line\| line.split(" - ").nth(1).is_some_and(\|rest\| rest.split_whitespace().nth(1) == Some("sysboxfs")))
143	+	}
144	+
145	+	/// Whether root here is root on the machine: no user namespace, every id
146	+	/// mapped to itself (`/proc/self/uid_map` is `0 0 4294967295`).
147	+	pub fn root_is_the_machines(uid_map: &str) -> bool {
148	+	let rows: Vec<Vec<&str>> = uid_map.lines().map(\|l\| l.split_whitespace().collect()).filter(\|r: &Vec<&str>\| r.len() == 3).collect();
149	+	rows.iter().any(\|r\| r[0] == "0" && r[1] == "0" && r[2].parse::<u64>().is_ok_and(\|n\| n >= 4_294_967_295))
150	+	}
151	+
152	+	/// What a nested runner found about its container.
153	+	#[derive(Debug, Default)]
154	+	pub struct Findings {
155	+	/// Engine sockets mounted in from outside.
156	+	pub sockets: Vec<String>,
157	+	/// An engine socket that is just there, where the runner's own engine
158	+	/// does not put one: a directory such as `/var/run` mounted from the
159	+	/// machine.
160	+	pub stray_sockets: Vec<String>,
161	+	/// `DOCKER_HOST` pointing somewhere.
162	+	pub docker_host: Option<String>,
163	+	pub sysbox: bool,
164	+	pub machine_root: bool,
165	+	}
166	+
167	+	impl Findings {
168	+	pub fn read() -> Findings {
169	+	let mountinfo = std::fs::read_to_string("/proc/self/mountinfo").unwrap_or_default();
170	+	let uid_map = std::fs::read_to_string("/proc/self/uid_map").unwrap_or_default();
171	+	let mut findings = Findings::from_text(&mountinfo, &uid_map);
172	+	for path in ["/var/run/docker.sock", "/run/docker.sock", "/run/containerd/containerd.sock", "/run/podman/podman.sock"] {
173	+	// `/var/run` is often `/run`: the same file is said once.
174	+	let same = \|a: &str, b: &str\| a == b \|\| std::fs::canonicalize(a).ok().zip(std::fs::canonicalize(b).ok()).is_some_and(\|(a, b)\| a == b);
175	+	if Path::new(path).exists() && !findings.sockets.iter().any(\|s\| same(s, path)) {
176	+	findings.stray_sockets.push(path.to_owned());
177	+	}
178	+	}
179	+	findings.docker_host = std::env::var("DOCKER_HOST").ok().filter(\|h\| !h.is_empty());
180	+	findings
181	+	}
182	+
183	+	pub fn from_text(mountinfo: &str, uid_map: &str) -> Findings {
184	+	Findings {
185	+	sockets: mounted_engine_sockets(mountinfo),
186	+	sysbox: under_sysbox(mountinfo),
187	+	machine_root: root_is_the_machines(uid_map),
188	+	..Findings::default()
189	+	}
190	+	}
191	+
192	+	/// What reaches the machine from here, one reason a line.
193	+	pub fn problems(&self) -> Vec<String> {
194	+	let mut problems = Vec::new();
195	+	for socket in &self.sockets {
196	+	problems.push(format!("{socket} is a container engine's socket mounted in from outside this container: whoever uses it controls the machine"));
197	+	}
198	+	for socket in &self.stray_sockets {
199	+	problems.push(format!("{socket} is a container engine's socket that this runner did not make (a folder mounted from the machine?)"));
200	+	}
201	+	if let Some(host) = &self.docker_host {
202	+	problems.push(format!("DOCKER_HOST is set ({host}): jobs would run on that engine, not this runner's own"));
203	+	}
204	+	if self.machine_root && !self.sysbox {
205	+	problems.push("root in this container is root on the machine (it has no user namespace of its own): run it with --runtime=sysbox-runc, and without --privileged".to_owned());
206	+	}
207	+	problems
208	+	}
209	+	}
210	+
211	+	/// Decides a nested runner's isolation, or refuses to start. With
212	+	/// [`UNSAFE_ENV`] it starts anyway, as `container`, and says why loudly.
213	+	pub fn check_nested(findings: &Findings, allow_unsafe: bool) -> Result<Isolation> {
214	+	let problems = findings.problems();
215	+	if problems.is_empty() {
216	+	return Ok(Isolation::NestedContainer);
217	+	}
218	+	if allow_unsafe {
219	+	return Ok(Isolation::Container);
220	+	}
221	+	bail!(
222	+	"this runner is not sandboxed from its machine, so it will not take work:\n - {}\nRun it as `docker run --runtime=sysbox-runc …` without the machine's Docker socket (see https://docs.g1t.sh/guides/self-hosted-runners/#run-it-as-a-container), or set {UNSAFE_ENV}=1 to run anyway with jobs able to reach the machine.",
223	+	problems.join("\n - ")
224	+	)
225	+	}
226	+
227	+	// --- Sandbox drivers --------------------------------------------------------
228	+
229	+	/// What makes and destroys a job's sandbox. Each job gets a fresh one,
230	+	/// which never touches the machine, and which is gone after the job.
231	+	/// Making and destroying a Docker sandbox is exec.rs; a driver for another
232	+	/// kind adds its own when it is built.
233	+	pub trait SandboxDriver {
234	+	/// What it is called in the log.
235	+	fn name(&self) -> &'static str;
236	+	/// Whether this machine can make sandboxes with it: the isolation it
237	+	/// gives here, or why it cannot.
238	+	fn preflight(&self) -> Result<Isolation>;
239	+	}
240	+
241	+	/// Linux, built: the runner's own Docker Engine, inside the runner's
242	+	/// Sysbox container (engine.rs), and a container per job on it (exec.rs),
243	+	/// with no mounts from the machine. `allow_unsafe`: [`UNSAFE_ENV`] is set.
244	+	pub struct NestedDocker {
245	+	pub allow_unsafe: bool,
246	+	}
247	+
248	+	impl SandboxDriver for NestedDocker {
249	+	fn name(&self) -> &'static str {
250	+	"Docker inside the runner's container"
251	+	}
252	+	fn preflight(&self) -> Result<Isolation> {
253	+	check_nested(&Findings::read(), self.allow_unsafe)
254	+	}
255	+	}
256	+
257	+	/// The same containers, on a Docker Engine shared with the machine: a
258	+	/// runner installed on the machine, or a container given its socket.
259	+	pub struct SharedDocker;
260	+
261	+	impl SandboxDriver for SharedDocker {
262	+	fn name(&self) -> &'static str {
263	+	"the machine's Docker"
264	+	}
265	+	fn preflight(&self) -> Result<Isolation> {
266	+	Ok(Isolation::Container)
267	+	}
268	+	}
269	+
270	+	/// The driver for jobs native to this OS, once there is one: what a
271	+	/// Windows or macOS runner in host mode says is coming.
272	+	pub fn native_driver() -> Option<Box<dyn SandboxDriver>> {
273	+	match std::env::consts::OS {
274	+	"windows" => Some(Box::new(HyperVContainer)),
275	+	"macos" => Some(Box::new(MacVm)),
276	+	_ => None,
277	+	}
278	+	}
279	+
280	+	/// Windows: not built. The design:
281	+	///
282	+	/// - A minimal launcher on the host, the only part that runs there. It
283	+	/// polls nothing and runs no job code: it creates a sandbox from a
284	+	/// signed image when the runner inside asks for one, and destroys it
285	+	/// when the job ends. Its whole job is "create from signed image" and
286	+	/// "destroy", so there is little to attack.
287	+	/// - Each job is a Windows container with Hyper-V isolation
288	+	/// (`docker run --isolation=hyperv`, or the HCS API directly): its own
289	+	/// utility VM and kernel, so a job that escapes its container is still
290	+	/// in a VM, not on the host. Process isolation shares the host's kernel
291	+	/// and is not enough.
292	+	/// - The image is pinned by digest and its signature checked before it is
293	+	/// used; the job gets the harness read-only, its own token, and a
294	+	/// scratch disk destroyed with the VM.
295	+	///
296	+	/// It needs a Windows host with Hyper-V and the Containers feature, and
297	+	/// cannot be built or tested on Linux. Until it is, Windows jobs run in
298	+	/// host mode, which is deprecated.
299	+	pub struct HyperVContainer;
300	+
301	+	impl SandboxDriver for HyperVContainer {
302	+	fn name(&self) -> &'static str {
303	+	"Windows containers with Hyper-V isolation"
304	+	}
305	+	fn preflight(&self) -> Result<Isolation> {
306	+	bail!("Hyper-V-isolated Windows containers are not built yet; Windows jobs run on the host (deprecated)")
307	+	}
308	+	}
309	+
310	+	/// macOS: not built. The design:
311	+	///
312	+	/// - The same minimal host launcher as on Windows, using Apple's
313	+	/// Virtualization framework (`VZVirtualMachine`) to boot an ephemeral
314	+	/// macOS VM per job from a signed base image, cloned copy-on-write
315	+	/// (APFS clones) so a fresh VM costs seconds, and deleted after.
316	+	/// - Apple's licence allows at most two macOS VMs per Mac at once, so a
317	+	/// Mac runs two jobs at a time at most, and the launcher refuses a
318	+	/// third.
319	+	/// - The runner and harness run inside the VM; the host keeps only the
320	+	/// launcher and the signed images.
321	+	///
322	+	/// It needs Apple silicon and macOS 13 or later, and cannot be built or
323	+	/// tested on Linux. Until it is, macOS jobs run in host mode, which is
324	+	/// deprecated.
325	+	pub struct MacVm;
326	+
327	+	impl MacVm {
328	+	/// Apple's macOS licence: two virtual copies per Mac.
329	+	pub const MAX_VMS: u32 = 2;
330	+	}
331	+
332	+	impl SandboxDriver for MacVm {
333	+	fn name(&self) -> &'static str {
334	+	"ephemeral macOS VMs"
335	+	}
336	+	fn preflight(&self) -> Result<Isolation> {
337	+	bail!("ephemeral macOS VMs (at most {} per Mac) are not built yet; macOS jobs run on the host (deprecated)", MacVm::MAX_VMS)
338	+	}
339	+	}
340	+
341	+	#[cfg(test)]
342	+	mod tests {
343	+	use super::*;
344	+
345	+	const PLAIN: &str = "\
346	+	611 590 0:52 / / rw,relatime master:180 - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/A,upperdir=/var/lib/docker/overlay2/x/diff
347	+	612 611 0:55 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
348	+	620 611 254:1 /var/lib/docker/volumes/g1t-runner/_data /data rw,relatime - ext4 /dev/vda1 rw
349	+	621 611 254:1 /var/lib/docker/containers/abc/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/vda1 rw
350	+	";
351	+
352	+	const WITH_SOCKET: &str = "\
353	+	611 590 0:52 / / rw,relatime master:180 - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/A
354	+	622 611 0:26 /docker.sock /run/docker.sock rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,size=3280376k,mode=755
355	+	";
356	+
357	+	const RENAMED_SOCKET: &str = "\
358	+	623 611 0:26 /docker.sock /tmp/engine rw,nosuid,nodev - tmpfs tmpfs rw
359	+	624 611 0:27 /containerd/containerd.sock /run/containerd/containerd.sock rw - tmpfs tmpfs rw
360	+	";
361	+
362	+	const SYSBOX: &str = "\
363	+	1000 990 0:60 / / rw,relatime - shiftfs /var/lib/sysbox/shiftfs/x rw
364	+	1001 1000 0:61 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
365	+	1002 1001 0:62 /proc/uptime /proc/uptime rw,nosuid,nodev,relatime - fuse sysboxfs rw,user_id=0,group_id=0,default_permissions,allow_other
366	+	1003 1000 254:1 /var/lib/sysbox/docker/abc /var/lib/docker rw,relatime - ext4 /dev/vda1 rw
367	+	";
368	+
369	+	#[test]
370	+	fn a_mounted_docker_socket_is_found() {
371	+	assert!(mounted_engine_sockets(PLAIN).is_empty());
372	+	assert_eq!(mounted_engine_sockets(WITH_SOCKET), vec!["/run/docker.sock".to_owned()]);
373	+	// The same socket under another name, and containerd's.
374	+	assert_eq!(mounted_engine_sockets(RENAMED_SOCKET), vec!["/tmp/engine".to_owned(), "/run/containerd/containerd.sock".to_owned()]);
375	+	// Sysbox's own Docker data mount is not a socket.
376	+	assert!(mounted_engine_sockets(SYSBOX).is_empty());
377	+	}
378	+
379	+	#[test]
380	+	fn escaped_paths_are_read() {
381	+	assert_eq!(mounted_engine_sockets("1 0 0:1 /a\\040b/docker.sock /x\\040y rw - tmpfs tmpfs rw"), vec!["/x y".to_owned()]);
382	+	}
383	+
384	+	#[test]
385	+	fn sysbox_and_a_user_namespace_are_told() {
386	+	assert!(under_sysbox(SYSBOX));
387	+	assert!(!under_sysbox(PLAIN));
388	+	assert!(root_is_the_machines(" 0 0 4294967295\n"));
389	+	assert!(!root_is_the_machines(" 0 165536 65536\n"));
390	+	}
391	+
392	+	#[test]
393	+	fn a_nested_runner_refuses_the_machines_socket_unless_told_loudly() {
394	+	let sysbox = Findings::from_text(SYSBOX, "0 165536 65536");
395	+	assert_eq!(check_nested(&sysbox, false).unwrap(), Isolation::NestedContainer);
396	+
397	+	let socket = Findings::from_text(&format!("{SYSBOX}{WITH_SOCKET}"), "0 165536 65536");
398	+	let refused = check_nested(&socket, false).unwrap_err().to_string();
399	+	assert!(refused.contains("/run/docker.sock"), "{refused}");
400	+	assert!(refused.contains(UNSAFE_ENV), "{refused}");
401	+	assert_eq!(check_nested(&socket, true).unwrap(), Isolation::Container);
402	+
403	+	// --privileged, without Sysbox: root here is the machine's.
404	+	let privileged = Findings::from_text(PLAIN, "0 0 4294967295");
405	+	assert!(check_nested(&privileged, false).unwrap_err().to_string().contains("sysbox-runc"));
406	+
407	+	let host = Findings { docker_host: Some("tcp://10.0.0.1:2375".into()), ..Findings::from_text(SYSBOX, "0 165536 65536") };
408	+	assert!(check_nested(&host, false).is_err());
409	+	}
410	+
411	+	#[test]
412	+	fn host_mode_says_it_is_deprecated() {
413	+	assert!(HOST_MODE_AT_START.starts_with("Deprecated:"));
414	+	assert!(HOST_MODE_AT_START.contains("will be removed"));
415	+	assert!(HOST_MODE_IN_JOB.starts_with("This job ran on the host, unsandboxed"));
416	+	assert!(HOST_MODE_IN_JOB.contains("Host mode will be removed"));
417	+	assert!(host_mode_refuses("acme/web").contains("acme/web is public"));
418	+	assert_eq!(Isolation::NestedContainer.as_str(), "nested-container");
419	+	assert_eq!(Isolation::Host.as_str(), "host");
420	+	}
421	+
422	+	#[test]
423	+	fn the_unbuilt_drivers_say_so() {
424	+	assert!(HyperVContainer.preflight().is_err());
425	+	assert!(MacVm.preflight().unwrap_err().to_string().contains("at most 2"));
426	+	assert_eq!(SharedDocker.preflight().unwrap(), Isolation::Container);
427	+	assert_eq!(MacVm::MAX_VMS, 2);
428	+	}
429	+	}

1	1		# syntax=docker/dockerfile:1.7
2	2		#
3		−	# The self-hosted runner's image: `g1t-runner`, the Docker CLI (to start
4		−	# each job in a sibling container through the host's Docker socket), and
5		−	# what a job needs to run in this container itself with --no-docker
6		−	# (git, bash, Node), for Kubernetes pods that take one job each.
	3	+	# The self-hosted runner's image: `g1t-runner` with a Docker Engine of its
	4	+	# own, for Sysbox. The runner starts `dockerd` inside this container and
	5	+	# runs each job in a fresh container on it, so jobs are containers inside
	6	+	# the runner's container and nothing reaches the machine: Sysbox gives the
	7	+	# container its own user namespace (its root is no one on the machine) and
	8	+	# lets it run Docker without --privileged. Never mount the machine's
	9	+	# /var/run/docker.sock: the runner refuses to start when it sees one.
7	10		#
8		−	# docker run -d --restart unless-stopped \
9		−	# -v /var/run/docker.sock:/var/run/docker.sock -v g1t-runner:/data \
	11	+	# docker run -d --name g1t-runner --restart unless-stopped \
	12	+	# --runtime=sysbox-runc \
	13	+	# -v g1t-runner:/data \
10	14		# -e G1T_RUNNER_URL=https://g1t.sh -e G1T_RUNNER_TOKEN=g1trt_… \
11	15		# g1t.sh/flagon-io/g1t-runner
12	16		#
	17	+	# PID 1 is docker-init (tini), which supervises g1t-runner and reaps what it
	18	+	# leaves; g1t-runner supervises dockerd (crates/runner/src/selfhosted/engine.rs).
	19	+	# git, bash and Node stay for host mode (G1T_RUNNER_NO_DOCKER=1), which is
	20	+	# deprecated.
	21	+	#
13	22		# Build context: a folder holding `g1t-runner` for the image's platform,
14	23		# the static Linux build that scripts/runner-release.mjs makes (or
15	24		# scripts/build-runner.mjs, for x64). The release workflow builds and
16	25		# pushes it for linux/amd64 and linux/arm64.
17	26
18		−	FROM docker:28-cli AS docker
	27	+	# Docker's static builds: the engine, containerd, runc, the CLI, tini.
	28	+	FROM docker:28-dind AS docker
19	29
20	30		FROM node:24-bookworm-slim
21	31		RUN apt-get update \
22		−	&& apt-get install -y --no-install-recommends git ca-certificates curl \
	32	+	&& apt-get install -y --no-install-recommends git ca-certificates curl iptables procps \
23	33		&& rm -rf /var/lib/apt/lists/* \
24		−	&& mkdir -p /data /home/runner && chown -R node:node /data /home/runner
25		−	COPY --from=docker /usr/local/bin/docker /usr/local/bin/docker
	34	+	&& mkdir -p /data /home/runner
	35	+	COPY --from=docker /usr/local/bin/dockerd /usr/local/bin/containerd /usr/local/bin/containerd-shim-runc-v2 \
	36	+	/usr/local/bin/ctr /usr/local/bin/runc /usr/local/bin/docker-proxy /usr/local/bin/docker-init \
	37	+	/usr/local/bin/docker /usr/local/bin/
26	38		COPY --chmod=0755 g1t-runner /usr/local/bin/g1t-runner
27		−	# Its configuration and credential live on this volume.
28		−	ENV G1T_RUNNER_DIR=/data
	39	+	# Its configuration and credential live on this volume. Sysbox keeps the
	40	+	# engine's /var/lib/docker on the machine's disk for this container alone.
	41	+	ENV G1T_RUNNER_DIR=/data \
	42	+	G1T_RUNNER_SANDBOX=nested
29	43		VOLUME /data
30		−	ENTRYPOINT ["g1t-runner"]
	44	+	ENTRYPOINT ["docker-init", "--", "g1t-runner"]
31	45		CMD ["register-and-run"]

826	826		} else if total >= model::MAX_RUNNERS {
827	827		return Ok(fail(FailureCode::Conflict, format!("A workspace has at most {} self-hosted runners.", model::MAX_RUNNERS)));
828	828		}
	829	+	// How it keeps work from its machine, if it said (older runners do not).
	830	+	let isolation = a.isolation.as_deref().filter(\|level\| model::ISOLATIONS.contains(level));
829	831		let id = new_id("rnr", at);
830	832		let credential = format!("{}{}", model::CREDENTIAL_PREFIX, random_hex(32));
831	833		let created = rfc3339(at);
832	834		self.db
833	835		.prepare(
834	836		"INSERT INTO runners (id, workspace, repo_id, repo, group_id, name, labels, os, arch, version, ephemeral, credential_hash,
835		−	rotated_at, last_seen_at, created_at, created_by)
836		−	VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
	837	+	rotated_at, last_seen_at, created_at, created_by, isolation)
	838	+	VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
837	839		)
838	840		.bind(&[
839	841		id.as_str().into(),
⋯
852	854		created.as_str().into(),
853	855		created.as_str().into(),
854	856		JsValue::NULL,
	857	+	optional(isolation),
855	858		])?
856	859		.run()
857	860		.await?;
⋯
1073	1076		// A job that deploys: its run's deployment is under way.
1074	1077		self.job_started(&job.id).await?;
1075	1078		let image = self.job_image(&job).await?;
	1079	+	let visibility = self.visibility(&job.repo_id).await;
1076	1080		return Ok(Some(Assignment {
1077	1081		kind: "workflow".into(),
1078	1082		id: job.id.clone(),
⋯
1082	1086		image,
1083	1087		token: Some(token),
1084	1088		env: None,
	1089	+	visibility,
1085	1090		}));
1086	1091		}
1087	1092
⋯
1144	1149		format!("The self-hosted runner {} took agent work: {}.", runner.name, task.title),
1145	1150		)])
1146	1151		.await;
	1152	+	let visibility = match &task.repo_id {
	1153	+	Some(repo_id) => self.visibility(repo_id).await,
	1154	+	None => None,
	1155	+	};
1147	1156		return Ok(Some(Assignment {
1148	1157		kind: "agent".into(),
1149	1158		id: task.id.clone(),
⋯
1153	1162		image: None,
1154	1163		token: None,
1155	1164		env: Some(env),
	1165	+	visibility,
1156	1166		}));
1157	1167		}
1158	1168		Ok(None)
1159	1169		}
1160	1170
	1171	+	/// A repository's visibility for a runner, `public` or `private`; none
	1172	+	/// when it cannot be read, and the runner decides without it.
	1173	+	async fn visibility(&self, repo_id: &str) -> Option<String> {
	1174	+	match self.repo_by_id(repo_id).await {
	1175	+	Ok(Some((repo, _))) => Some(if repo.is_private { "private" } else { "public" }.to_owned()),
	1176	+	_ => None,
	1177	+	}
	1178	+	}
	1179	+
1161	1180		/// The image a job's `container:` names, when it names one plainly.
1162	1181		async fn job_image(&self, job: &JobRow) -> Result<Option<String>> {
1163	1182		let Some(run) = self.run_row(&job.run_id).await? else { return Ok(None) };

1096	1096
1097	1097		[[package]]
1098	1098		name = "g1t-runner"
1099		−	version = "0.1.2"
	1099	+	version = "0.2.0"
1100	1100		dependencies = [
1101	1101		"anyhow",
1102	1102		"base64 0.22.1",

358	358		/// Replace a runner of the same name instead of refusing.
359	359		#[serde(default)]
360	360		pub replace: bool,
	361	+	/// How the runner keeps work from its machine: `nested-container` (a
	362	+	/// container per job, inside the runner's own sandboxed container),
	363	+	/// `container` (a container per job, on a Docker Engine it shares with
	364	+	/// its machine) or `host` (directly on the machine). Kept, and shown
	365	+	/// nowhere yet; older runners leave it out.
	366	+	#[serde(default)]
	367	+	pub isolation: Option<String>,
361	368		}
362	369
	370	+	/// The isolation levels a runner may say it has (`RegisterArgs::isolation`).
	371	+	pub const ISOLATIONS: &[&str] = &["nested-container", "container", "host"];
	372	+
363	373		#[derive(Clone, Debug, Serialize, Deserialize)]
364	374		pub struct Registered {
365	375		pub runner: Runner,
⋯
406	416		/// started with: `MODE`, the run's short-lived credentials, the model
407	417		/// proxy's address.
408	418		pub env: Option<Map<String, Value>>,
	419	+	/// The repository's visibility, `public` or `private`, when g1t knows
	420	+	/// it: a runner that runs work directly on its machine refuses a
	421	+	/// public repository's. Older g1t leaves it out.
	422	+	#[serde(default)]
	423	+	pub visibility: Option<String>,
409	424		}
410	425
411	426		#[derive(Clone, Debug, Default, Serialize, Deserialize)]
⋯
686	701		image: None,
687	702		token: Some("t".into()),
688	703		env: None,
	704	+	visibility: Some("private".into()),
689	705		})
690	706		.unwrap();
691	707		assert_eq!(assignment["timeout_minutes"], 60);
	708	+	assert_eq!(assignment["visibility"], "private");
	709	+	// An older runner's registration, without an isolation level.
	710	+	let register: RegisterArgs = serde_json::from_value(serde_json::json!({ "token": "g1trt_x", "name": "a", "os": "linux", "arch": "x64" })).unwrap();
	711	+	assert_eq!(register.isolation, None);
692	712		}
693	713		}

1	1		[package]
2	2		name = "g1t-runner"
3		−	version = "0.1.2"
	3	+	version = "0.2.0"
4	4		edition.workspace = true
5	5		license.workspace = true
6	6		description = "The program inside a g1t sandbox: runs an agent, checks, a catch-up merge, a review or a GitHub Actions job, and reports back. Also the self-hosted runner, g1t-runner register/run, on your own machines."

880	880		job.docker_hosted = false;
881	881		}
882	882		}
	883	+	// A self-hosted runner's word on how this job was run (selfhosted/exec.rs).
	884	+	if let Ok(notice) = std::env::var("G1T_JOB_NOTICE") {
	885	+	job.log.line(&format!("##[warning]{notice}"));
	886	+	}
883	887		// Actions first, then containers, as GitHub's "Set up job" and
884	888		// "Initialize containers". Each action's `pre` (read from the actions
885	889		// just downloaded) is a step of its own, "Pre <step>", before the

1	+	-- How a self-hosted runner keeps work from its machine, as it said when it
2	+	-- registered: nested-container, container or host. Null for runners that
3	+	-- registered before runners said so.
4	+	ALTER TABLE runners ADD COLUMN isolation TEXT;

5	5		//!
6	6		//! \| Route \| Body \|
7	7		//! \| --- \| --- \|
8		−	//! \| `POST /runners/register` \| `token`, `name`, `labels`, `os`, `arch`, `version`, `ephemeral`, `group`, `replace` \|
	8	+	//! \| `POST /runners/register` \| `token`, `name`, `labels`, `os`, `arch`, `version`, `ephemeral`, `group`, `replace`, `isolation` \|
9	9		//! \| `POST /runners/{id}/poll` \| `version`, `running`, `wait_ms` \|
10	10		//! \| `POST /runners/{id}/finished` \| `id`, `exit_code`, `reason` \|
11	11		//! \| `POST /runners/{id}/remove` \| nothing \|
⋯
56	56		"ephemeral": body["ephemeral"].as_bool().unwrap_or(false),
57	57		"group": body["group"].as_str(),
58	58		"replace": body["replace"].as_bool().unwrap_or(false),
	59	+	"isolation": body["isolation"].as_str(),
59	60		});
60	61		return Ok(Some(answer(services, "runner_register", &args).await?));
61	62		}

245	245		stop it in a later one; write a schedule's `cron` in UTC.
246	246		- Status. Not scheduled.
247	247
	248	+	### Self-hosted Windows and macOS jobs run on the host
	249	+
	250	+	On a self-hosted runner, a job native to Windows or macOS runs directly on
	251	+	the machine, unsandboxed, in host mode, which is deprecated. Only Linux
	252	+	jobs run in a sandbox of their own, a container inside the runner's own
	253	+	container under Sysbox.
	254	+
	255	+	- Why. Sandboxes for them are not built yet: Windows containers with
	256	+	Hyper-V isolation, and ephemeral macOS virtual machines (two per Mac at
	257	+	most, by Apple's licence).
	258	+	- Instead. Run Linux jobs in
	259	+	[the runner's container](/guides/self-hosted-runners/#run-it-as-a-container),
	260	+	and keep Windows and macOS runners for repositories whose code you
	261	+	trust. Host mode refuses public repositories' work.
	262	+	- Status. Planned. See
	263	+	[Windows and macOS](/guides/self-hosted-runners/#windows-and-macos).
	264	+
	265	+	### A self-hosted job's Docker can reach its runner's container
	266	+
	267	+	On a runner in its container under Sysbox, a job that uses Docker gets an
	268	+	engine of its own that is privileged inside the runner's container. It
	269	+	cannot reach the machine, but it can reach the runner's container,
	270	+	including the runner's credential.
	271	+
	272	+	- Instead. Use ephemeral runners for code you do not trust, or
	273	+	register with `--no-job-docker`.
	274	+	- Status. Not scheduled.
	275	+
248	276		### No `gh` command in jobs
249	277
250	278		The runner does not include the `gh` command, and pointing it at g1t

150	150		case "docker":
151	151		return [
152	152		{
153		−	title: "Register and run, with jobs in sibling containers",
	153	+	title: "Register and run under Sysbox, each job in a container inside the runner's",
154	154		lines: [
155		−	`docker run -d --name g1t-runner --restart unless-stopped -v /var/run/docker.sock:/var/run/docker.sock -v g1t-runner:/data -e G1T_RUNNER_DIR=/data ${RUNNER_IMAGE} register-and-run --url ${url} --token ${value}`,
	155	+	`docker run -d --name g1t-runner --restart unless-stopped --runtime=sysbox-runc -v g1t-runner:/data ${RUNNER_IMAGE} register-and-run --url ${url} --token ${value}`,
156	156		],
157	157		},
158	158		];
⋯
232	232		))}
233	233		</ol>
234	234		<p className="text-xs text-faint">
235		−	Jobs run in Docker containers by default; add <code className="font-mono">--no-docker</code> to run them on the machine itself. Add{" "}
	235	+	Each job runs in a container of its own; on Linux, run the Docker command under Sysbox so neither the runner nor its jobs reach the machine.{" "}
	236	+	<code className="font-mono">--no-docker</code>, which runs jobs on the machine itself, is deprecated. Add{" "}
236	237		<code className="font-mono">--labels gpu,cuda</code> for jobs to ask for, or <code className="font-mono">--ephemeral</code> for a runner that takes
237	238		one job and removes itself.{" "}
238	239		<a href="https://docs.g1t.sh/guides/self-hosted-runners/" className="underline underline-offset-2">

277	277		if self.docker_output(&["version", "--format", "{{.Server.Version}}"]).is_some() {
278	278		return true;
279	279		}
280		−	self.log.line("##[error]This job needs Docker (`services`, `container` or a Docker action), and Docker does not answer here. On a self-hosted runner, run the runner directly on a machine with Docker (`--no-docker`).");
	280	+	self.log.line("##[error]This job needs Docker (`services`, `container` or a Docker action), and Docker does not answer here. On a self-hosted runner, run the runner's image under Sysbox, which gives each job that uses Docker an engine of its own.");
281	281		false
282	282		}
283	283
⋯
355	355		// A self-hosted machine without Docker runs the steps as before,
356	356		// on the machine, without the containers.
357	357		if !self.docker_hosted && self.docker_output(&["version", "--format", "{{.Server.Version}}"]).is_none() {
358		−	self.log.line("##[warning]Docker does not answer on this runner, so the job's `services` and `container` are not started and its steps run on the machine. Run the runner directly on a machine with Docker (`--no-docker`) to start them.");
	358	+	self.log.line("##[warning]Docker does not answer on this runner, so the job's `services` and `container` are not started and its steps run on the machine. Run the runner's image under Sysbox to give each job Docker of its own.");
359	359		return true;
360	360		}
361	361		self.log.line("##[group]Initialize containers");

22	22		pub token: Option<String>,
23	23		#[serde(default)]
24	24		pub env: Option<Map<String, Value>>,
	25	+	/// `public` or `private`; none from an older g1t.
	26	+	#[serde(default)]
	27	+	pub visibility: Option<String>,
25	28		}
26	29
27	30		#[derive(Debug, Default, Deserialize)]

45	45		/// Checks for new versions and updates itself.
46	46		#[serde(default = "yes")]
47	47		pub auto_update: bool,
	48	+	/// Each job container's memory limit (`--memory`): `8g`. Three
	49	+	/// quarters of the machine's (or the runner container's) memory if
	50	+	/// left out.
	51	+	#[serde(default)]
	52	+	pub memory: Option<String>,
	53	+	/// Each job container's CPUs (`--cpus`): `2`, `1.5`. Every CPU if left out.
	54	+	#[serde(default)]
	55	+	pub cpus: Option<String>,
	56	+	/// The most processes in a job container (`--pids-limit`). 4096 if left out.
	57	+	#[serde(default)]
	58	+	pub pids_limit: Option<u32>,
	59	+	/// With nested-container isolation, a job that uses Docker gets an
	60	+	/// engine of its own. Off: jobs have no Docker.
	61	+	#[serde(default = "yes")]
	62	+	pub job_docker: bool,
	63	+	/// The image a job's own engine runs: `docker:28-dind` if left out.
	64	+	#[serde(default)]
	65	+	pub job_docker_image: Option<String>,
48	66		}
49	67
50	68		fn yes() -> bool {
⋯
122	140		}
123	141
124	142		#[test]
	143	+	fn an_older_configuration_still_reads() {
	144	+	let old = r#"{"url":"https://g1t.sh","api":"https://api.g1t.sh","runner":"rnr_1","name":"a","credential":"g1tr_x",
	145	+	"workspace":"acme","labels":["self-hosted"],"work_dir":"/tmp/w"}"#;
	146	+	let config: Config = serde_json::from_str(old).unwrap();
	147	+	assert!(config.docker && config.job_docker && config.auto_update);
	148	+	assert_eq!(config.memory, None);
	149	+	}
	150	+
	151	+	#[test]
125	152		fn a_configuration_survives_a_round_trip() {
126	153		let dir = std::env::temp_dir().join(format!("g1t-runner-config-{}", std::process::id()));
127	154		let config = Config {
⋯
141	168		agent_image: None,
142	169		harness: None,
143	170		auto_update: true,
	171	+	memory: Some("4g".into()),
	172	+	cpus: None,
	173	+	pids_limit: None,
	174	+	job_docker: true,
	175	+	job_docker_image: None,
144	176		};
145	177		save(&dir, &config).unwrap();
146	178		let back = load(&dir).unwrap();
147	179		assert_eq!(back.runner, "rnr_1");
148	180		assert_eq!(back.labels, config.labels);
	181	+	assert_eq!(back.memory.as_deref(), Some("4g"));
149	182		forget(&dir);
150	183		assert!(load(&dir).is_err());
151	184		let _ = std::fs::remove_dir_all(&dir);