Skip to content
Sign up

flagon-io/g1t

public
00

Where people and agents ship software together. The open-source git platform for the whole job: issues, agents, checks and deploys to the edge.

g1t.sh
gitagentscloudflarerustforgemcp

Commit

A job that runs again is no longer stopped by the time cap of the attempt before it. A sandbox's Durable Object is named for its work (actions:{job} for a workflow job), and a re-run keeps the job's id, so every attempt of a job runs in the same object; each attempt scheduled its time cap's alarm there and none cleared it. The Desktop release's Linux job showed it: attempt 1 started at 12:47:17 with a 60-minute cap, so its alarm fired at 13:50:17, four minutes into attempt 3, and stopped attempt 3's build mid-compile (its log's last line is at 13:50:17). Now a sandbox clears the alarm, the cap and the stop reason an earlier attempt left when it starts, and the alarm stops a run only once that run has used its own whole cap, so alarms already scheduled before this deploy cannot do it either.

That job then waited ten minutes to fail as "The runner stopped answering.", since the container library tells a sandbox g1t destroyed that it exited cleanly, and a clean exit was taken as success with nothing to report. A sandbox g1t stopped now fails whatever exit code comes back, saying why (its time cap, or mining), at once. No deploy of the runner ran at the time: its Worker was deployed at 12:32:56 and next at 14:03:57. A deploy that changes the image leaves sandboxes that started within two hours to finish (rollout_active_grace_period), which covers every workflow job on g1t's machines; the Cloudflare guide now says so, and the Actions guide says a re-run has its whole time limit.

syntaqxcommitted Parent4d0236bBrowse files
5 files+81−100/5 viewed
+3−0
14311431 the page to read it. Its jobs have ids of their own, so a link to an
14321432 earlier attempt's job keeps showing that job's log.
14331433
1434+A job that runs again has its whole time limit from when it starts: the
1435+time cap of the attempt before it never stops it.
1436+
14341437 #### Debug logging
14351438
14361439 Each re-run asks whether to **Enable debug logging**. The new attempt's
+6−1
295295 runner's image is tagged with a hash of what it is built from, so a deploy
296296 builds it only when the registry does not already have it. When nothing
297297 the image is built from changed, the deploy leaves running sandboxes
298−alone.
298+alone. When it did, Cloudflare replaces sandboxes on the new image but
299+leaves any that started within the last two hours
300+(`rollout_active_grace_period` in `services/runner/wrangler.jsonc`) to
301+finish first. A workflow job on g1t's machines runs for at most 60
302+minutes, so a deploy never cuts one short; an agent run whose guardrails
303+allow more than two hours can be.
299304
300305 Jobs choose larger machines with `runs-on: g1t-2core` or `g1t-4core`. Your
301306 account's Containers limits must allow `standard-4`.
+21−8
9292 import { buildMentionPrompt, describeThread, handleMention, jobTokenRefusal, planMention } from "./mentions";
9393 import { instructionsFor, repoInstructions, withBlock } from "./repo-instructions";
9494 import { cancelTask, enqueueTask, handedOverStep, selfHostedRoute, taskEnv, taskRepo } from "./self-hosted";
95−import { answered, describeError, tellStopped, withinTimeCap } from "./lifecycle";
95+import { answered, capReached, describeError, stoppedShort, tellStopped, withinTimeCap } from "./lifecycle";
9696 import {
9797 ABUSE_EXIT_CODE,
9898 ABUSE_HOST,
480480 throw error;
481481 }
482482 await this.ctx.storage.put("run", run);
483− await this.ctx.storage.delete(["abuse", "stopReason", "remote", "misdirected"]);
483+ // What an earlier attempt in this object left: its reason, its cap
484+ // and its cap's alarm, which would otherwise stop this attempt when
485+ // the earlier one's time ran out (re-runs use the same object).
486+ await this.ctx.storage.delete(["abuse", "stopReason", "remote", "misdirected", "timeCap"]);
487+ this.deleteSchedules("timeUp");
484488 if (meter) await this.ctx.storage.put("meter", { ...meter, started: Date.now() });
485489 await this.ctx.storage.put("started", Date.now());
486490 const who = meter ? { workspace: meter.workspace, repo: meter.repo } : owner;
681685
682686 /** The run's time cap has passed: stop it, as stopped for time. */
683687 async timeUp(): Promise<void> {
684− // It already stopped: nothing to stop.
685− if (!(await this.ctx.storage.get<number>("started"))) return;
686− const tracked = await this.ctx.storage.get<TrackedRun>("agentRun");
688+ // It already stopped (nothing to stop), or the run in it now is a later
689+ // attempt whose cap has not passed: this alarm is an earlier attempt's,
690+ // left by a version of g1t that did not clear it, and the run now has
691+ // its own.
692+ const started = await this.ctx.storage.get<number>("started");
687693 const minutes = (await this.ctx.storage.get<number>("timeCap")) ?? 0;
694+ if (!capReached(started, minutes, Date.now())) {
695+ if (started) console.log("an earlier attempt's time cap passed; the run now keeps going", this.ctx.id.toString());
696+ return;
697+ }
698+ const tracked = await this.ctx.storage.get<TrackedRun>("agentRun");
688699 // A workflow job or a build has no run to halt: it fails saying why.
689700 await this.ctx.storage.put("stopReason", timeCapMessage(minutes));
690701 if (tracked) await reportRun(this.env.WORK, tracked, { halt: "time", error: timeCapMessage(minutes) });
816827 const flagged = (await this.ctx.storage.get<boolean>("abuse")) ?? false;
817828 // Why it stopped, when g1t stopped it: said in place of a plain failure.
818829 const why = flagged ? ABUSE_MESSAGE : ((await this.ctx.storage.get<string>("stopReason")) ?? null);
830+ // Stopped by g1t, it failed whatever exit code the library passed on.
831+ const short = stoppedShort(exitCode, why);
819832 const ended = await this.closeRun(
820− exitCode === 0 ? "succeeded" : "failed",
821− exitCode === 0 ? undefined : (why ?? `The sandbox exited with ${exitCode}.`),
833+ short ? "failed" : "succeeded",
834+ short ? (why ?? `The sandbox exited with ${exitCode}.`) : undefined,
822835 );
823836 await this.settleStopped(started, tracked);
824837 await this.ctx.storage.delete("started");
825− if (exitCode === 0 && !flagged) return;
838+ if (!short) return;
826839 const run = await this.ctx.storage.get<Run>("run");
827840 // A person stopped it: g1t has already left the pull request for them.
828841 if (ended === "stopped" && run && STOP_ENDS.has(run.kind)) return;
+29−1
11 import assert from "node:assert/strict";
22 import { test } from "node:test";
33
4−import { answered, describeError, tellStopped, withinTimeCap } from "./lifecycle.ts";
4+import { answered, capReached, describeError, stoppedShort, tellStopped, withinTimeCap } from "./lifecycle.ts";
55
66 const quiet = { waitMs: 0 };
77
8484 assert.equal(withinTimeCap(now - 1000, null, now, 180), false);
8585 assert.equal(withinTimeCap(now - 1000, 0, now, 180), false);
8686 });
87+
88+test("an earlier attempt's time cap does not stop the attempt running now", () => {
89+ const minute = 60_000;
90+ // Desktop release, Linux job: attempt 1 started 12:47:17 with a 60-minute
91+ // cap, so its alarm fired at 13:50:17, four minutes into attempt 3 in the
92+ // same Durable Object.
93+ const attempt3 = Date.UTC(2026, 9, 12, 13, 46, 15);
94+ const alarm = Date.UTC(2026, 9, 12, 13, 50, 17);
95+ assert.equal(capReached(attempt3, 60, alarm), false);
96+ // Its own alarm, at its cap and the alarm's grace, does stop it.
97+ assert.equal(capReached(attempt3, 60, attempt3 + 63 * minute), true);
98+ assert.equal(capReached(attempt3, 60, attempt3 + 60 * minute), true);
99+ assert.equal(capReached(attempt3, 60, attempt3 + 60 * minute - 1), false);
100+});
101+
102+test("a sandbox with no run or no cap is never stopped for time", () => {
103+ const now = Date.now();
104+ assert.equal(capReached(undefined, 60, now), false);
105+ assert.equal(capReached(now - 120 * 60_000, undefined, now), false);
106+ assert.equal(capReached(now - 120 * 60_000, 0, now), false);
107+});
108+
109+test("a sandbox g1t stopped fails even when its exit code says 0", () => {
110+ assert.equal(stoppedShort(0, null), false);
111+ assert.equal(stoppedShort(0, "It ran into its time cap of 60 minutes."), true);
112+ assert.equal(stoppedShort(137, null), true);
113+ assert.equal(stoppedShort(1, "why"), true);
114+});
+22−0
6767 return now < started + (capMinutes * 60 + graceSeconds) * 1000;
6868 }
6969
70+/**
71+ * Whether the run a sandbox holds now has used its whole time cap, so a
72+ * `timeUp` alarm may stop it. A sandbox's Durable Object is named for its
73+ * work (`actions:{job}` and the like), and a re-run of that work starts in
74+ * the same object; an alarm an earlier attempt left must not stop the
75+ * attempt running now, which started later and has its own alarm.
76+ */
77+export function capReached(started: number | null | undefined, capMinutes: number | null | undefined, now: number): boolean {
78+ if (typeof started !== "number" || typeof capMinutes !== "number" || capMinutes <= 0) return false;
79+ return now >= started + capMinutes * 60 * 1000;
80+}
81+
82+/**
83+ * Whether a sandbox's stop ends its work as a failure. A sandbox g1t
84+ * stopped (at its time cap, say) can come back with exit code 0, since the
85+ * container library reports a destroyed container as a clean exit; the
86+ * reason g1t stopped it, not the exit code, says how it ended.
87+ */
88+export function stoppedShort(exitCode: number, why: string | null): boolean {
89+ return exitCode !== 0 || why !== null;
90+}
91+
7092 /** An error as one line, for a log: its message, or what was thrown. */
7193 export function describeError(error: unknown): string {
7294 if (error instanceof Error) return error.message || error.name;

Comments

Sign in to comment.