Sign in

flagon-io/g1t

public

g1t/services/runner/src/model-env.ts

199 lines7,858 bytesCodeBlame

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Agents as a team: lifecycle, merge queue, billing and a new shell1/** The kinds of work a g1t agent does. Each is routed on its own. */
2export type AgentTask = "implement" | "review" | "update" | "plan";
Members can read a private repository's pull request forks3
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier4/**
5 * How capable, and how costly, a model is. g1t's hosted models come in
6 * two: `small` for work a smaller model does as well, `large` for the rest.
7 */
8export type Tier = "small" | "large";
9
Agents as a team: lifecycle, merge queue, billing and a new shell10/** Where one kind of work goes: what people see, and what is sent. */
11export type ModelRoute = {
12 /** The model's public name, e.g. `Claude Sonnet 5.5`. */
13 modelName: string;
14 /** The identifier sent to the provider. */
15 model: string;
16};
Members can read a private repository's pull request forks17
Agents as a team: lifecycle, merge queue, billing and a new shell18/**
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier19 * g1t's routing policy for the runs it pays the model for. Nobody
20 * assigning an agent picks a model; the work decides, here, and the
21 * operator changes it in one place (`AGENT_ROUTING` in wrangler.jsonc).
22 */
23export type AgentRouting = {
24 /** The model behind each tier. */
25 tiers: Record<Tier, ModelRoute>;
26 /**
27 * The tier each kind of work runs on. `change` decides by the change the
28 * work reads: small when it is small and touches nothing sensitive.
29 */
30 tasks: Record<AgentTask, Tier | "change">;
31 /** The largest change `change` sends to the small tier. */
32 smallChange: { files: number; lines: number };
33 /** Labels on the issue behind the work that send `change` to the large tier. */
34 largeLabels: string[];
35};
36
37/** What a change is, as far as routing cares. */
38export type ChangeSize = {
39 files: number;
40 /** Lines added and removed. */
41 lines: number;
42 /**
43 * What it touches that runs, configures or guards things: CI, secrets,
44 * infrastructure, ownership (work's confidence.rs `sensitive`).
45 */
46 sensitive: string[];
47};
48
49/** What g1t knows about one piece of work when it routes it. */
50export type RouteSignals = {
51 /** The change the work reads; null or absent when g1t does not know it. */
52 change?: ChangeSize | null;
53 /** Labels on the issue the work is for. */
54 labels?: string[];
55 /** The last attempt at the same work failed. */
56 retry?: boolean;
57};
58
59/** The routing g1t ships with, for whatever the configuration leaves out. */
60export const DEFAULT_ROUTING: AgentRouting = {
61 tiers: {
62 small: { modelName: "Claude Haiku 4.5", model: "claude-haiku-4-5-20251001" },
63 large: { modelName: "Claude Sonnet 5.5", model: "claude-sonnet-5-5" },
64 },
65 tasks: { implement: "large", review: "change", update: "small", plan: "small" },
66 smallChange: { files: 10, lines: 200 },
67 largeLabels: ["security"],
68};
69
70/**
71 * The routing in `AGENT_ROUTING`, with anything it leaves out taken from
72 * `DEFAULT_ROUTING`. An unset or unreadable value is the default.
73 */
74export function parseRouting(json: string | undefined): AgentRouting {
75 let given: Partial<AgentRouting> = {};
76 try {
77 given = json ? (JSON.parse(json) as Partial<AgentRouting>) : {};
78 } catch {
79 console.log("AGENT_ROUTING is not JSON; using the default routing");
80 }
81 return {
82 tiers: { ...DEFAULT_ROUTING.tiers, ...given.tiers },
83 tasks: { ...DEFAULT_ROUTING.tasks, ...given.tasks },
84 smallChange: { ...DEFAULT_ROUTING.smallChange, ...given.smallChange },
85 largeLabels: given.largeLabels ?? DEFAULT_ROUTING.largeLabels,
86 };
87}
88
89/**
90 * The tier one piece of work runs on: the cheapest that can do it.
91 * Planning and catching up are small; making a change is large; a review
92 * is small for a small change that touches nothing sensitive, and large
93 * for anything else, including a change g1t does not know the size of.
94 * A retry after a failed attempt is always large, so that work the small
95 * tier could not finish goes up rather than failing again the same way.
Agents as a team: lifecycle, merge queue, billing and a new shell96 */
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier97export function chooseTier(task: AgentTask, signals: RouteSignals, routing: AgentRouting = DEFAULT_ROUTING): Tier {
98 if (signals.retry) return "large";
99 const rule = routing.tasks[task] ?? "large";
100 if (rule !== "change") return rule;
101 const change = signals.change;
102 if (!change || change.files === 0) return "large";
103 const large = new Set(routing.largeLabels.map((label) => label.toLowerCase()));
104 if ((signals.labels ?? []).some((label) => large.has(label.toLowerCase()))) return "large";
105 const small =
106 change.sensitive.length === 0 &&
107 change.files <= routing.smallChange.files &&
108 change.lines <= routing.smallChange.lines;
109 return small ? "small" : "large";
110}
Agents as a team: lifecycle, merge queue, billing and a new shell111
Members can read a private repository's pull request forks112/** The settings that decide where model requests go. */
113export type ModelRouting = {
Agents as a team: lifecycle, merge queue, billing and a new shell114 /**
115 * The provider's key. Not needed when the gateway holds it and requests
116 * authenticate to the gateway instead.
117 */
Members can read a private repository's pull request forks118 ANTHROPIC_API_KEY?: string;
119 /** A Cloudflare AI Gateway id; empty sends requests to the provider directly. */
120 AI_GATEWAY_ID: string;
121 CLOUDFLARE_ACCOUNT_ID: string;
Agents as a team: lifecycle, merge queue, billing and a new shell122 /** Authenticates to the gateway, if it requires it. */
Members can read a private repository's pull request forks123 AI_GATEWAY_TOKEN?: string;
124};
125
Agents as a team: lifecycle, merge queue, billing and a new shell126/** What a run is for, attached to each of its requests at the gateway. */
127export type RunTags = { repo: string; pull: number };
128
129/** Whether there is a way to reach a model at all. */
130export function canReachModel(env: ModelRouting): boolean {
131 return Boolean(env.ANTHROPIC_API_KEY || (env.AI_GATEWAY_ID && env.AI_GATEWAY_TOKEN));
132}
133
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier134/**
135 * The model variables of a run on g1t's hosted models: the tier's model
136 * for the work, and the small tier's for the harness's own small tasks.
137 */
138export function tierVars(routing: AgentRouting, tier: Tier): Record<string, string> {
139 const route = routing.tiers[tier];
140 return {
141 ANTHROPIC_MODEL: route.model,
142 // Recorded at the top of the session, so anyone can see what ran.
143 AGENT_MODEL_NAME: route.modelName,
144 ANTHROPIC_DEFAULT_HAIKU_MODEL: routing.tiers.small.model,
145 ANTHROPIC_SMALL_FAST_MODEL: routing.tiers.small.model,
146 };
147}
148
Members can read a private repository's pull request forks149/** Where the sandbox sends model requests, and what it sends with them. */
Agents as a team: lifecycle, merge queue, billing and a new shell150export function modelEnv(
151 env: ModelRouting,
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier152 routing: AgentRouting,
Agents as a team: lifecycle, merge queue, billing and a new shell153 task: AgentTask,
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier154 tier: Tier,
Agents as a team: lifecycle, merge queue, billing and a new shell155 tags: RunTags,
156): Record<string, string> {
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier157 const vars = tierVars(routing, tier);
Agents as a team: lifecycle, merge queue, billing and a new shell158 if (env.ANTHROPIC_API_KEY) vars.ANTHROPIC_API_KEY = env.ANTHROPIC_API_KEY;
159 if (!env.AI_GATEWAY_ID) return vars;
160
161 vars.ANTHROPIC_BASE_URL = `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${env.AI_GATEWAY_ID}/anthropic`;
162 // The gateway logs these with every request, so spend and failures can
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier163 // be read per kind of work, tier, repository and pull request.
164 const headers = [`cf-aig-metadata: ${JSON.stringify({ task, tier, ...tags })}`];
Agents as a team: lifecycle, merge queue, billing and a new shell165 if (env.AI_GATEWAY_TOKEN) {
166 vars.AI_GATEWAY_TOKEN = env.AI_GATEWAY_TOKEN;
167 headers.push(`cf-aig-authorization: Bearer ${env.AI_GATEWAY_TOKEN}`);
168 // With the provider's key stored in the gateway, the sandbox never
169 // holds it. The harness still wants the variable set.
170 vars.ANTHROPIC_API_KEY ??= env.AI_GATEWAY_TOKEN;
Members can read a private repository's pull request forks171 }
Agents as a team: lifecycle, merge queue, billing and a new shell172 vars.ANTHROPIC_CUSTOM_HEADERS = headers.join("\n");
Members can read a private repository's pull request forks173 return vars;
174}
Auto model routing: the cheapest tier that can do each piece of work, a retry goes up a tier, and each run records its tier175
176/** Lines added and removed across a change's files. */
177export function changeSize(files: { additions: number; deletions: number }[], sensitive: string[]): ChangeSize {
178 return {
179 files: files.length,
180 lines: files.reduce((sum, file) => sum + file.additions + file.deletions, 0),
181 sensitive,
182 };
183}
184
185/** A past run of the same work, as the work service lists it. */
186export type PastAttempt = { status: string; halted?: string | null; title?: string | null };
187
188/**
189 * Whether the latest attempt at the same work failed: it failed, or g1t
190 * stopped it at a cap of its guardrails. A person stopping it is not a
191 * failure. `title` narrows it to the same plan, whose runs have no pull
192 * request to tell them apart.
193 */
194export function lastAttemptFailed(newestFirst: PastAttempt[], title?: string): boolean {
195 const last = newestFirst[0];
196 if (!last) return false;
197 if (title !== undefined && (last.title ?? "").trim() !== title.trim()) return false;
198 return last.status === "failed" || (last.status === "stopped" && Boolean(last.halted));
199}