Skip to content
Sign in

flagon-io/g1t

public
0
1,451 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1/**
2 * The context service: the context hub's catalog, search and scorecards.
3 *
4 * - **Catalog.** What a workspace builds and runs, as entities (projects,
5 * apps, APIs, packages, languages, owners, environments, integrations,
6 * docs) and relations between them. Built by itself: on every push to a
7 * project's default branch it reads the project's manifests and docs
8 * whose blobs changed (at most `MAX_READS` files a push) and joins them
9 * with what projects, deployments, identity and integrations know.
10 * Rebuilding is idempotent: entity ids are hashes of what they are.
11 * - **Search.** Docs, catalog entities, issues, pull requests and kept
12 * memory, embedded with Workers AI into a Vectorize index whose rows
13 * carry their workspace, project and whether they are private, so a
14 * query reads only what its reader may. Matching words in D1 answers
15 * when the index cannot, and fills in after it.
16 * - **Backfill.** Once per workspace (and again on request): the catalog
17 * for every project, memory candidates from docs, manifests and the last
18 * merged pull requests, and the search index filled. One queued job per
19 * project, capped and metered.
20 * - **Run context.** The Context section every g1t agent run starts with.
21 *
22 * Reached through service bindings: `POST /rpc/<method>`.
23 */
24
25import {
26 type Backfill,
27 type CaptureItem,
28 type Catalog,
29 type ContextStatus,
30 type Entity,
31 type EntityDetail,
32 type EntityKind,
33 ENTITY_KINDS,
34 type G1tEvent,
35 type Memory,
36 type Project,
37 type ProjectDeploys,
38 type RelationKind,
39 type RunContext,
40 type Scorecard,
41 type SearchHit,
42 type SearchKind,
43 type SearchResult,
44 type ServiceBinding,
45 type User,
46 type Viewer,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look47 ComputeGate,
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put48 billingClient,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look49 embeddingEstimateMicros,
50 localRefusal,
51 currentMovedPath,
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API52 currentWorkspaceSlug,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look53 repoMove,
54 staleMovedPaths,
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API55 deploymentsClient,
56 fail,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look57 granted,
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API58 identityClient,
59 integrationsClient,
60 memoryReviewClient,
61 ok,
62 projectsClient,
63 reposClient,
64 securityClient,
65 staleSlugs,
66 workClient,
67 type Result,
68} from "@g1t/contracts";
69
70import { assemble, authorsOf, integrationEntities, type EntityDraft, type FileRecord, type ProjectInput, type Surroundings } from "./assemble";
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared71import { EXTRACT_VERSION, extract, interesting, type FileFacts } from "./extract";
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API72import { composeRunContext, type ContextNote, type ProjectContext } from "./runcontext";
73import { evaluate } from "./scorecards";
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look74import { allowedKinds, countVisible, indexFilter, memoryReadable, merge, projectReadable, readable, runMemoryReadable, type IndexMeta, type Reader } from "./visibility";
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API75
76type Job =
77 | { type: "backfill_project"; workspace: string; slug: string }
78 | { type: "backfill_memory"; workspace: string };
79
80type Env = {
81 DB: D1Database;
82 AI?: Ai;
83 VECTORS?: Vectorize;
84 JOBS: Queue<Job>;
85 REPOS: ServiceBinding;
86 PROJECTS: ServiceBinding;
87 WORK: ServiceBinding;
88 IDENTITY: ServiceBinding;
89 DEPLOYMENTS: ServiceBinding;
90 INTEGRATIONS: ServiceBinding;
91 SECURITY?: ServiceBinding;
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put92 /** Told the month's embedding cost so far, which it charges once the month is over. */
93 BILLING?: ServiceBinding;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API94};
95
Merge branch 'main' into actions-toolkit-oidc-artifacts96/**
97 * Workers AI's embedding model: 768 dimensions, as the index was made with.
98 * Pinned, not a default staff choose in sudo: vectors from another model
99 * mean nothing beside these, so changing it means a new index, rebuilt.
100 * The catalogue (billing's `gateway_models`) lists it with its price.
101 */
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API102const EMBED_MODEL = "@cf/baai/bge-base-en-v1.5";
103/** What it costs, in millionths of a dollar per token ($0.067 per million). */
104const MICROS_PER_TOKEN = 0.067;
105/** Past this many tokens in a month, a workspace's new text goes unindexed; text search still finds it. */
106const MONTHLY_TOKENS = 20_000_000;
107/** The most text embedded for one row; the model reads 512 tokens. */
108const EMBED_CHARS = 2000;
109const EMBED_BATCH = 50;
110/** The most files read from a repository for one project's rebuild. */
111const MAX_READS = 15;
112/** Of them, workflows. */
113const MAX_WORKFLOW_READS = 3;
114/** The most projects one push rebuilds. */
115const MAX_PROJECTS_PER_PUSH = 5;
116/** A backfill's caps. */
117const BACKFILL_PROJECTS = 50;
118const BACKFILL_PULLS = 20;
119const BACKFILL_ITEMS = 30;
120/** A backfill still marked running after this long is taken to have died. */
121const BACKFILL_STALE_MS = 30 * 60 * 1000;
122const ITEM_CHARS = 4000;
123const SNIPPET_CHARS = 280;
124/** Kinds every project shares rather than owns. */
125const SHARED: Set<EntityKind> = new Set(["owner", "language", "integration"]);
126
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look127/** One compute gate per isolate, so entitlements are kept between calls. */
128let computeGate: ComputeGate | null = null;
129function gateFor(billing: ServiceBinding): ComputeGate {
130 computeGate ??= new ComputeGate(billing);
131 return computeGate;
132}
133
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API134const now = () => new Date().toISOString();
135const month = () => now().slice(0, 7);
136
137function isMember(viewer: Viewer, workspace: string): boolean {
138 return !!viewer?.workspaces?.some((m) => m.slug === workspace.toLowerCase());
139}
140
141async function sha(text: string): Promise<string> {
142 const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text));
143 return [...new Uint8Array(digest)].map((b) => b.toString(16).padStart(2, "0")).join("");
144}
145
146/** An entity's id: the same for the same thing, however often it is rebuilt. */
147export async function entityId(workspace: string, kind: string, key: string): Promise<string> {
148 return `ent_${(await sha(`${workspace}\u0000${kind}\u0000${key.toLowerCase()}`)).slice(0, 24)}`;
149}
150
151function snippet(text: string | null | undefined): string {
152 const line = (text ?? "").replace(/\s+/g, " ").trim();
153 return line.length <= SNIPPET_CHARS ? line : `${line.slice(0, SNIPPET_CHARS - 1)}…`;
154}
155
156type EntityRow = {
157 id: string;
158 workspace: string;
159 kind: EntityKind;
160 key: string;
161 name: string;
162 summary: string | null;
163 project_id: string | null;
164 project: string | null;
165 repo_id: string | null;
166 private: number;
167 data: string;
168 source: string;
169 ref: string | null;
170 updated_at: string;
171};
172
173type ItemRow = {
174 id: string;
175 workspace: string;
176 kind: "doc" | "issue" | "pull";
177 entity_id: string | null;
178 project: string | null;
179 private: number;
180 title: string;
181 text: string;
182 url: string | null;
183 by: string | null;
184 updated_at: string;
185};
186
187function toEntity(row: EntityRow): Entity {
188 let data: Record<string, unknown> = {};
189 try {
190 data = JSON.parse(row.data);
191 } catch {}
192 return {
193 id: row.id,
194 workspace: row.workspace,
195 kind: row.kind,
196 key: row.key,
197 name: row.name,
198 summary: row.summary,
199 project: row.project,
200 private: !!row.private,
201 data,
202 source: row.source,
203 ref: row.ref,
204 updatedAt: row.updated_at,
205 };
206}
207
208/** Where a memory came from, in a few words. */
209function memorySource(memory: Memory): string {
210 const ref = memory.source.reference ?? "";
211 const number = memory.source.number;
212 switch (memory.source.kind) {
213 case "doc":
214 return ref.split(":").slice(2).join(":") || "a doc";
215 case "review":
216 return number ? `review on #${number}` : "a review";
217 case "pr":
218 return number ? `#${number}` : "a merged pull request";
219 case "run":
220 return number ? `agent run on #${number}` : "an agent run";
221 default:
222 return `added by ${memory.createdBy}`;
223 }
224}
225
226/** What one workspace's rebuilds share: asked of other services once. */
227class WorkspaceCache {
228 private members?: Promise<string[]>;
229 private deploys?: Promise<ProjectDeploys[]>;
230 private connections?: Promise<Surroundings["integrations"]>;
231 constructor(
232 private readonly env: Env,
233 readonly workspace: string,
234 readonly actor: User,
235 ) {}
236
237 memberNames(): Promise<string[]> {
238 this.members ??= identityClient(this.env.IDENTITY)
239 .listMembers(this.workspace, this.actor)
240 .then((found) => (found.ok ? found.value.map((member) => member.username) : []))
241 .catch(() => []);
242 return this.members;
243 }
244
245 deployments(): Promise<ProjectDeploys[]> {
246 this.deploys ??= deploymentsClient(this.env.DEPLOYMENTS)
247 .overview(this.workspace, this.actor)
248 .then((found) => (found.ok ? found.value : []))
249 .catch(() => []);
250 return this.deploys;
251 }
252
253 integrations(): Promise<Surroundings["integrations"]> {
254 this.connections ??= integrationsClient(this.env.INTEGRATIONS)
255 .list(this.workspace, this.actor)
256 .then((found) =>
257 found.ok
258 ? found.value
259 .filter((connection) => connection.kind !== "models")
260 .map((connection) => ({
261 id: connection.id,
262 provider: connection.provider,
263 name: connection.name,
264 kind: connection.kind,
265 repo: connection.config.repo ?? null,
266 }))
267 : [],
268 )
269 .catch(() => []);
270 return this.connections;
271 }
272}
273
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared274type ScanStats = { entities: number; candidates: number; kept: number; indexed: number; pruned?: number };
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API275
276class Context {
277 constructor(private readonly env: Env) {}
278
279 private get db() {
280 return this.env.DB;
281 }
282
283 private async workspaceActor(slug: string): Promise<User | null> {
284 const workspace = await identityClient(this.env.IDENTITY).getWorkspace(slug);
285 if (!workspace) return null;
286 return {
287 id: workspace.id,
288 username: workspace.slug,
289 kind: "workspace",
290 verified: true,
291 workspaces: [{ slug: workspace.slug, role: "member" }],
292 };
293 }
294
295 // ---- Search index --------------------------------------------------------
296
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look297 /**
298 * Whether semantic search is open to a workspace: embeddings are compute,
299 * so only on a paid plan or the trial (`localRefusal`). Text search
300 * answers for everyone else. Closed when billing cannot say, which
301 * leaves text search; open where there is no billing service at all.
302 */
303 private async semanticOpen(workspace: string): Promise<boolean> {
304 if (!this.env.BILLING) return true;
305 const ent = await gateFor(this.env.BILLING).entitlements(workspace);
306 return ent != null && localRefusal(ent, "embedding", false) == null;
307 }
308
309 /**
310 * Embeds and stores rows in the search index, within the workspace's
311 * monthly allowance, reserving what it costs with billing first and
312 * settling what it did. Never throws.
313 */
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API314 private async index(workspace: string, rows: { id: string; text: string; meta: IndexMeta }[]): Promise<number> {
315 const { AI, VECTORS } = this.env;
316 if (!AI || !VECTORS || rows.length === 0) return 0;
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look317 if (!(await this.semanticOpen(workspace))) return 0;
318 let reservation: string | null = null;
319 let spentTokens = 0;
320 if (this.env.BILLING) {
321 const estimate = rows.reduce((sum, row) => sum + Math.ceil(Math.min(row.text.length, EMBED_CHARS) / 4), 0);
322 const admitted = await gateFor(this.env.BILLING).admit({
323 workspace,
324 repo: { namespace: workspace, name: rows[0].meta.project ?? "" },
325 public: rows.every((row) => !row.meta.private),
326 kind: "embedding",
327 estimateMicros: embeddingEstimateMicros(estimate),
328 });
329 if (!admitted.ok) {
330 console.log("embeddings not started for", workspace, admitted.code, admitted.message);
331 return 0;
332 }
333 reservation = admitted.reservation?.id ?? null;
334 }
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API335 try {
336 const used = await this.db
337 .prepare("SELECT tokens FROM usage WHERE workspace = ? AND month = ?")
338 .bind(workspace, month())
339 .first<{ tokens: number }>();
340 if ((used?.tokens ?? 0) >= MONTHLY_TOKENS) return 0;
341 let stored = 0;
342 for (let at = 0; at < rows.length; at += EMBED_BATCH) {
343 const batch = rows.slice(at, at + EMBED_BATCH);
344 const texts = batch.map((row) => row.text.slice(0, EMBED_CHARS));
345 const embedded = (await AI.run(EMBED_MODEL, { text: texts })) as { data?: number[][] };
346 const vectors = (embedded.data ?? []).map((values, i) => ({
347 id: batch[i].id,
348 values,
349 metadata: batch[i].meta as unknown as Record<string, VectorizeVectorMetadata>,
350 }));
351 if (vectors.length) await VECTORS.upsert(vectors);
352 stored += vectors.length;
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put353 const tokens = (rows: { text: string }[]) => rows.reduce((sum, row) => sum + Math.ceil(row.text.length / 4), 0);
354 const all = texts.map((text) => ({ text }));
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look355 spentTokens += tokens(all);
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put356 await this.meter(workspace, tokens(all), tokens(all.filter((_, i) => batch[i].meta.private)));
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API357 }
358 return stored;
359 } catch (error) {
360 console.error("could not index", rows.length, "rows for", workspace, error);
361 return 0;
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look362 } finally {
363 if (reservation && this.env.BILLING) {
364 await gateFor(this.env.BILLING).settle(reservation, embeddingEstimateMicros(spentTokens));
365 }
366 }
367 }
368
369 /** Drops a repository's rows kept under any of `workspaces`, and their index entries. */
370 private async forgetRepo(repoId: string, workspaces: string[]): Promise<void> {
371 const marks = workspaces.map(() => "?").join(", ");
372 const items = await this.db
373 .prepare(`SELECT id FROM items WHERE repo_id = ? AND workspace IN (${marks})`)
374 .bind(repoId, ...workspaces)
375 .all<{ id: string }>();
376 for (let i = 0; i < items.results.length; i += 100) {
377 await this.unindex(items.results.slice(i, i + 100).map((row) => row.id));
378 }
379 const projects = `SELECT project_id FROM entities WHERE repo_id = ? AND workspace IN (${marks}) AND project_id IS NOT NULL`;
380 await this.db.batch([
381 this.db.prepare(`DELETE FROM relations WHERE project_id IN (${projects})`).bind(repoId, ...workspaces),
382 this.db.prepare(`DELETE FROM files WHERE project_id IN (${projects})`).bind(repoId, ...workspaces),
383 this.db.prepare(`DELETE FROM items WHERE repo_id = ? AND workspace IN (${marks})`).bind(repoId, ...workspaces),
384 this.db.prepare(`DELETE FROM scans WHERE repo_id = ? AND workspace IN (${marks})`).bind(repoId, ...workspaces),
385 this.db.prepare(`DELETE FROM entities WHERE repo_id = ? AND workspace IN (${marks})`).bind(repoId, ...workspaces),
386 ]);
387 }
388
389 /** A deleted workspace's hub: everything but its usage, which billing has already read. */
390 private async forgetWorkspace(slug: string): Promise<void> {
391 const items = await this.db.prepare("SELECT id FROM items WHERE workspace = ?").bind(slug).all<{ id: string }>();
392 for (let i = 0; i < items.results.length; i += 100) {
393 await this.unindex(items.results.slice(i, i + 100).map((row) => row.id));
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API394 }
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look395 await this.db.batch(
396 ["items", "relations", "entities", "scans", "backfills"].map((table) =>
397 this.db.prepare(`DELETE FROM ${table} WHERE workspace = ?`).bind(slug),
398 ),
399 );
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API400 }
401
402 private async unindex(ids: string[]): Promise<void> {
403 if (!this.env.VECTORS || ids.length === 0) return;
404 try {
405 await this.env.VECTORS.deleteByIds(ids);
406 } catch (error) {
407 console.error("could not take", ids.length, "rows out of the index", error);
408 }
409 }
410
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put411 /**
412 * Records what embedding cost. Of it, `billableTokens` are private
413 * text's: public repositories' text and searches are never charged.
414 * Billing is told the month's billable total, which it charges at cost
415 * plus its margin once the month is over (its `embedding_tokens` meter).
416 * A failure to tell billing only delays it: the next call sends the
417 * whole month again.
418 */
419 private async meter(workspace: string, tokens: number, billableTokens = 0): Promise<void> {
420 const total = await this.db
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API421 .prepare(
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put422 `INSERT INTO usage (workspace, month, tokens, cost_micros, billable_micros) VALUES (?1, ?2, ?3, ?4, ?5)
423 ON CONFLICT (workspace, month) DO UPDATE SET
424 tokens = tokens + ?3, cost_micros = cost_micros + ?4, billable_micros = billable_micros + ?5
425 RETURNING billable_micros`,
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API426 )
Team plan, an open-source pool, monthly trials and honest metering; the sidebar for everyone; a workspace that stays put427 .bind(workspace, month(), tokens, Math.ceil(tokens * MICROS_PER_TOKEN), billableTokens * MICROS_PER_TOKEN)
428 .first<{ billable_micros: number }>();
429 if (this.env.BILLING && total && billableTokens > 0) {
430 await billingClient(this.env.BILLING)
431 .notePending(workspace, "context", Math.ceil(total.billable_micros))
432 .catch((error) => console.error("could not tell billing what embedding cost", workspace, error));
433 }
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API434 }
435
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look436 /**
437 * A query's embedding, for semantic search. Too small to reserve one by
438 * one (a few hundred tokens, a fraction of a cent): it needs the plan or
439 * the trial, as indexing does, and is metered with the month's usage.
440 */
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API441 private async embedQuery(workspace: string, query: string): Promise<number[] | null> {
442 if (!this.env.AI || !this.env.VECTORS) return null;
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look443 if (!(await this.semanticOpen(workspace))) return null;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API444 try {
445 const embedded = (await this.env.AI.run(EMBED_MODEL, { text: [query.slice(0, EMBED_CHARS)] })) as { data?: number[][] };
446 await this.meter(workspace, Math.ceil(query.length / 4));
447 return embedded.data?.[0] ?? null;
448 } catch (error) {
449 console.error("could not embed a query", error);
450 return null;
451 }
452 }
453
454 // ---- Building the catalog --------------------------------------------------
455
456 /** The files of a project worth reading, with their blobs, found in a few tree reads. */
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared457 private async candidates(project: Project, actor: User, ref: string): Promise<{ files: { path: string; hash: string }[]; siblings: string[]; head: string | null; partial: boolean } | null> {
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API458 if (project.source.kind !== "hosted") return null;
459 const repos = reposClient(this.env.REPOS);
460 const { repo, rootDir: root } = project.source;
461 const at = (path: string) => [root, path].filter(Boolean).join("/");
462 const top = await repos.tree(repo, actor, ref, root);
463 if (!top.ok) return null;
464 const files: { path: string; hash: string }[] = [];
465 const siblings = top.value.entries.map((entry) => entry.name);
466 const blobs = (entries: { name: string; hash: string; kind: string }[], dir: string) => {
467 for (const entry of entries) {
468 if (entry.kind !== "blob" && entry.kind !== "exec") continue;
469 const path = dir ? `${dir}/${entry.name}` : entry.name;
470 if (interesting(path)) files.push({ path, hash: entry.hash });
471 }
472 };
473 blobs(top.value.entries, "");
474 const dirs = new Set(top.value.entries.filter((entry) => entry.kind === "tree").map((entry) => entry.name));
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared475 // A folder that could not be listed leaves the list partial.
476 let partial = false;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API477 const look = async (dir: string) => {
478 const found = await repos.tree(repo, actor, ref, at(dir)).catch(() => null);
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared479 if (!found?.ok) partial = true;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API480 return found?.ok ? found.value.entries : [];
481 };
482 for (const dir of ["docs", "doc", "runbooks"]) if (dirs.has(dir)) blobs(await look(dir), dir);
483 for (const dir of [".g1t", ".github"]) {
484 if (!dirs.has(dir)) continue;
485 const inside = await look(dir);
486 blobs(inside, dir);
487 if (inside.some((entry) => entry.name === "workflows" && entry.kind === "tree")) blobs(await look(`${dir}/workflows`), `${dir}/workflows`);
488 }
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared489 return { files, siblings, head: top.value.head?.hash ?? null, partial };
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API490 }
491
492 /**
493 * Builds one project's place in the catalog from its default branch (or
494 * `commit`), reading only files whose blobs changed unless `force`.
495 * Indexes what changed and sends what its docs say to memory.
496 */
497 async scan(project: Project, cache: WorkspaceCache, commit: string | null, force: boolean): Promise<ScanStats> {
498 const stats: ScanStats = { entities: 0, candidates: 0, kept: 0, indexed: 0 };
499 if (project.source.kind !== "hosted") return stats;
500 const { repo, repoId, rootDir, defaultBranch } = project.source;
501 const ref = commit ?? defaultBranch;
502 const found = await this.candidates(project, cache.actor, ref);
503 if (!found) return stats;
504 const workspace = project.workspace;
505
506 // What each file says: stored for unchanged blobs, read for the rest.
507 const stored = new Map(
508 (
509 await this.db.prepare("SELECT path, hash, facts FROM files WHERE project_id = ?").bind(project.id).all<{ path: string; hash: string; facts: string }>()
510 ).results.map((row) => [row.path, row]),
511 );
512 const files: FileRecord[] = [];
513 const changed: string[] = [];
514 const writes: D1PreparedStatement[] = [];
515 let reads = 0;
516 let workflowReads = 0;
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared517 // Whether every file is known as this version of extract reads it: only
518 // then are the doc candidates it no longer suggests let go.
519 let complete = !found.partial;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API520 const repos = reposClient(this.env.REPOS);
521 for (const file of found.files) {
522 const before = stored.get(file.path);
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared523 const kept = before ? (JSON.parse(before.facts) as FileFacts) : null;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API524 const workflow = file.path.includes("/workflows/");
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared525 // Facts an older extract read are read again, as a changed file is.
526 const fresh = before && before.hash === file.hash && kept?.version === EXTRACT_VERSION && !force;
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API527 const canRead = reads < MAX_READS && (!workflow || workflowReads < MAX_WORKFLOW_READS);
528 if (fresh || !canRead) {
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared529 if (!fresh) complete = false;
530 if (kept) files.push({ path: file.path, facts: kept });
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API531 continue;
532 }
533 reads++;
534 if (workflow) workflowReads++;
535 const blob = await repos.blob(repo, cache.actor, found.head ?? ref, [rootDir, file.path].filter(Boolean).join("/")).catch(() => null);
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared536 if (!blob?.ok || blob.value.text == null) {
537 complete = false;
538 continue;
539 }
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API540 const facts = extract(file.path, blob.value.text, { project: project.name, siblings: found.siblings });
541 files.push({ path: file.path, facts });
542 changed.push(file.path);
543 writes.push(
544 this.db
545 .prepare(
546 `INSERT INTO files (project_id, path, hash, facts, read_at) VALUES (?, ?, ?, ?, ?)
547 ON CONFLICT (project_id, path) DO UPDATE SET hash = excluded.hash, facts = excluded.facts, read_at = excluded.read_at`,
548 )
549 .bind(project.id, file.path, file.hash, JSON.stringify(facts), now()),
550 );
551 }
552 const present = new Set(found.files.map((file) => file.path));
553 for (const path of stored.keys()) {
554 if (!present.has(path)) writes.push(this.db.prepare("DELETE FROM files WHERE project_id = ? AND path = ?").bind(project.id, path));
555 }
556
557 // What g1t knows about it besides its files.
558 const [members, deploys, integrations, graph, log] = await Promise.all([
559 cache.memberNames(),
560 cache.deployments(),
561 cache.integrations(),
562 projectsClient(this.env.PROJECTS).graph(project.id).catch(() => ({ dependsOn: [], usedBy: [] })),
563 repos.log(repo, cache.actor, found.head ?? ref, 100).catch(() => null),
564 ]);
565 const deploy = deploys.find((d) => d.slug === project.slug) ?? null;
566 const around: Surroundings = {
567 owners: authorsOf(log?.ok ? log.value : [], members),
568 dependsOn: graph.dependsOn.map((dep) => ({ slug: dep.slug, as: dep.as })),
569 deploy: deploy
570 ? {
571 enabled: deploy.enabled,
572 production: deploy.production ? { url: deploy.production.url, commit: deploy.production.commit, deployedAt: deploy.production.deployedAt } : null,
573 previews: deploy.previews,
574 latest: deploy.latest ? { status: deploy.latest.status, kind: deploy.latest.kind, error: deploy.latest.error, createdAt: deploy.latest.createdAt } : null,
575 }
576 : null,
577 integrations,
578 };
579 const input: ProjectInput = {
580 id: project.id,
581 workspace,
582 slug: project.slug,
583 name: project.name,
584 description: project.description,
585 private: project.private,
586 repoId,
587 repo,
588 rootDir,
589 defaultBranch,
590 };
591 const built = assemble(input, files, around);
592 const drafts: EntityDraft[] = [...built.entities, ...integrationEntities(integrations)];
593
594 // Entities: upserted; the project's own that it no longer has, removed.
595 const previous = new Map(
596 (
597 await this.db
598 .prepare("SELECT id, name, summary FROM entities WHERE workspace = ? AND (project_id = ? OR project_id IS NULL)")
599 .bind(workspace, project.id)
600 .all<{ id: string; name: string; summary: string | null }>()
601 ).results.map((row) => [row.id, row]),
602 );
603 const ids: string[] = [];
604 const toIndex: { id: string; text: string; meta: IndexMeta }[] = [];
605 const at = now();
606 for (const draft of drafts) {
607 const id = await entityId(workspace, draft.kind, draft.key);
608 ids.push(id);
609 const shared = SHARED.has(draft.kind);
610 const source = draft.kind === "app" || draft.kind === "environment" ? "deployments" : draft.kind === "integration" ? "integrations" : "scan";
611 writes.push(
612 this.db
613 .prepare(
614 `INSERT INTO entities (id, workspace, kind, key, name, summary, project_id, project, repo_id, private, data, source, ref, updated_at)
615 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
616 ON CONFLICT (workspace, kind, key) DO UPDATE SET name = excluded.name, summary = excluded.summary,
617 project_id = excluded.project_id, project = excluded.project, repo_id = excluded.repo_id, private = excluded.private,
618 data = excluded.data, source = excluded.source, ref = excluded.ref, updated_at = excluded.updated_at`,
619 )
620 .bind(
621 id,
622 workspace,
623 draft.kind,
624 draft.key,
625 draft.name,
626 draft.summary,
627 shared ? null : project.id,
628 shared ? null : project.slug,
629 shared ? null : repoId,
630 shared ? 0 : project.private ? 1 : 0,
631 JSON.stringify(draft.data),
632 source,
633 draft.ref,
634 at,
635 ),
636 );
637 const before = previous.get(id);
638 if (force || !before || before.name !== draft.name || before.summary !== draft.summary) {
639 toIndex.push({
640 id,
641 text: `${draft.kind} ${draft.name}. ${draft.summary ?? ""}`,
642 meta: {
643 workspace,
644 kind: draft.kind,
645 project: shared ? "" : project.slug,
646 private: shared ? false : project.private,
647 title: draft.name,
648 snippet: snippet(draft.summary),
649 url: draft.ref ?? "",
650 source: "catalog",
651 by: "",
652 at,
653 },
654 });
655 }
656 }
657 const gone = (
658 await this.db
659 .prepare("SELECT id FROM entities WHERE project_id = ? AND id NOT IN (SELECT value FROM json_each(?))")
660 .bind(project.id, JSON.stringify(ids))
661 .all<{ id: string }>()
662 ).results.map((row) => row.id);
663 if (gone.length) {
664 writes.push(this.db.prepare("DELETE FROM entities WHERE id IN (SELECT value FROM json_each(?))").bind(JSON.stringify(gone)));
665 writes.push(this.db.prepare("DELETE FROM items WHERE entity_id IN (SELECT value FROM json_each(?))").bind(JSON.stringify(gone)));
666 }
667
668 // Relations: the project's own, replaced whole.
669 writes.push(this.db.prepare("DELETE FROM relations WHERE project_id = ?").bind(project.id));
670 for (const relation of built.relations) {
671 const [from, to] = await Promise.all([entityId(workspace, relation.from.kind, relation.from.key), entityId(workspace, relation.to.kind, relation.to.key)]);
672 writes.push(
673 this.db
674 .prepare("INSERT OR REPLACE INTO relations (workspace, from_id, kind, to_id, project_id, updated_at) VALUES (?, ?, ?, ?, ?, ?)")
675 .bind(workspace, from, relation.kind, to, project.id, at),
676 );
677 }
678
679 // Docs that changed: their pieces, for search.
680 const repoPath = `/${repo.namespace}/${repo.name}`;
681 const chunkIds: string[] = [];
682 for (const file of files) {
683 const doc = file.facts.doc;
684 if (!doc || (!changed.includes(file.path) && !force)) continue;
685 const docId = await entityId(workspace, "doc", `${project.slug}:${doc.path}`);
686 writes.push(this.db.prepare("DELETE FROM items WHERE entity_id = ?").bind(docId));
687 const url = `${repoPath}/blob/${defaultBranch}/${[rootDir, doc.path].filter(Boolean).join("/")}`;
688 doc.chunks.forEach((text, i) => {
689 const id = `${docId}:${i}`;
690 chunkIds.push(id);
691 writes.push(
692 this.db
693 .prepare(
694 `INSERT OR REPLACE INTO items (id, workspace, kind, entity_id, project_id, project, repo_id, private, title, text, url, by, updated_at)
695 VALUES (?, ?, 'doc', ?, ?, ?, ?, ?, ?, ?, ?, NULL, ?)`,
696 )
697 .bind(id, workspace, docId, project.id, project.slug, repoId, project.private ? 1 : 0, `${doc.title} (${doc.path})`, text, url, at),
698 );
699 toIndex.push({
700 id,
701 text: `${doc.title}\n${text}`,
702 meta: { workspace, kind: "doc", project: project.slug, private: project.private, title: `${doc.title} (${doc.path})`, snippet: snippet(text), url, source: doc.path, by: "", at },
703 });
704 });
705 }
706 writes.push(
707 this.db
708 .prepare(
709 `INSERT INTO scans (project_id, workspace, repo_id, "commit", tests, scanned_at) VALUES (?, ?, ?, ?, ?, ?)
710 ON CONFLICT (project_id) DO UPDATE SET workspace = excluded.workspace, "commit" = excluded."commit", tests = excluded.tests, scanned_at = excluded.scanned_at`,
711 )
712 .bind(project.id, workspace, repoId, found.head, built.tests ? 1 : 0, at),
713 );
714 for (let i = 0; i < writes.length; i += 50) await this.db.batch(writes.slice(i, i + 50));
715 await this.unindex(gone);
716 stats.entities = drafts.length;
717 stats.indexed = await this.index(workspace, toIndex);
718
719 // What changed files say worth remembering, as memory candidates.
720 const items: CaptureItem[] = built.hints
721 .filter((hint) => force || changed.includes(hint.path))
722 .map((hint) => ({
723 scope: "project",
724 repoId,
725 kind: hint.kind,
726 text: hint.text,
727 confidence: hint.confidence,
728 source: "doc",
729 reference: `doc:${repoId}:${[rootDir, hint.path].filter(Boolean).join("/")}`,
730 evidence: hint.evidence,
731 }));
732 for (let i = 0; i < items.length; i += 50) {
733 const captured = await memoryReviewClient(this.env.WORK)
734 .captureMemories(workspace, items.slice(i, i + 50), "g1t")
735 .catch(() => null);
736 stats.candidates += captured?.added ?? 0;
737 stats.kept += captured?.kept ?? 0;
738 }
Merge memory from docs: only docs on how to work here, whole sentences, no near-duplicates, stale doc suggestions cleared739 // Candidates from this project's docs that are still waiting but that
740 // its docs, as read now, no longer suggest: a line since changed, or one
741 // the rules for what is worth remembering leave out. Kept and dismissed
742 // memory is never touched.
743 if (complete) {
744 const pruned = await memoryReviewClient(this.env.WORK)
745 .pruneDocCandidates(workspace, repoId, built.hints.map((hint) => hint.text))
746 .catch(() => null);
747 stats.pruned = pruned?.removed ?? 0;
748 }
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API749 return stats;
750 }
751
752 // ---- Issues, pull requests and memory in search ------------------------
753
754 private async repoOf(repoId: string): Promise<{ namespace: string; name: string } | null> {
755 const response = await this.env.REPOS.fetch("https://repos/rpc/path_by_id", {
756 method: "POST",
757 headers: { "content-type": "application/json" },
758 body: JSON.stringify({ id: repoId }),
759 });
760 return response.ok ? ((await response.json()) as { namespace: string; name: string } | null) : null;
761 }
762
763 /** Stores and indexes issues and pull requests as search rows. */
764 private async putItems(
765 workspace: string,
766 project: Project | null,
767 repoId: string,
768 rows: { kind: "issue" | "pull"; number: number; title: string; body: string | null; by: string | null; updatedAt: string; url: string; private: boolean }[],
769 ): Promise<number> {
770 const writes: D1PreparedStatement[] = [];
771 const toIndex: { id: string; text: string; meta: IndexMeta }[] = [];
772 for (const row of rows) {
773 const id = `${row.kind}:${repoId}#${row.number}`;
774 const title = `#${row.number} ${row.title}`;
775 const text = (row.body ?? "").slice(0, ITEM_CHARS);
776 writes.push(
777 this.db
778 .prepare(
779 `INSERT OR REPLACE INTO items (id, workspace, kind, entity_id, project_id, project, repo_id, private, title, text, url, by, updated_at)
780 VALUES (?, ?, ?, NULL, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
781 )
782 .bind(id, workspace, row.kind, project?.id ?? null, project?.slug ?? null, repoId, row.private ? 1 : 0, title, text, row.url, row.by, row.updatedAt),
783 );
784 toIndex.push({
785 id,
786 text: `${row.title}\n${text}`,
787 meta: { workspace, kind: row.kind, project: project?.slug ?? "", private: row.private, title, snippet: snippet(text || row.title), url: row.url, source: row.kind === "issue" ? "issue" : "pull request", by: row.by ?? "", at: row.updatedAt },
788 });
789 }
790 for (let i = 0; i < writes.length; i += 50) await this.db.batch(writes.slice(i, i + 50));
791 return this.index(workspace, toIndex);
792 }
793
794 private async indexIssueOrPull(kind: "issue" | "pull", repoId: string, number: number): Promise<void> {
795 const path = await this.repoOf(repoId);
796 if (!path) return;
797 const workspace = path.namespace.toLowerCase();
798 const actor = await this.workspaceActor(workspace);
799 if (!actor) return;
800 const [repo, projects] = await Promise.all([reposClient(this.env.REPOS).get(path, actor), projectsClient(this.env.PROJECTS).byRepo(repoId)]);
801 if (!repo.ok || repo.value.forkOf) return;
802 const work = workClient(this.env.WORK);
803 const base = `/${path.namespace}/${path.name}`;
804 if (kind === "issue") {
805 const found = await work.getIssue(path, number, actor);
806 if (!found.ok) return;
807 const issue = found.value.issue;
808 await this.putItems(workspace, projects[0] ?? null, repoId, [
809 { kind, number, title: issue.title, body: issue.body, by: issue.author.username, updatedAt: issue.updatedAt, url: `${base}/issues/${number}`, private: repo.value.isPrivate },
810 ]);
811 } else {
812 const found = await work.getPull(path, number, actor);
813 if (!found.ok) return;
814 const pull = found.value.pull as { title: string; body: string | null; agent: string; updatedAt?: string; author?: { username: string } };
815 await this.putItems(workspace, projects[0] ?? null, repoId, [
816 { kind, number, title: pull.title, body: pull.body, by: pull.author?.username ?? pull.agent, updatedAt: pull.updatedAt ?? now(), url: `${base}/pull/${number}`, private: repo.value.isPrivate },
817 ]);
818 }
819 }
820
821 /** A memory in search while it is kept, and out of it otherwise. */
822 private async indexMemories(workspace: string, memories: Memory[]): Promise<number> {
823 const kept = memories.filter((memory) => (memory.status ?? "kept") === "kept");
824 await this.unindex(memories.filter((memory) => !kept.includes(memory)).map((memory) => `memory:${memory.id}`));
825 const projects = await projectsClient(this.env.PROJECTS)
826 .list(workspace, (await this.workspaceActor(workspace)) ?? null)
827 .then((found) => (found.ok ? found.value : []))
828 .catch(() => [] as Project[]);
829 const slugOf = (memory: Memory) =>
830 memory.scope === "project" && memory.repo
831 ? (projects.find(
832 (project) =>
833 project.source.kind === "hosted" &&
834 project.primary &&
835 project.source.repo.namespace.toLowerCase() === memory.repo!.namespace.toLowerCase() &&
836 project.source.repo.name.toLowerCase() === memory.repo!.name.toLowerCase(),
837 )?.slug ?? "")
838 : "";
839 return this.index(
840 workspace,
841 kept.map((memory) => ({
842 id: `memory:${memory.id}`,
843 text: `${memory.kind}: ${memory.text}`,
844 meta: {
845 workspace,
846 kind: "memory",
847 project: slugOf(memory),
848 // Memory is for members, whatever its project.
849 private: true,
850 title: `${memory.kind[0].toUpperCase()}${memory.kind.slice(1)}${memory.scope === "workspace" ? " (workspace)" : ""}`,
851 snippet: snippet(memory.text),
852 url: memory.repo ? `/${memory.repo.namespace}/${memory.repo.name}/memory` : `/${workspace}/-/memory`,
853 source: memorySource(memory),
854 by: memory.createdBy,
855 at: memory.updatedAt,
856 },
857 })),
858 );
859 }
860
861 // ---- Events --------------------------------------------------------------
862
863 async onEvent(event: G1tEvent): Promise<void> {
864 switch (event.type) {
865 case "git.push": {
866 if (!event.data.defaultBranch) return;
867 const projects = (await projectsClient(this.env.PROJECTS).byRepo(event.data.repoId)).slice(0, MAX_PROJECTS_PER_PUSH);
868 if (projects.length === 0) return;
869 const actor = await this.workspaceActor(projects[0].workspace);
870 if (!actor) return;
871 const cache = new WorkspaceCache(this.env, projects[0].workspace, actor);
872 for (const project of projects) await this.scan(project, cache, event.data.after, false);
873 return;
874 }
875 case "issue.opened":
876 case "issue.updated":
877 case "issue.closed":
878 return this.indexIssueOrPull("issue", event.data.repoId, event.data.number);
879 case "pull.ready":
880 case "pull.merged":
881 return this.indexIssueOrPull("pull", event.data.repoId, event.data.number);
882 case "memory.changed": {
883 const { memoryId, workspace, status } = event.data;
884 if (status !== "kept") return this.unindex([`memory:${memoryId}`]);
885 const memories = await memoryReviewClient(this.env.WORK).memoriesById(workspace, [memoryId]);
886 await this.indexMemories(workspace, memories);
887 return;
888 }
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look889 case "repo.transferred":
890 case "repo.renamed": {
891 // What the hub knew about the repository under its old path goes,
892 // index included (its rows carry the path: a transfer's old
893 // workspace, a rename's old name); the workspace it is in now is
894 // built again, which reads it where and as it is now. Nothing here
895 // is the only copy.
896 const move = repoMove(event)!;
897 const current = await currentMovedPath(this.env.REPOS, move);
898 const stale = staleMovedPaths(move, current);
899 if (stale.length === 0) return;
900 const workspace = current.split("/")[0]!.toLowerCase();
901 await this.forgetRepo(move.repoId, [...new Set(stale.map((path) => path.split("/")[0]!.toLowerCase()))]);
902 const actor = await this.workspaceActor(workspace);
903 if (actor) await this.backfill({ actor, workspace });
904 return;
905 }
906 case "repo.deleted":
907 case "repo.purged": {
908 // Deleted, it is hidden: what the hub knew of it goes, index
909 // included, so no search or agent finds it. A restore builds it
910 // again; a purge finds nothing left.
911 await this.forgetRepo(event.data.repoId, [event.data.namespace.toLowerCase()]);
912 return;
913 }
914 case "repo.restored": {
915 const workspace = event.data.namespace.toLowerCase();
916 const actor = await this.workspaceActor(workspace);
917 if (actor) await this.backfill({ actor, workspace });
918 return;
919 }
920 case "workspace.deleted": {
921 await this.forgetWorkspace(event.data.slug);
922 return;
923 }
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API924 case "workspace.renamed": {
925 const current = await currentWorkspaceSlug(this.env.IDENTITY, event.data);
926 for (const old of staleSlugs(event.data, current)) {
927 await this.db.batch(
928 ["entities", "relations", "items", "scans", "usage"].map((table) =>
929 this.db.prepare(`UPDATE OR IGNORE ${table} SET workspace = ? WHERE workspace = ?`).bind(current, old),
930 ),
931 );
932 await this.db.prepare("DELETE FROM backfills WHERE workspace = ?").bind(old).run();
933 }
934 // The index's rows carry the slug: build them again under the new one.
935 const actor = await this.workspaceActor(current);
936 if (actor) await this.backfill({ actor, workspace: current });
937 return;
938 }
939 default:
940 return;
941 }
942 }
943
944 // ---- Backfill ------------------------------------------------------------
945
946 private async backfillRow(workspace: string): Promise<Backfill | null> {
947 const row = await this.db.prepare("SELECT * FROM backfills WHERE workspace = ?").bind(workspace).first<Record<string, unknown>>();
948 if (!row) return null;
949 return {
950 workspace,
951 status: row.status as Backfill["status"],
952 by: String(row.by),
953 projects: Number(row.projects),
954 done: Number(row.done),
955 entities: Number(row.entities),
956 candidates: Number(row.candidates),
957 kept: Number(row.kept),
958 indexed: Number(row.indexed),
959 error: (row.error as string | null) ?? null,
960 startedAt: String(row.started_at),
961 finishedAt: (row.finished_at as string | null) ?? null,
962 };
963 }
964
965 async backfill(a: { actor: User; workspace: string }): Promise<Result<Backfill>> {
966 const workspace = a.workspace.toLowerCase();
967 if (!isMember(a.actor, workspace)) return fail("forbidden", "Only members can rebuild a workspace's context.");
968 const running = await this.backfillRow(workspace);
969 if (running?.status === "running" && Date.now() - Date.parse(running.startedAt) < BACKFILL_STALE_MS) return ok(running);
970 const actor = (await this.workspaceActor(workspace)) ?? a.actor;
971 const listed = await projectsClient(this.env.PROJECTS).list(workspace, actor);
972 if (!listed.ok) return listed;
973 const projects = listed.value.filter((project) => project.source.kind === "hosted").slice(0, BACKFILL_PROJECTS);
974 const at = now();
975 await this.db
976 .prepare(
977 `INSERT OR REPLACE INTO backfills (workspace, status, by, projects, done, entities, candidates, kept, indexed, error, started_at, finished_at)
978 VALUES (?, ?, ?, ?, 0, 0, 0, 0, 0, NULL, ?, ?)`,
979 )
980 .bind(workspace, projects.length ? "running" : "done", a.actor.username, projects.length, at, projects.length ? null : at)
981 .run();
982 const jobs: { body: Job }[] = [
983 ...projects.map((project) => ({ body: { type: "backfill_project", workspace, slug: project.slug } as Job })),
984 { body: { type: "backfill_memory", workspace } },
985 ];
986 for (let i = 0; i < jobs.length; i += 100) await this.env.JOBS.sendBatch(jobs.slice(i, i + 100));
987 return ok((await this.backfillRow(workspace))!);
988 }
989
990 async runJob(job: Job): Promise<void> {
991 const actor = await this.workspaceActor(job.workspace);
992 if (!actor) return;
993 if (job.type === "backfill_memory") {
994 const memories = await memoryReviewClient(this.env.WORK).searchMemories(job.workspace, null, { limit: 100 });
995 const indexed = await this.indexMemories(job.workspace, memories);
996 await this.db.prepare("UPDATE backfills SET indexed = indexed + ? WHERE workspace = ?").bind(indexed, job.workspace).run();
997 return;
998 }
999 const stats: ScanStats = { entities: 0, candidates: 0, kept: 0, indexed: 0 };
1000 let error: string | null = null;
1001 try {
1002 const found = await projectsClient(this.env.PROJECTS).get(job.workspace, job.slug, actor);
1003 if (found.ok && found.value.source.kind === "hosted") {
1004 const project = found.value;
1005 const source = project.source as Extract<Project["source"], { kind: "hosted" }>;
1006 const cache = new WorkspaceCache(this.env, job.workspace, actor);
1007 Object.assign(stats, await this.scan(project, cache, null, true));
1008 if (project.primary) {
1009 // Decisions from merged pull requests, and people's corrections in their reviews.
1010 const seeded = await memoryReviewClient(this.env.WORK).seedFromPulls(source.repoId, BACKFILL_PULLS).catch(() => null);
1011 stats.candidates += seeded?.added ?? 0;
1012 stats.kept += seeded?.kept ?? 0;
1013 const work = workClient(this.env.WORK);
1014 const base = `/${source.repo.namespace}/${source.repo.name}`;
1015 const [issues, pulls] = await Promise.all([
1016 work.listIssues(source.repo, actor).catch(() => null),
1017 work.listPulls(source.repo, actor, "closed").catch(() => null),
1018 ]);
1019 stats.indexed += await this.putItems(job.workspace, project, source.repoId, [
1020 ...(issues?.ok ? issues.value.slice(0, BACKFILL_ITEMS) : []).map((issue) => ({
1021 kind: "issue" as const,
1022 number: issue.number,
1023 title: issue.title,
1024 body: issue.body,
1025 by: issue.author.username,
1026 updatedAt: issue.updatedAt,
1027 url: `${base}/issues/${issue.number}`,
1028 private: project.private,
1029 })),
1030 ...(pulls?.ok ? pulls.value.filter((pull) => pull.status === "merged").slice(0, BACKFILL_ITEMS) : []).map((pull) => ({
1031 kind: "pull" as const,
1032 number: pull.number,
1033 title: pull.title,
1034 body: pull.body,
1035 by: (pull as { author?: { username: string } }).author?.username ?? pull.agent,
1036 updatedAt: pull.mergedAt ?? now(),
1037 url: `${base}/pull/${pull.number}`,
1038 private: project.private,
1039 })),
1040 ]);
1041 }
1042 }
1043 } catch (caught) {
1044 error = String(caught).slice(0, 300);
1045 console.error("backfill of", job.workspace, job.slug, "failed", caught);
1046 }
1047 await this.db
1048 .prepare(
1049 `UPDATE backfills SET done = done + 1, entities = entities + ?, candidates = candidates + ?, kept = kept + ?, indexed = indexed + ?,
1050 error = COALESCE(?, error),
1051 status = CASE WHEN done + 1 >= projects THEN 'done' ELSE status END,
1052 finished_at = CASE WHEN done + 1 >= projects THEN ? ELSE finished_at END
1053 WHERE workspace = ?`,
1054 )
1055 .bind(stats.entities, stats.candidates, stats.kept, stats.indexed, error, now(), job.workspace)
1056 .run();
1057 }
1058
1059 // ---- Reading -------------------------------------------------------------
1060
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1061 /**
1062 * Who is reading, and what of the workspace they may see. Someone who can
1063 * read every repository in it (an owner, a member while its base
1064 * permission is Read or more, its own token) sees everything; anyone else
1065 * sees the projects the projects service lists for them, which are those
1066 * whose repositories they can read.
1067 */
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1068 private async reader(workspace: string, viewer: Viewer): Promise<Reader> {
1069 const member = isMember(viewer, workspace);
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1070 const full = member && !!viewer && granted(viewer, { id: "", namespace: workspace, isPrivate: true }) != null;
1071 if (full) return { workspace, member, full, visible: new Set() };
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1072 const listed = await projectsClient(this.env.PROJECTS).list(workspace, viewer).catch(() => null);
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1073 const projects = listed?.ok ? listed.value : [];
1074 return {
1075 workspace,
1076 member,
1077 full,
1078 visible: new Set(projects.map((p) => p.slug)),
1079 privateVisible: projects.some((p) => p.private),
1080 repos: new Set(
1081 projects.flatMap((p) => (p.source.kind === "hosted" ? [`${p.source.repo.namespace}/${p.source.repo.name}`.toLowerCase()] : [])),
1082 ),
1083 };
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1084 }
1085
1086 private visibleRow(row: { private: number; project: string | null }, reader: Reader): boolean {
1087 return readable({ workspace: reader.workspace, kind: "entity", project: row.project ?? "", private: !!row.private }, reader);
1088 }
1089
1090 async status(a: { workspace: string; viewer: Viewer }): Promise<Result<ContextStatus>> {
1091 const workspace = a.workspace.toLowerCase();
1092 if (!isMember(a.viewer, workspace)) return fail("not_found", "There is no such workspace.");
1093 let backfill = await this.backfillRow(workspace);
1094 // The hub is never empty for long: a workspace's first look starts it.
1095 if (!backfill) {
1096 const actor = await this.workspaceActor(workspace);
1097 if (actor) {
1098 const started = await this.backfill({ actor: { ...actor, username: "g1t" }, workspace });
1099 backfill = started.ok ? started.value : null;
1100 }
1101 }
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1102 // Counted over what the viewer may read: a member whose base permission
1103 // is None counts only the projects they were given.
1104 const reader = await this.reader(workspace, a.viewer);
1105 const [counted, usage] = await Promise.all([
1106 this.db
1107 .prepare("SELECT kind, project, private, COUNT(*) AS n FROM entities WHERE workspace = ? GROUP BY kind, project, private")
1108 .bind(workspace)
1109 .all<{ kind: EntityKind; project: string | null; private: number; n: number }>(),
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1110 this.db.prepare("SELECT tokens, cost_micros FROM usage WHERE workspace = ? AND month = ?").bind(workspace, month()).first<{ tokens: number; cost_micros: number }>(),
1111 ]);
1112 return ok({
1113 backfill,
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1114 counts: countVisible(counted.results, reader),
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1115 usage: { month: month(), tokens: usage?.tokens ?? 0, costMicros: usage?.cost_micros ?? 0 },
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1116 // Embeddings are compute: a paid plan or the trial (semanticOpen).
1117 semantic: !!(this.env.AI && this.env.VECTORS) && (await this.semanticOpen(workspace)),
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1118 });
1119 }
1120
1121 async catalog(a: { workspace: string; viewer: Viewer; kind?: EntityKind | null; project?: string | null }): Promise<Result<Catalog>> {
1122 const workspace = a.workspace.toLowerCase();
1123 const reader = await this.reader(workspace, a.viewer);
1124 let sql = "SELECT * FROM entities WHERE workspace = ?";
1125 const params: unknown[] = [workspace];
1126 if (a.kind) {
1127 sql += " AND kind = ?";
1128 params.push(a.kind);
1129 }
1130 if (a.project) {
1131 sql += " AND (project = ? OR project IS NULL)";
1132 params.push(a.project.toLowerCase());
1133 }
1134 sql += " ORDER BY kind, name COLLATE NOCASE LIMIT 2000";
1135 const rows = (await this.db.prepare(sql).bind(...params).all<EntityRow>()).results.filter((row) => this.visibleRow(row, reader));
1136 const ids = new Set(rows.map((row) => row.id));
1137 const relations = (
1138 await this.db.prepare("SELECT from_id, kind, to_id FROM relations WHERE workspace = ? LIMIT 10000").bind(workspace).all<{ from_id: string; kind: RelationKind; to_id: string }>()
1139 ).results
1140 .filter((row) => ids.has(row.from_id) && ids.has(row.to_id))
1141 .map((row) => ({ from: row.from_id, kind: row.kind, to: row.to_id }));
1142 const built = await this.db.prepare("SELECT MAX(scanned_at) AS at FROM scans WHERE workspace = ?").bind(workspace).first<{ at: string | null }>();
1143 return ok({ entities: rows.map(toEntity), relations, builtAt: built?.at ?? null });
1144 }
1145
1146 async entity(a: { workspace: string; viewer: Viewer; kind: EntityKind; id: string }): Promise<Result<EntityDetail>> {
1147 const workspace = a.workspace.toLowerCase();
1148 if (!ENTITY_KINDS.includes(a.kind)) return fail("invalid", `kind is one of ${ENTITY_KINDS.join(", ")}.`);
1149 const reader = await this.reader(workspace, a.viewer);
1150 const row =
1151 (await this.db.prepare("SELECT * FROM entities WHERE workspace = ? AND id = ?").bind(workspace, a.id).first<EntityRow>()) ??
1152 (await this.db.prepare("SELECT * FROM entities WHERE workspace = ? AND kind = ? AND key = ? COLLATE NOCASE").bind(workspace, a.kind, a.id).first<EntityRow>());
1153 if (!row || row.kind !== a.kind || !this.visibleRow(row, reader)) return fail("not_found", `There is no such ${a.kind} in ${workspace}'s catalog.`);
1154 const [out, into] = await Promise.all([
1155 this.db
1156 .prepare("SELECT r.kind AS rel, e.* FROM relations r JOIN entities e ON e.id = r.to_id WHERE r.from_id = ? LIMIT 200")
1157 .bind(row.id)
1158 .all<EntityRow & { rel: RelationKind }>(),
1159 this.db
1160 .prepare("SELECT r.kind AS rel, e.* FROM relations r JOIN entities e ON e.id = r.from_id WHERE r.to_id = ? LIMIT 200")
1161 .bind(row.id)
1162 .all<EntityRow & { rel: RelationKind }>(),
1163 ]);
1164 const relations = [
1165 ...out.results.filter((r) => this.visibleRow(r, reader)).map((r) => ({ kind: r.rel, direction: "out" as const, entity: toEntity(r) })),
1166 ...into.results.filter((r) => this.visibleRow(r, reader)).map((r) => ({ kind: r.rel, direction: "in" as const, entity: toEntity(r) })),
1167 ];
1168 return ok({ entity: toEntity(row), relations });
1169 }
1170
1171 async search(a: { workspace: string; viewer: Viewer; query: string; project?: string | null; kinds?: SearchKind[] | null; limit?: number | null }): Promise<Result<SearchResult>> {
1172 const workspace = a.workspace.toLowerCase();
1173 const query = (a.query ?? "").trim().slice(0, 500);
1174 if (!query) return fail("invalid", "Say what to search for.");
1175 const reader = await this.reader(workspace, a.viewer);
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1176 if (!reader.full && !reader.member && reader.visible.size === 0) return ok({ query, hits: [], mode: "text" });
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1177 const limit = Math.min(Math.max(a.limit ?? 20, 1), 50);
1178 const kinds = allowedKinds(reader, a.kinds);
1179 const wants = (kind: SearchKind) => !kinds || kinds.includes(kind);
1180 const project = a.project?.toLowerCase() || null;
1181
1182 // Semantic: the index, filtered to what this reader may see.
1183 let semantic: SearchHit[] = [];
1184 let mode: SearchResult["mode"] = "text";
1185 const vector = await this.embedQuery(workspace, query);
1186 if (vector && this.env.VECTORS) {
1187 try {
1188 const found = await this.env.VECTORS.query(vector, { topK: 20, returnMetadata: "all", filter: indexFilter(reader, { project, kinds }) as VectorizeVectorMetadataFilter });
1189 mode = "semantic";
1190 semantic = found.matches
1191 .map((match) => ({ id: match.id, score: match.score, meta: match.metadata as unknown as IndexMeta }))
1192 .filter((match) => match.meta && readable(match.meta, reader))
1193 .map((match) => ({
1194 kind: match.meta.kind as SearchKind,
1195 id: match.id.startsWith("memory:") ? match.id.slice(7) : match.id,
1196 title: match.meta.title,
1197 snippet: match.meta.snippet,
1198 project: match.meta.project || null,
1199 url: match.meta.url || null,
1200 score: match.score,
1201 source: match.meta.source,
1202 by: match.meta.by || null,
1203 updatedAt: match.meta.at || null,
1204 }));
1205 // Memory changes after it is indexed: show only what is still kept.
1206 const memoryIds = semantic.filter((hit) => hit.kind === "memory").map((hit) => hit.id);
1207 if (memoryIds.length) {
1208 const kept = new Set(
1209 (await memoryReviewClient(this.env.WORK).memoriesById(workspace, memoryIds))
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1210 .filter((memory) => (memory.status ?? "kept") === "kept" && memoryReadable(memory.repo, reader))
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1211 .map((memory) => memory.id),
1212 );
1213 semantic = semantic.filter((hit) => hit.kind !== "memory" || kept.has(hit.id));
1214 }
1215 } catch (error) {
1216 console.error("semantic search failed; matching words instead", error);
1217 }
1218 }
1219
1220 // Text: every word, in the catalog, docs, issues and pull requests, and memory.
1221 const words = query.toLowerCase().split(/\s+/).filter(Boolean).slice(0, 6);
1222 const like = (columns: string) => words.map(() => `(${columns}) LIKE ?`).join(" AND ");
1223 const patterns = words.map((word) => `%${word.replace(/[%_]/g, "")}%`);
1224 const text: SearchHit[] = [];
1225 const entityKinds = ENTITY_KINDS.filter(wants);
1226 if (entityKinds.length) {
1227 const rows = await this.db
1228 .prepare(
1229 `SELECT * FROM entities WHERE workspace = ? AND kind IN (SELECT value FROM json_each(?)) ${project ? "AND (project = ? OR project IS NULL)" : ""}
1230 AND ${like("lower(name || ' ' || COALESCE(summary, ''))")} LIMIT 30`,
1231 )
1232 .bind(workspace, JSON.stringify(entityKinds), ...(project ? [project] : []), ...patterns)
1233 .all<EntityRow>();
1234 for (const row of rows.results.filter((r) => this.visibleRow(r, reader))) {
1235 text.push({ kind: row.kind, id: row.id, title: row.name, snippet: snippet(row.summary), project: row.project, url: row.ref, score: 0.3, source: "catalog", by: null, updatedAt: row.updated_at });
1236 }
1237 }
1238 const itemKinds = (["doc", "issue", "pull"] as const).filter(wants);
1239 if (itemKinds.length) {
1240 const rows = await this.db
1241 .prepare(
1242 `SELECT * FROM items WHERE workspace = ? AND kind IN (SELECT value FROM json_each(?)) ${project ? "AND project = ?" : ""}
1243 AND ${like("lower(title || ' ' || text)")} ORDER BY updated_at DESC LIMIT 30`,
1244 )
1245 .bind(workspace, JSON.stringify(itemKinds), ...(project ? [project] : []), ...patterns)
1246 .all<ItemRow>();
1247 for (const row of rows.results.filter((r) => this.visibleRow(r, reader))) {
1248 text.push({ kind: row.kind, id: row.id, title: row.title, snippet: snippet(row.text), project: row.project, url: row.url, score: 0.2, source: row.kind === "doc" ? "doc" : row.kind, by: row.by, updatedAt: row.updated_at });
1249 }
1250 }
1251 if (reader.member && wants("memory")) {
1252 const memories = await memoryReviewClient(this.env.WORK).searchMemories(workspace, query, { limit: 10 }).catch(() => [] as Memory[]);
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1253 for (const memory of memories.filter((m) => memoryReadable(m.repo, reader))) {
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1254 text.push({
1255 kind: "memory",
1256 id: memory.id,
1257 title: `${memory.kind[0].toUpperCase()}${memory.kind.slice(1)}${memory.scope === "workspace" ? " (workspace)" : ""}`,
1258 snippet: snippet(memory.text),
1259 project: null,
1260 url: memory.repo ? `/${memory.repo.namespace}/${memory.repo.name}/memory` : `/${workspace}/-/memory`,
1261 score: memory.pinned ? 0.35 : 0.25,
1262 source: memorySource(memory),
1263 by: memory.createdBy,
1264 updatedAt: memory.updatedAt,
1265 });
1266 }
1267 }
1268 return ok({ query, hits: merge(semantic, text, limit), mode });
1269 }
1270
1271 async scorecards(a: { workspace: string; viewer: Viewer }): Promise<Result<Scorecard[]>> {
1272 const workspace = a.workspace.toLowerCase();
1273 if (!isMember(a.viewer, workspace)) return fail("forbidden", "Scorecards are for members of the workspace.");
1274 const listed = await projectsClient(this.env.PROJECTS).list(workspace, a.viewer);
1275 if (!listed.ok) return listed;
1276 const [entities, scans, deploys, security] = await Promise.all([
1277 this.db
1278 .prepare("SELECT * FROM entities WHERE workspace = ? AND kind IN ('project', 'doc')")
1279 .bind(workspace)
1280 .all<EntityRow>(),
1281 this.db.prepare("SELECT project_id, tests FROM scans WHERE workspace = ?").bind(workspace).all<{ project_id: string; tests: number }>(),
1282 deploymentsClient(this.env.DEPLOYMENTS)
1283 .overview(workspace, a.viewer)
1284 .then((found) => (found.ok ? found.value : []))
1285 .catch(() => [] as ProjectDeploys[]),
1286 this.env.SECURITY
1287 ? securityClient(this.env.SECURITY)
1288 .workspace(workspace, a.viewer)
1289 .then((found) => (found.ok ? found.value : null))
1290 .catch(() => null)
1291 : Promise.resolve(null),
1292 ]);
1293 const tests = new Map(scans.results.map((row) => [row.project_id, !!row.tests]));
1294 const cards: Scorecard[] = [];
1295 for (const project of listed.value) {
1296 if (project.source.kind !== "hosted") continue;
1297 const own = entities.results.filter((row) => row.project_id === project.id);
1298 const entry = own.find((row) => row.kind === "project");
1299 const data = entry ? toEntity(entry).data : {};
1300 const deploy = deploys.find((d) => d.slug === project.slug);
1301 const repoId = project.source.repoId;
1302 const findings = security?.find((repo) => repo.repoId === repoId);
1303 const rules = evaluate({
1304 name: project.name,
1305 owners: Array.isArray(data.owners) ? (data.owners as string[]) : [],
1306 docs: own.filter((row) => row.kind === "doc").map((row) => String(toEntity(row).data.path ?? "")),
1307 tests: tests.get(project.id) ?? false,
1308 testCommand: Array.isArray(data.testCommands) && data.testCommands.length ? String(data.testCommands[0]) : null,
1309 deploy: deploy
1310 ? {
1311 enabled: deploy.enabled,
1312 production: deploy.production ? { url: deploy.production.url } : null,
1313 latest: deploy.latest ? { kind: deploy.latest.kind, status: deploy.latest.status, error: deploy.latest.error } : null,
1314 }
1315 : null,
1316 secretFindings: security ? (findings?.secrets ?? 0) : null,
1317 });
1318 const applies = rules.filter((rule) => rule.status !== "na");
1319 cards.push({
1320 project: project.slug,
1321 name: project.name,
1322 repo: project.source.repo,
1323 passed: applies.filter((rule) => rule.status === "pass").length,
1324 total: applies.length,
1325 rules,
1326 });
1327 }
1328 return ok(cards);
1329 }
1330
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1331 /**
1332 * The Context section for an agent starting work, holding only what the
1333 * person it acts for (`requester`) may read: an outside collaborator's run
1334 * is told the project's memory, never the workspace's, and only the
1335 * projects around it they can read. No requester is the workspace's own
1336 * step. Never fails a run: on any trouble, nothing.
1337 */
1338 async runContext(a: { repoId: string; task?: string; budget?: number; requester?: Viewer }): Promise<RunContext> {
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1339 try {
1340 const projects = (await projectsClient(this.env.PROJECTS).byRepo(a.repoId)).slice(0, 2);
1341 if (projects.length === 0) return { text: null, sources: [] };
1342 const workspace = projects[0].workspace;
1343 const actor = await this.workspaceActor(workspace);
1344 if (!actor) return { text: null, sources: [] };
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1345 const reader = a.requester ? await this.reader(workspace, a.requester) : null;
1346 const home = projects.find((project) => project.source.kind === "hosted");
1347 const repoKey = home?.source.kind === "hosted" ? `${home.source.repo.namespace}/${home.source.repo.name}` : "";
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1348 const cache = new WorkspaceCache(this.env, workspace, actor);
1349 const deploys = await cache.deployments();
1350 const live = new Map(deploys.map((d) => [d.slug, d]));
1351 const contexts: ProjectContext[] = [];
1352 for (const project of projects) {
1353 if (project.source.kind !== "hosted") continue;
1354 const rows = (await this.db.prepare("SELECT * FROM entities WHERE workspace = ? AND project_id = ?").bind(workspace, project.id).all<EntityRow>()).results.map(toEntity);
1355 const entry = rows.find((row) => row.kind === "project");
1356 const graph = await projectsClient(this.env.PROJECTS).graph(project.id).catch(() => ({ dependsOn: [], usedBy: [] }));
1357 const deploy = live.get(project.slug);
1358 contexts.push({
1359 slug: project.slug,
1360 name: project.name,
1361 repo: `${project.source.repo.namespace}/${project.source.repo.name}`,
1362 rootDir: project.source.rootDir,
1363 languages: Array.isArray(entry?.data.languages) ? (entry!.data.languages as string[]) : [],
1364 packages: rows.filter((row) => row.kind === "package").map((row) => row.name).slice(0, 5),
1365 testCommands: Array.isArray(entry?.data.testCommands) ? (entry!.data.testCommands as string[]) : [],
1366 owners: Array.isArray(entry?.data.owners) ? (entry!.data.owners as string[]) : [],
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1367 dependsOn: graph.dependsOn
1368 .filter((dep) => projectReadable(dep.slug, reader))
1369 .map((dep) => ({ slug: dep.slug, as: dep.as, url: live.get(dep.slug)?.production?.url ?? null })),
1370 usedBy: graph.usedBy.filter((dep) => projectReadable(dep.slug, reader)).map((dep) => ({ slug: dep.slug })),
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1371 environments: deploy?.enabled
1372 ? [{ name: "Production", url: deploy.production?.url ?? null, status: deploy.latest?.kind === "production" ? deploy.latest.status : deploy.production ? "ready" : null }]
1373 : [],
1374 docs: rows.filter((row) => row.kind === "doc").map((row) => String(row.data.path ?? row.name)),
1375 });
1376 }
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1377 // Workspace memory (no project) only for a run its members may be told it.
1378 const slugs = new Set([...(reader && !reader.member ? [] : [""]), ...projects.map((project) => project.slug)]);
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1379 const review = memoryReviewClient(this.env.WORK);
1380 // The memories closest to the task, less the pinned ones every run already has.
1381 let memories: ContextNote[] = [];
1382 const task = (a.task ?? "").trim();
1383 const vector = task ? await this.embedQuery(workspace, task.slice(0, 1500)) : null;
1384 if (vector && this.env.VECTORS) {
1385 const found = await this.env.VECTORS.query(vector, { topK: 20, returnMetadata: "all", filter: { workspace, kind: "memory" } }).catch(() => null);
1386 const ids = (found?.matches ?? [])
1387 .filter((match) => slugs.has(String((match.metadata as { project?: string } | undefined)?.project ?? "")) && match.score >= 0.5)
1388 .map((match) => match.id.slice("memory:".length));
1389 if (ids.length) {
1390 const byId = new Map((await review.memoriesById(workspace, ids)).map((memory) => [memory.id, memory]));
1391 memories = ids
1392 .map((id) => byId.get(id))
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1393 .filter((memory): memory is Memory => !!memory && (memory.status ?? "kept") === "kept" && !memory.pinned && runMemoryReadable(memory, reader, repoKey))
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1394 .slice(0, 6)
1395 .map((memory) => ({ id: memory.id, kind: memory.kind, text: memory.text, source: memorySource(memory) }));
1396 }
1397 }
1398 const shown = new Set(memories.map((note) => note.id));
1399 const decisions = (await review.searchMemories(workspace, null, { repoIds: [a.repoId], limit: 100 }).catch(() => [] as Memory[]))
Invite-only launch: sign in with GitHub, repository access and lifecycle, many emails, a new look1400 .filter((memory) => memory.kind === "decision" && !memory.pinned && !shown.has(memory.id) && runMemoryReadable(memory, reader, repoKey))
Agents get guardrails, run credentials, an audit log, a context hub, repository instructions and mentions; security upkeep; snake_case API1401 .sort((x, y) => y.updatedAt.localeCompare(x.updatedAt))
1402 .slice(0, 3)
1403 .map((memory) => ({ id: memory.id, kind: memory.kind, text: memory.text, source: memorySource(memory) }));
1404 return composeRunContext({ projects: contexts, memories, decisions, budget: Math.min(Math.max(a.budget ?? 4000, 500), 12_000) });
1405 } catch (error) {
1406 console.error("no run context for", a.repoId, error);
1407 return { text: null, sources: [] };
1408 }
1409 }
1410}
1411
1412export default {
1413 async fetch(request: Request, env: Env): Promise<Response> {
1414 const match = new URL(request.url).pathname.match(/^\/rpc\/([a-z_]+)$/);
1415 if (request.method !== "POST" || !match) return new Response("Not found\n", { status: 404 });
1416 const service = new Context(env);
1417 const args = (await request.json().catch(() => ({}))) as any;
1418 switch (match[1]) {
1419 case "catalog":
1420 return Response.json(await service.catalog(args));
1421 case "entity":
1422 return Response.json(await service.entity(args));
1423 case "search":
1424 return Response.json(await service.search(args));
1425 case "scorecards":
1426 return Response.json(await service.scorecards(args));
1427 case "backfill":
1428 return Response.json(await service.backfill(args));
1429 case "status":
1430 return Response.json(await service.status(args));
1431 case "run_context":
1432 return Response.json(await service.runContext(args));
1433 default:
1434 return new Response("Unknown method\n", { status: 404 });
1435 }
1436 },
1437
1438 async queue(batch: MessageBatch<G1tEvent | Job>, env: Env): Promise<void> {
1439 const service = new Context(env);
1440 for (const message of batch.messages) {
1441 try {
1442 if (batch.queue === "g1t-context-jobs") await service.runJob(message.body as Job);
1443 else await service.onEvent(message.body as G1tEvent);
1444 message.ack();
1445 } catch (error) {
1446 console.error("context could not handle", (message.body as { type?: string }).type, error);
1447 message.retry();
1448 }
1449 }
1450 },
1451} satisfies ExportedHandler<Env, G1tEvent | Job>;

This file's history is long; its oldest lines are credited to the oldest commit read.