Skip to content
Sign up

flagon-io/g1t

public
0
273 linesCodeBlameRaw

Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.

Docs: a workspace knowledge base people and agents write together1/**
2 * What a page's room saves to D1 after a burst of edits: the Markdown
3 * rendition, the search index, backlinks, history, and telling people
4 * newly mentioned in the page. Run by the room (src/room.ts), which owns
5 * the live document; nothing here reads the document itself.
6 */
The docs service answers every artifacts call: docs can be made, listed, shared, moved, trashed, restored, searched, versioned and edited live in their own rooms, agents read, write and recall them only where their person and everyone in the conversation can, and folio events go out on the bus, while Docs' pages keep working as before.7import { newId, notifyClient, type DocCitation, type DocVersionKind, type FeedNotification, type FolioKind, type FolioVersionKind, type ServiceBinding } from "@g1t/contracts";
Docs: a workspace knowledge base people and agents write together8
Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store9import { publishDocEvent } from "./events.ts";
The docs service answers every artifacts call: docs can be made, listed, shared, moved, trashed, restored, searched, versioned and edited live in their own rooms, agents read, write and recall them only where their person and everyone in the conversation can, and folio events go out on the bus, while Docs' pages keep working as before.10import { fileStore, type FileStoreEnv } from "./files.ts";
11import { workspaceReadable } from "./folios/access-store.ts";
12import { publishFolioEvent } from "./folios/events.ts";
13import type { Rendition } from "./kinds/types.ts";
Docs: a workspace knowledge base people and agents write together14import { excerpt, searchText } from "./markdown.ts";
15import { linkedPageIds, pageSlug } from "./slugs.ts";
16
17/** How long edits gather into one version before the next starts. */
18export const VERSION_EVERY_MS = 10 * 60 * 1000;
19/** The largest Yjs state a version keeps; past it, only its Markdown. */
20const MAX_VERSION_STATE = 1_500_000;
21
Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store22export type SaveEnv = { DB: D1Database; NOTIFY?: ServiceBinding; EVENTS?: ServiceBinding };
Docs: a workspace knowledge base people and agents write together23
24export type Save = {
25 page_id: string;
26 markdown: string;
Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store27 /** Code the document cites (src/citations.ts `bodyCitations`). */
28 citations: DocCitation[];
Docs: a workspace knowledge base people and agents write together29 /** The editors since the last save, member keys, last one last. */
30 editors: string[];
31 /** People mentioned in the document now (usernames, lowercased). */
32 mentioned: string[];
33 /** The usernames of the people who made these edits, lowercased: nobody is told they mentioned themselves. */
34 editor_names: string[];
35 /** The whole document, for a version. */
36 state: Uint8Array;
37 /** A version to record now, whatever the time since the last. */
38 version: { kind: DocVersionKind; note: string | null; authors: string[] } | null;
39 /** Editors since the last version, for a timed one. */
40 pending_authors: string[];
41 /** When the last version was recorded (ms), or 0. */
42 last_version_at: number;
43 /** The workspace's slug as the room last heard it, for links in notifications. */
44 workspace_slug: string | null;
45};
46
47type PageRow = { id: string; workspace_id: string; space_id: string; title: string; mentioned: string; markdown: string; slug: string; archived_at: string | null };
48
Docs index by meaning: passages of every page and project doc, embedded on save and recalled for agents; hybrid search for people49/** Saves; returns whether a version was recorded, and its id, and whether the Markdown changed (so the room indexes it again, src/indexer.ts). */
50export async function save(env: SaveEnv, input: Save, now = new Date()): Promise<{ version_id: string | null; changed: boolean }> {
Docs: a workspace knowledge base people and agents write together51 const at = now.toISOString();
52 const page = await env.DB.prepare(
53 "SELECT p.id, p.workspace_id, p.space_id, p.title, p.mentioned, p.markdown, p.archived_at, s.slug AS slug FROM pages p JOIN spaces s ON s.id = p.space_id WHERE p.id = ?",
54 )
55 .bind(input.page_id)
56 .first<PageRow>();
Docs index by meaning: passages of every page and project doc, embedded on save and recalled for agents; hybrid search for people57 if (!page) return { version_id: null, changed: false };
Docs: a workspace knowledge base people and agents write together58 const changed = page.markdown !== input.markdown;
59 const last = input.editors[input.editors.length - 1] ?? null;
60 const statements: D1PreparedStatement[] = [];
61 if (changed || last) {
62 statements.push(
63 env.DB.prepare("UPDATE pages SET markdown = ?, updated_at = ?, updated_by = COALESCE(?, updated_by) WHERE id = ?").bind(input.markdown, at, last, page.id),
64 );
65 }
66 if (changed) {
67 statements.push(env.DB.prepare("DELETE FROM pages_fts WHERE page_id = ?").bind(page.id));
68 statements.push(env.DB.prepare("INSERT INTO pages_fts (page_id, title, body) VALUES (?, ?, ?)").bind(page.id, page.title, searchText(input.markdown)));
69 statements.push(env.DB.prepare("DELETE FROM page_links WHERE from_page = ?").bind(page.id));
70 for (const to of linkedPageIds(input.markdown).filter((id) => id !== page.id).slice(0, 200)) {
71 statements.push(env.DB.prepare("INSERT OR IGNORE INTO page_links (from_page, to_page) VALUES (?, ?)").bind(page.id, to));
72 }
Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store73 // What it cites, from its text; the header's own stay.
74 statements.push(env.DB.prepare("DELETE FROM citations WHERE page_id = ? AND source = 'body'").bind(page.id));
75 for (const c of input.citations) {
76 statements.push(
77 env.DB.prepare("INSERT OR IGNORE INTO citations (page_id, repo, path, kind, label, ref, source) VALUES (?, ?, ?, ?, ?, ?, 'body')").bind(page.id, c.repo, c.path, c.kind, c.label ?? "", c.ref),
78 );
79 }
Docs: a workspace knowledge base people and agents write together80 }
81 // A version: asked for (an agent's edit, a suggestion, a restore), or
82 // the first save after enough time since the last one.
83 let versionId: string | null = null;
84 const timed = input.pending_authors.length > 0 && now.getTime() - input.last_version_at >= VERSION_EVERY_MS;
85 if (input.version || (timed && changed)) {
86 versionId = newId("ver", now.getTime());
87 const authors = input.version?.authors.length ? input.version.authors : input.pending_authors;
88 const state = input.state.byteLength <= MAX_VERSION_STATE ? input.state : null;
89 statements.push(
90 env.DB.prepare("INSERT INTO page_versions (id, page_id, created_at, kind, authors, note, markdown, state) VALUES (?, ?, ?, ?, ?, ?, ?, ?)").bind(
91 versionId,
92 page.id,
93 at,
94 input.version?.kind ?? "edit",
95 JSON.stringify([...new Set(authors)]),
96 input.version?.note ?? null,
97 input.markdown,
98 state,
99 ),
100 );
101 }
102 // People mentioned for the first time.
103 let told: string[] = [];
104 try {
105 told = JSON.parse(page.mentioned) as string[];
106 } catch {
107 told = [];
108 }
109 const editors = new Set(input.editor_names);
110 const fresh = input.mentioned.filter((name) => !told.includes(name) && !editors.has(name));
111 if (fresh.length || input.mentioned.length !== told.length) {
112 statements.push(env.DB.prepare("UPDATE pages SET mentioned = ? WHERE id = ?").bind(JSON.stringify([...new Set([...told.filter((id) => input.mentioned.includes(id)), ...fresh])]), page.id));
113 }
114 if (statements.length) await env.DB.batch(statements);
Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store115 // A version is what the rest of g1t hears of: at most every ten minutes of editing, and each agent edit, suggestion and restore.
116 const kind = input.version?.kind ?? "edit";
117 if (versionId && kind !== "created" && input.workspace_slug && !page.archived_at) {
118 await publishDocEvent(
119 env.EVENTS,
120 "doc.page.updated",
121 {
122 workspace: input.workspace_slug,
123 workspaceId: page.workspace_id,
124 pageId: page.id,
125 spaceId: page.space_id,
126 title: page.title,
127 path: `/${input.workspace_slug}/-/docs/${page.slug}/${pageSlug(page.title, page.id)}`,
128 versionId,
129 kind,
130 authors: [...new Set(input.version?.authors.length ? input.version.authors : input.pending_authors)],
131 },
132 last,
133 );
134 }
Docs: a workspace knowledge base people and agents write together135 if (fresh.length && env.NOTIFY && !page.archived_at) {
136 const slug = input.workspace_slug;
137 if (slug) {
138 const href = `/${slug}/-/docs/${page.slug}/${pageSlug(page.title, page.id)}`;
139 const notification = (username: string): FeedNotification => ({
140 id: `doc-mention:${page.id}:${username}`,
141 kind: "mention",
142 workspace: slug,
143 title: `You were mentioned in ${page.title || "Untitled"}`,
144 body: excerpt(input.markdown, 140),
145 href,
146 actor: { kind: last?.startsWith("agent:") ? "agent" : "user", id: last?.slice(last.indexOf(":") + 1) ?? "", name: "Docs" },
147 created_at: at,
148 });
149 const notify = notifyClient(env.NOTIFY);
150 await Promise.all(fresh.map((name) => notify.notify({ username: name }, notification(name)).catch(() => undefined)));
151 }
152 }
Docs index by meaning: passages of every page and project doc, embedded on save and recalled for agents; hybrid search for people153 return { version_id: versionId, changed };
Docs: a workspace knowledge base people and agents write together154}
The docs service answers every artifacts call: docs can be made, listed, shared, moved, trashed, restored, searched, versioned and edited live in their own rooms, agents read, write and recall them only where their person and everyone in the conversation can, and folio events go out on the bus, while Docs' pages keep working as before.155
156// ── Folios (Artifacts mode) ─────────────────────────────────────────────
157
158export type SaveFolioEnv = SaveEnv & FileStoreEnv;
159
160export type SaveFolio = {
161 folio_id: string;
162 /** What the kind rendered from the document (src/kinds/types.ts). */
163 rendition: Rendition;
164 /** The editors since the last save, member keys, last one last. */
165 editors: string[];
166 /** The usernames of the people who made these edits, lowercased. */
167 editor_names: string[];
168 state: Uint8Array;
169 version: { kind: FolioVersionKind; note: string | null; authors: string[] } | null;
170 pending_authors: string[];
171 last_version_at: number;
172 workspace_slug: string | null;
173};
174
175type FolioSaveRow = { id: string; workspace_id: string; kind: FolioKind; title: string; text: string; preview: string | null; mentioned: string; space_id: string | null; trashed_at: string | null };
176
177/**
178 * What a folio's room saves to D1 after a burst of edits: its text
179 * rendition, card, search text, links, citations and history. Returns the
180 * version recorded (if one was), whether the text changed (so the room
181 * indexes it again), and the people newly mentioned, whom the room tells
182 * only if they can read the folio (src/folios/notify.ts).
183 */
184export async function saveFolio(env: SaveFolioEnv, input: SaveFolio, now = new Date()): Promise<{ version_id: string | null; changed: boolean; mentioned: string[]; last: string | null }> {
185 const at = now.toISOString();
186 const db = env.DB;
187 const folio = await db.prepare("SELECT id, workspace_id, kind, title, text, preview, mentioned, space_id, trashed_at FROM folios WHERE id = ?").bind(input.folio_id).first<FolioSaveRow>();
188 if (!folio) return { version_id: null, changed: false, mentioned: [], last: null };
189 const r = input.rendition;
190 const changed = folio.text !== r.text;
191 const last = input.editors[input.editors.length - 1] ?? null;
192 const statements: D1PreparedStatement[] = [];
193 const preview = r.preview ? JSON.stringify(r.preview) : null;
194 // A folio made from text that already reads as the kind renders it still needs its card.
195 if (!changed && preview !== folio.preview) statements.push(db.prepare("UPDATE folios SET preview = ? WHERE id = ?").bind(preview, folio.id));
196 if (changed) {
197 statements.push(
198 db
199 .prepare("UPDATE folios SET text = ?, excerpt = ?, preview = ?, edited_at = ?, edited_by = COALESCE(?, edited_by), updated_at = ? WHERE id = ?")
200 .bind(r.text, excerpt(r.text), preview, at, last, at, folio.id),
201 db.prepare("DELETE FROM folios_fts WHERE folio_id = ?").bind(folio.id),
202 db.prepare("INSERT INTO folios_fts (folio_id, kind, title, body) VALUES (?, ?, ?, ?)").bind(folio.id, folio.kind, folio.title, searchText(r.text)),
203 db.prepare("DELETE FROM folio_links WHERE from_folio = ?").bind(folio.id),
204 db.prepare("DELETE FROM folio_citations WHERE folio_id = ? AND source = 'body'").bind(folio.id),
205 );
206 for (const to of r.links.filter((id) => id !== folio.id).slice(0, 200)) {
207 statements.push(db.prepare("INSERT OR IGNORE INTO folio_links (from_folio, to_folio) VALUES (?, ?)").bind(folio.id, to));
208 }
209 for (const c of r.citations) {
210 statements.push(
211 db.prepare("INSERT OR IGNORE INTO folio_citations (folio_id, repo, path, kind, label, ref, source) VALUES (?, ?, ?, ?, ?, ?, 'body')").bind(folio.id, c.repo, c.path, c.kind, c.label ?? "", c.ref),
212 );
213 }
214 }
215 // A version: asked for, or the first save with changes after enough time.
216 let versionId: string | null = null;
217 const timed = input.pending_authors.length > 0 && now.getTime() - input.last_version_at >= VERSION_EVERY_MS;
218 if (input.version || (timed && changed)) {
219 versionId = newId("ver", now.getTime());
220 const authors = input.version?.authors.length ? input.version.authors : input.pending_authors;
221 let state: Uint8Array | null = input.state.byteLength <= MAX_VERSION_STATE ? input.state : null;
222 let stateKey: string | null = null;
223 if (!state) {
224 // Too large for a row: the file store keeps it.
225 try {
226 stateKey = `docs/versions/${folio.id}/${versionId}`;
227 await fileStore(env).put(stateKey, input.state, "application/octet-stream");
228 } catch (error) {
229 console.error("folios could not keep a large version's state; its text is kept", folio.id, String(error));
230 stateKey = null;
231 state = null;
232 }
233 }
234 statements.push(
235 db
236 .prepare("INSERT INTO folio_versions (id, folio_id, created_at, kind, authors, note, text, state, state_key) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)")
237 .bind(versionId, folio.id, at, input.version?.kind ?? "edit", JSON.stringify([...new Set(authors)]), input.version?.note ?? null, r.text, state, stateKey),
238 );
239 }
240 let told: string[] = [];
241 try {
242 told = JSON.parse(folio.mentioned) as string[];
243 } catch {
244 told = [];
245 }
246 const editors = new Set(input.editor_names);
247 const fresh = r.mentions.filter((name) => !told.includes(name) && !editors.has(name));
248 if (fresh.length || r.mentions.length !== told.length) {
249 statements.push(db.prepare("UPDATE folios SET mentioned = ? WHERE id = ?").bind(JSON.stringify([...new Set([...told.filter((n) => r.mentions.includes(n)), ...fresh])]), folio.id));
250 }
251 if (statements.length) await db.batch(statements);
252 const kind = input.version?.kind ?? "edit";
253 if (versionId && kind !== "created" && input.workspace_slug && !folio.trashed_at) {
254 const open = await workspaceReadable(db, folio.id).catch(() => false);
255 await publishFolioEvent(
256 env.EVENTS,
257 "folio.updated",
258 {
259 workspace: input.workspace_slug,
260 workspaceId: folio.workspace_id,
261 folioId: folio.id,
262 kind: folio.kind,
263 spaceId: folio.space_id,
264 title: open ? folio.title : null,
265 versionId,
266 versionKind: kind,
267 authors: [...new Set(input.version?.authors.length ? input.version.authors : input.pending_authors)],
268 },
269 last,
270 );
271 }
272 return { version_id: versionId, changed, mentioned: folio.trashed_at ? [] : fresh, last };
273}