Skip to content

Commit

Docs know what code they describe; a project's docs folder in Docs; Docs events; files on any S3 store

- Pages cite code (/cite, links to files, a Describes list); when a merge or a push to the default branch changes cited code, the page is marked possibly out of date, its owners are told, and a banner names the change (only to those who can read it); Mark as current, a stale list and badges - A project's README and docs/ folder show read-only in Docs and search, kept current on push, per viewer's access - doc.page.created, updated, archived and stale events (never on a repository's timeline or webhooks) - Self-hosted files on any S3-compatible store, signed by hand (SigV4) - Agents: stale_pages, and edits that bring a stale page up to date clear it (marks_current); the keep-the-docs-current routine starts from stale pages - Docs sidebar controls show on touch screens

syntaqxcommitted Parent702239dBrowse files
48 files+3132−890/48 viewed
+83−9
11 ---
22 title: Docs
3−description: Write a workspace's specs, runbooks, decisions and onboarding together, live, with your agents. Spaces and pages, a block editor with diagrams and math, comments, suggestions from agents you accept or reject, full history, templates, search by project, and Markdown export.
3+description: Write a workspace's specs, runbooks, decisions and onboarding together, live, with your agents. Spaces and pages, a block editor with diagrams and math, comments, suggestions from agents you accept or reject, pages that cite code and say when it changed, projects' docs folders, full history, templates, search by project, and Markdown export.
44 ---
55
66 import { Steps } from '@astrojs/starlight/components';
2424 | **Pages** | A tree inside each space. A page can hold other pages. Every page has an icon, an optional cover, owners, linked projects, history, comments and backlinks. |
2525 | **The editor** | Blocks you add with `/`: text, headings, lists, to-dos, toggles, callouts, code, tables, images and files, Mermaid diagrams, math, and live cards for issues, pull requests, channels, projects and other pages. |
2626 | **Agents** | Read the pages the person they work for can read. Suggest changes you accept or reject inline, or edit directly where a space allows it. Every change is attributed in the history. |
27+| **Code** | A page cites the code it describes. When a merged pull request changes that code, the page says it may be out of date, and its owners are told. A project's `docs/` folder can sit beside your spaces, read-only. |
2728
2829 Docs is its own mode, not a tab in a project. To see the pages about one
2930 project, filter Docs by that project.
123124 | Math | `/math` or `/latex`. Click the formula to change it. | `$$ … $$` |
124125 | Embed from g1t | `/embed`, then paste the address of an issue, pull request, channel, project or page. The card shows its live title and state (open, merged, closed). | a link |
125126 | Date | `/date` | the date |
127+| Cite code | `/cite`: choose a repository and a path, and what there the page describes (the file or folder, a symbol, an endpoint, an environment variable). See [Pages that cite code](#pages-that-cite-code). | a link to the file |
126128
127129 Inline:
128130
230232 you, depending on the space.
231233 </Aside>
232234
235+## Pages that cite code
236+
237+A page about code says which code. When that code changes, the page tells
238+you it may be out of date, instead of quietly going wrong.
239+
240+**Cite code in the text.** Type `/cite`, choose a repository and a path, and
241+say what the page describes there:
242+
243+| What it describes | For example |
244+| --- | --- |
245+| A file or folder | `src/export.ts`, `src/export` (everything in it) |
246+| A symbol | `exportCsv` in `src/export.ts` |
247+| An endpoint | `POST /v1/exports` in `api/routes.rs` |
248+| An environment variable | `EXPORT_BUCKET` in `wrangler.jsonc` |
249+
250+The path can be a pattern: `*` matches within a folder, `**` across folders
251+(`src/**/*.sql`), `?` one character. The citation is a chip in the text that
252+opens the code at the commit it was cited at. A link to a file in a repository
253+(`/acme/web/blob/main/src/export.ts`), pasted or written by an agent, counts
254+as a citation too.
255+
256+**Say what the whole page describes.** Under the title, **Describes** lists
257+repositories and paths the page is about. Press **+** to add one, or remove
258+one from the same place. Anyone who can edit the page can change it.
259+
260+**When the code changes.** When a pull request that changes a cited path is
261+merged, or a commit is pushed straight to the default branch, the page is
262+marked **possibly out of date**:
263+
264+- A banner on the page says which change and which paths: "Possibly out of
265+ date since acme/web#431 changed src/export.ts". **Review changes** opens
266+ the pull request's changes.
267+- The page's owners are told in their notifications.
268+- The page shows **Possibly stale** on its card, a dot in the sidebar's tree,
269+ and in **Possibly stale** in the sidebar, which lists every such page you
270+ can read. Docs' home shows the latest under **Possibly out of date**.
271+
272+Read what changed, update the page if it needs it, then press **Mark as
273+current**. That needs edit access, and clears it for everyone.
274+
275+Only the change's repository decides who sees it: someone who can't read
276+that repository sees "a change you can't see" instead of its name, and is
277+told without it.
278+
279+**Agents keep pages current.** An agent asked to bring a page up to date
280+reads what changed, then edits the page or suggests the change as usual,
281+saying the edit brings it up to date: once it is applied (or you accept the
282+suggestion), the page is current again. An agent only learns of changes in
283+repositories the person it works for can read.
284+
233285 ## History
234286
235287 **⋯ → History** lists every version of the page: when, who (people and agents),
280332 Agents search the same way, and only find what the people they answer can
281333 read.
282334
335+## Projects' docs
336+
337+A repository's own docs (its `docs/` folder and its README) stay in the
338+repository and change through pull requests. Docs can show them beside your
339+spaces, so one sidebar and one search cover both.
340+
341+<Steps>
342+
343+1. On Docs' home, press **Show a project's docs**, or **+** next to
344+ **Projects' docs** in the sidebar.
345+2. Choose a repository you can read. Its README and every Markdown file under
346+ `docs/` on the default branch appear in the sidebar, in their folders.
347+
348+</Steps>
349+
350+- The files are read-only here, rendered as Code renders them, with links and
351+ pictures pointing into the repository. **Edit in Code** opens the file;
352+ change it there or in a pull request.
353+- They follow the default branch: every push reads the changed files again.
354+- Search finds them with your pages, marked with the repository's name.
355+- Each person sees only the repositories they can read, whoever added them.
356+- Whoever added a project's docs, or a workspace owner, can stop showing them
357+ from the bin icon on any of its pages.
358+
359+A project's docs show its first 300 Markdown files; a file over 512 KB is
360+listed but not shown.
361+
283362 ## Export
284363
285364 - **⋯ → Export Markdown** downloads a page as a `.md` file.
292371
293372 ## Coming soon <Soon />
294373
295−- **Pages that know what they describe.** A page that cites code (a path, an
296− endpoint, a setting) is marked possibly stale when a merged pull request
297− changes it, and its owners are told. When an agent owns the page, it drafts
298− the update.
299374 - **"Write this up" from a thread** in one click, linked both ways.
300−- **A documenter agent** that keeps a space current after merges and writes
301− the weekly summary.
302−- **A project's `docs/` folder as a read-only space**, in the same tree and
303− search as your own spaces. Editing one of its pages opens a pull request.
375+- **A documenter agent** that keeps a space current after merges (updating
376+ the pages marked possibly out of date) and writes the weekly summary.
377+- **Editing a project's docs from Docs**, opening a pull request for you.
304378
305379 ## Pricing
306380
+44−3
22 * g1t's own blocks in the page editor, beside BlockNote's: callouts,
33 * Mermaid diagrams, math, and cards for g1t things (an issue, a pull
44 * request, a channel, a project, another page); and inline mentions of
5− * people, agents and pages, and dates. The docs service reads the same
5+ * people, agents and pages, dates, and citations of code (a file, folder
6+ * or pattern in a repository, and the symbol, endpoint or environment
7+ * variable there the page describes). The docs service reads the same
68 * names and attributes when it writes Markdown (services/docs
79 * src/markdown.ts), so keep the two in step. Browser-only: loaded with
810 * the editor.
1012 import { BlockNoteSchema, createCodeBlockSpec, defaultBlockSpecs, defaultInlineContentSpecs } from "@blocknote/core";
1113 import { createReactBlockSpec, createReactInlineContentSpec } from "@blocknote/react";
1214 import { codeBlockOptions } from "@blocknote/code-block";
13−import { AlertTriangle, CheckCircle2, CircleDot, FileText, GitPullRequest, Hash, Info, OctagonAlert, Package } from "lucide-react";
15+import { AlertTriangle, Braces, CheckCircle2, CircleDot, FileCode2, FileText, GitPullRequest, Hash, Info, KeyRound, OctagonAlert, Package, Route } from "lucide-react";
1416 import { useEffect, useId, useState, type ReactNode } from "react";
1517
18+import { citationHref } from "../../lib/docs";
19+
1620 /** What an embed shows, from the site (`-/docs/api?embed=`). */
1721 export type EmbedCard = { kind: "issue" | "pull" | "channel" | "project" | "page" | "link"; title: string; subtitle: string | null; state: string | null; href: string };
1822
281285 },
282286 );
283287
288+const CITATION_ICONS: Record<string, ReactNode> = {
289+ path: <FileCode2 size={12} aria-hidden="true" />,
290+ symbol: <Braces size={12} aria-hidden="true" />,
291+ endpoint: <Route size={12} aria-hidden="true" />,
292+ env: <KeyRound size={12} aria-hidden="true" />,
293+};
294+
295+/**
296+ * Code the page cites, inline: what it names (the path, or the symbol,
297+ * endpoint or variable there), linking to it in Code at the commit it was
298+ * cited at. When a merge changes it, the page is marked possibly out of
299+ * date (services/docs src/staleness.ts).
300+ */
301+export const Citation = createReactInlineContentSpec(
302+ {
303+ type: "citation",
304+ propSchema: { repo: { default: "" }, path: { default: "" }, kind: { default: "path" }, label: { default: "" }, ref: { default: "" } },
305+ content: "none",
306+ },
307+ {
308+ render: ({ inlineContent }) => {
309+ const p = inlineContent.props;
310+ const shown = p.kind !== "path" && p.label ? p.label : p.path;
311+ return (
312+ <a
313+ href={citationHref({ repo: p.repo, path: p.path, ref: p.ref || null })}
314+ aria-label={`${shown} in ${p.repo}${p.kind !== "path" ? `, ${p.path}` : ""}`}
315+ className="inline-flex items-center gap-1 rounded border border-line bg-raised px-1 py-px align-baseline font-mono text-[0.85em] text-fg no-underline hover:border-line-strong"
316+ >
317+ <span className="text-faint">{CITATION_ICONS[p.kind] ?? CITATION_ICONS.path}</span>
318+ {shown}
319+ </a>
320+ );
321+ },
322+ },
323+);
324+
284325 export const schema = BlockNoteSchema.create({
285326 blockSpecs: {
286327 ...defaultBlockSpecs,
290331 math: MathBlock(),
291332 embed: Embed(),
292333 },
293− inlineContentSpecs: { ...defaultInlineContentSpecs, mention: Mention, date: DateChip },
334+ inlineContentSpecs: { ...defaultInlineContentSpecs, mention: Mention, date: DateChip, citation: Citation },
294335 });
295336
296337 export type DocSchema = typeof schema;
+331−0
1+/**
2+ * Docs and code: citing code in a page (the Cite code dialog), the
3+ * header's "Describes" list, the banner on a page possibly out of date,
4+ * and showing a project's docs folder in Docs. Plan: docs/WORKSPACE.md,
5+ * "Agents and docs" and "Docs and repository docs".
6+ */
7+import { DOC_CITATION_KIND_LABELS, type DocCitationKind, type DocDescribes, type DocRepoSpace, type DocStaleness } from "@g1t/contracts";
8+import { AlertTriangle, Check, FileCode2, FolderGit2, GitCommitHorizontal, GitPullRequest, Plus, X } from "lucide-react";
9+import { useEffect, useState } from "react";
10+import { Link } from "react-router";
11+
12+import { citationHref } from "../../lib/docs";
13+import { Button, ErrorText, Field, Input, TimeAgo } from "../ui";
14+import { Combobox } from "../ui/combobox";
15+import { Dialog, DialogContent, DialogDescription, DialogFooter, DialogHeader, DialogTitle } from "../ui/dialog";
16+import { Hint } from "../ui/hint";
17+import { Popover, PopoverContent, PopoverTrigger } from "../ui/popover";
18+import { SelectField } from "../ui/select";
19+import { docsQuery, docsRequest } from "./actions";
20+
21+export type RepoChoice = { repo: string; default_branch: string; private: boolean };
22+
23+/** The workspace's repositories the viewer can read, asked for once the dialog that needs them opens. */
24+export function useRepoChoices(slug: string, wanted: boolean): { repos: RepoChoice[] | null; error: string | null } {
25+ const [repos, setRepos] = useState<RepoChoice[] | null>(null);
26+ const [error, setError] = useState<string | null>(null);
27+ useEffect(() => {
28+ if (!wanted || repos) return;
29+ let cancelled = false;
30+ void docsQuery<RepoChoice[]>(slug, { repos: "1" }).then((r) => {
31+ if (cancelled) return;
32+ if (r.ok) setRepos(r.value);
33+ else setError(r.error.message);
34+ });
35+ return () => {
36+ cancelled = true;
37+ };
38+ }, [slug, wanted, repos]);
39+ return { repos, error };
40+}
41+
42+function RepoPicker({ repos, value, onChange, id }: { repos: RepoChoice[] | null; value: string; onChange: (repo: string) => void; id?: string }) {
43+ return (
44+ <Combobox
45+ id={id}
46+ value={value}
47+ onValueChange={onChange}
48+ placeholder={repos ? "Choose a repository" : "Loading repositories…"}
49+ searchPlaceholder="Find a repository"
50+ emptyText="No repository you can read matches."
51+ disabled={!repos}
52+ options={(repos ?? []).map((r) => ({ value: r.repo, label: r.repo, description: r.private ? "Private" : undefined, icon: <FolderGit2 size={14} /> }))}
53+ className="w-full"
54+ />
55+ );
56+}
57+
58+/** What the Cite code dialog inserts: a citation chip's attributes. */
59+export type CitationInput = { repo: string; path: string; kind: DocCitationKind; label: string; ref: string };
60+
61+const LABEL_HINTS: Record<DocCitationKind, string> = {
62+ path: "",
63+ symbol: "The function, type or class, e.g. exportCsv",
64+ endpoint: "The method and route, e.g. POST /v1/exports",
65+ env: "The variable's name, e.g. EXPORT_BUCKET",
66+};
67+
68+/**
69+ * Cite code: a repository, a path in it (a file, a folder, or a pattern
70+ * like `src/export/**`), and what there the page describes. The citation
71+ * is pinned to the default branch's head, and the page is marked possibly
72+ * out of date when a merge changes it.
73+ */
74+export function CiteDialog({ slug, open, onOpenChange, onCite, projects = [] }: { slug: string; open: boolean; onOpenChange: (open: boolean) => void; onCite: (c: CitationInput) => void; projects?: string[] }) {
75+ const { repos, error: loadError } = useRepoChoices(slug, open);
76+ const [repo, setRepo] = useState("");
77+ const [path, setPath] = useState("");
78+ const [kind, setKind] = useState<DocCitationKind>("path");
79+ const [label, setLabel] = useState("");
80+ const [busy, setBusy] = useState(false);
81+ const [error, setError] = useState<string | null>(null);
82+ useEffect(() => {
83+ if (!open) return;
84+ setError(null);
85+ setPath("");
86+ setLabel("");
87+ setKind("path");
88+ }, [open]);
89+ // The page's own project first, when it has one.
90+ useEffect(() => {
91+ if (!repo && repos?.length) setRepo(repos.find((r) => projects.includes(r.repo))?.repo ?? repos[0]!.repo);
92+ }, [repos, repo, projects]);
93+ const submit = async () => {
94+ if (!repo || !path.trim()) return setError("Choose a repository and name a path in it.");
95+ if (kind !== "path" && !label.trim()) return setError(`Name the ${kind === "env" ? "variable" : kind}.`);
96+ setBusy(true);
97+ setError(null);
98+ const found = await docsQuery<{ repo: string; path: string; ref: string | null }>(slug, { cite: repo, path: path.trim() });
99+ setBusy(false);
100+ if (!found.ok) return setError(found.error.message);
101+ onCite({ repo: found.value.repo, path: found.value.path, kind, label: kind === "path" ? "" : label.trim(), ref: found.value.ref ?? "" });
102+ onOpenChange(false);
103+ };
104+ return (
105+ <Dialog open={open} onOpenChange={onOpenChange}>
106+ <DialogContent className="max-w-lg">
107+ <DialogHeader>
108+ <DialogTitle className="flex items-center gap-2">
109+ <FileCode2 size={16} /> Cite code
110+ </DialogTitle>
111+ <DialogDescription>Link this page to the code it describes. When a merged pull request changes it, the page&apos;s owners hear that it may be out of date.</DialogDescription>
112+ </DialogHeader>
113+ <form
114+ className="space-y-4"
115+ onSubmit={(e) => {
116+ e.preventDefault();
117+ void submit();
118+ }}
119+ >
120+ <Field label="Repository">
121+ <RepoPicker repos={repos} value={repo} onChange={setRepo} />
122+ </Field>
123+ <Field label="Path" hint="A file, a folder (everything in it), or a pattern such as src/export/** or *.sql.">
124+ <Input value={path} onChange={(e) => setPath(e.target.value)} placeholder="src/export.ts" autoComplete="off" spellCheck={false} className="font-mono" />
125+ </Field>
126+ <Field label="What it describes">
127+ <SelectField value={kind} onValueChange={(v) => setKind(v as DocCitationKind)} options={(Object.keys(DOC_CITATION_KIND_LABELS) as DocCitationKind[]).map((k) => ({ value: k, label: DOC_CITATION_KIND_LABELS[k] }))} className="w-full" aria-label="What it describes" />
128+ </Field>
129+ {kind !== "path" && (
130+ <Field label={kind === "symbol" ? "Symbol" : kind === "endpoint" ? "Endpoint" : "Variable"} hint={LABEL_HINTS[kind]}>
131+ <Input value={label} onChange={(e) => setLabel(e.target.value)} autoComplete="off" spellCheck={false} className="font-mono" />
132+ </Field>
133+ )}
134+ {(error || loadError) && <ErrorText>{error ?? loadError}</ErrorText>}
135+ <DialogFooter>
136+ <Button type="submit" variant="accent" disabled={busy || !repos}>
137+ {busy ? "Checking…" : "Cite"}
138+ </Button>
139+ </DialogFooter>
140+ </form>
141+ </DialogContent>
142+ </Dialog>
143+ );
144+}
145+
146+/** The header's "Describes": which code the page is about, editable by anyone who can edit it. */
147+export function Describes({ slug, describes, editable, onSave }: { slug: string; describes: DocDescribes[]; editable: boolean; onSave: (next: DocDescribes[]) => Promise<unknown> }) {
148+ const [open, setOpen] = useState(false);
149+ const { repos } = useRepoChoices(slug, open);
150+ const [repo, setRepo] = useState("");
151+ const [path, setPath] = useState("");
152+ useEffect(() => {
153+ if (!repo && repos?.length) setRepo(describes[0]?.repo ?? repos[0]!.repo);
154+ }, [repos, repo, describes]);
155+ if (!describes.length && !editable) return null;
156+ const add = async () => {
157+ const clean = path.trim().replace(/^\/+|\/+$/g, "");
158+ if (!repo || !clean) return;
159+ await onSave([...describes, { repo, path: clean }]);
160+ setPath("");
161+ };
162+ return (
163+ <span className="flex flex-wrap items-center gap-1.5">
164+ <span>Describes</span>
165+ {describes.map((d) => (
166+ <Link key={`${d.repo}:${d.path}`} to={citationHref({ repo: d.repo, path: d.path, ref: null })} className="inline-flex items-center gap-1 rounded-full border border-line px-2 py-0.5 font-mono text-[0.6875rem] text-muted hover:border-line-strong hover:text-fg">
167+ <FileCode2 size={11} aria-hidden="true" />
168+ {d.repo}:{d.path}
169+ </Link>
170+ ))}
171+ {editable && (
172+ <Popover open={open} onOpenChange={setOpen}>
173+ <Hint label={describes.length ? "Change what this page describes" : "Say which code this page describes"}>
174+ <PopoverTrigger asChild>
175+ <button type="button" aria-label="Edit what this page describes" className="inline-flex size-5 items-center justify-center rounded-full border border-dashed border-line text-faint hover:border-line-strong hover:text-fg">
176+ <Plus size={11} />
177+ </button>
178+ </PopoverTrigger>
179+ </Hint>
180+ <PopoverContent align="start" className="w-96 p-4 text-sm">
181+ <p className="font-medium">Code this page describes</p>
182+ <p className="mt-1 text-xs leading-relaxed text-muted">When a merged pull request changes any of it, the page is marked possibly out of date and its owners are told.</p>
183+ {describes.length > 0 && (
184+ <ul className="mt-3 space-y-1">
185+ {describes.map((d, i) => (
186+ <li key={`${d.repo}:${d.path}`} className="flex items-center gap-2 rounded-md bg-raised px-2 py-1 font-mono text-xs">
187+ <span className="min-w-0 grow truncate">
188+ {d.repo}:{d.path}
189+ </span>
190+ <button type="button" aria-label={`Remove ${d.repo}:${d.path}`} onClick={() => void onSave(describes.filter((_, j) => j !== i))} className="text-faint hover:text-danger">
191+ <X size={13} />
192+ </button>
193+ </li>
194+ ))}
195+ </ul>
196+ )}
197+ <form
198+ className="mt-3 space-y-2"
199+ onSubmit={(e) => {
200+ e.preventDefault();
201+ void add();
202+ }}
203+ >
204+ <RepoPicker repos={repos} value={repo} onChange={setRepo} />
205+ <div className="flex gap-2">
206+ <Input value={path} onChange={(e) => setPath(e.target.value)} placeholder="src/export or src/**/*.sql" aria-label="Path" autoComplete="off" spellCheck={false} className="font-mono" />
207+ <Button type="submit" variant="quiet" disabled={!repo || !path.trim()}>
208+ Add
209+ </Button>
210+ </div>
211+ </form>
212+ </PopoverContent>
213+ </Popover>
214+ )}
215+ </span>
216+ );
217+}
218+
219+function changeLink(c: DocStaleness["changes"][number]): { href: string; label: string } | null {
220+ if (!c.visible || !c.repo) return null;
221+ if (c.pull) return { href: `/${c.repo}/pull/${c.pull.number}`, label: `${c.repo}#${c.pull.number}` };
222+ if (c.commit) return { href: `/${c.repo}/commit/${c.commit}`, label: `${c.repo}@${c.commit.slice(0, 7)}` };
223+ return null;
224+}
225+
226+/**
227+ * "Possibly out of date since acme/web#431 changed src/export.ts":
228+ * the newest change the reader can see, the rest counted; Review changes
229+ * opens it; Mark as current clears it for everyone.
230+ */
231+export function StaleBanner({ staleness, canMark, onMark }: { staleness: DocStaleness; canMark: boolean; onMark: () => Promise<unknown> }) {
232+ const [busy, setBusy] = useState(false);
233+ const latest = staleness.changes[0];
234+ if (!latest) return null;
235+ const link = changeLink(latest);
236+ const more = staleness.changes.length - 1;
237+ return (
238+ <div role="status" className="mt-6 flex flex-wrap items-center gap-x-3 gap-y-2 rounded-lg border border-warn/30 bg-warn/8 px-3 py-2 text-sm">
239+ <AlertTriangle size={15} className="shrink-0 text-warn" aria-hidden="true" />
240+ <span className="min-w-0 grow text-fg-soft">
241+ Possibly out of date since{" "}
242+ {link ? (
243+ <>
244+ <Link to={link.href} className="font-medium text-fg hover:underline">
245+ {link.label}
246+ </Link>
247+ {latest.paths.length > 0 && (
248+ <>
249+ {" "}
250+ changed <span className="font-mono text-[0.8125rem] text-fg">{latest.paths[0]}</span>
251+ {latest.paths.length > 1 && ` and ${latest.paths.length - 1} more`}
252+ </>
253+ )}
254+ </>
255+ ) : (
256+ "a change you can't see touched code it cites"
257+ )}
258+ {more > 0 && <span className="text-muted"> · {more === 1 ? "1 more change" : `${more} more changes`}</span>}
259+ <span className="text-faint">
260+ {" "}
261+ · <TimeAgo at={latest.at} />
262+ </span>
263+ </span>
264+ {link && (
265+ <Link to={latest.pull ? `${link.href}?tab=changes` : link.href} className="inline-flex h-7 items-center gap-1.5 rounded-md px-2 text-xs font-medium text-fg hover:bg-raised">
266+ {latest.pull ? <GitPullRequest size={13} /> : <GitCommitHorizontal size={13} />} Review changes
267+ </Link>
268+ )}
269+ {canMark && (
270+ <button
271+ type="button"
272+ disabled={busy}
273+ onClick={async () => {
274+ setBusy(true);
275+ await onMark();
276+ setBusy(false);
277+ }}
278+ className="inline-flex h-7 items-center gap-1.5 rounded-md bg-warn/15 px-2.5 text-xs font-medium text-warn hover:bg-warn/25 disabled:opacity-60"
279+ >
280+ <Check size={13} /> {busy ? "Marking…" : "Mark as current"}
281+ </button>
282+ )}
283+ </div>
284+ );
285+}
286+
287+/** Show a project's docs: choose a repository the viewer can read; its `docs/` folder and README appear in the sidebar and search. */
288+export function RepoDocsDialog({ slug, open, onOpenChange, shown, onAdded }: { slug: string; open: boolean; onOpenChange: (open: boolean) => void; shown: string[]; onAdded: (space: DocRepoSpace) => void }) {
289+ const { repos, error: loadError } = useRepoChoices(slug, open);
290+ const [repo, setRepo] = useState("");
291+ const [busy, setBusy] = useState(false);
292+ const [error, setError] = useState<string | null>(null);
293+ const choices = repos?.filter((r) => !shown.includes(r.repo)) ?? null;
294+ return (
295+ <Dialog open={open} onOpenChange={onOpenChange}>
296+ <DialogContent className="max-w-lg">
297+ <DialogHeader>
298+ <DialogTitle className="flex items-center gap-2">
299+ <FolderGit2 size={16} /> Show a project&apos;s docs
300+ </DialogTitle>
301+ <DialogDescription>The repository&apos;s docs folder and README, read from its default branch, appear beside your spaces and in search, and follow every push. Only people who can read the repository see them. Changes go through the repository.</DialogDescription>
302+ </DialogHeader>
303+ <form
304+ className="space-y-4"
305+ onSubmit={async (e) => {
306+ e.preventDefault();
307+ if (!repo) return;
308+ setBusy(true);
309+ setError(null);
310+ const added = await docsRequest<DocRepoSpace>(slug, "add_repo_space", { repo });
311+ setBusy(false);
312+ if (!added.ok) return setError(added.error.message);
313+ onAdded(added.value);
314+ onOpenChange(false);
315+ }}
316+ >
317+ <Field label="Repository">
318+ <RepoPicker repos={choices} value={repo} onChange={setRepo} />
319+ </Field>
320+ {choices && choices.length === 0 && <p className="text-xs text-faint">Every repository you can read is already shown.</p>}
321+ {(error || loadError) && <ErrorText>{error ?? loadError}</ErrorText>}
322+ <DialogFooter>
323+ <Button type="submit" variant="accent" disabled={busy || !repo}>
324+ {busy ? "Reading its docs…" : "Show its docs"}
325+ </Button>
326+ </DialogFooter>
327+ </form>
328+ </DialogContent>
329+ </Dialog>
330+ );
331+}
+26−2
1515 import { BlockNoteViewEditor, SuggestionMenuController, ThreadsSidebar, getDefaultReactSlashMenuItems, useCreateBlockNote } from "@blocknote/react";
1616 import { BlockNoteView } from "@blocknote/shadcn";
1717 import type { DocFile, DocRole, DocSuggestion, DocsLiveEvent, Result } from "@g1t/contracts";
18−import { AlertTriangle, AtSign, Calendar, CheckCircle2, FileText, GitPullRequest, Info, Link2, Sigma, Workflow } from "lucide-react";
18+import { AlertTriangle, AtSign, Calendar, CheckCircle2, FileCode2, FileText, GitPullRequest, Info, Link2, Sigma, Workflow } from "lucide-react";
1919 import { type ReactNode, useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
2020
2121 import { canDo, cursorColour } from "../../lib/docs";
2222 import { schema, type DocEditorInstance } from "./blocks";
23+import { CiteDialog } from "./code";
2324 import { EditorSkeleton } from "./editor-skeleton";
2425 import type { PageThread } from "./page-parts";
2526 import { DocsProvider, type LiveStatus } from "./provider";
4748 renderSuggestion?: (suggestion: DocSuggestion) => ReactNode;
4849 /** Comments on the whole page (not on a passage), live from the document. */
4950 onPageThreads?: (threads: PageThread[]) => void;
51+ /** The projects the page and its space are about: Cite code offers them first. */
52+ projects?: string[];
5053 };
5154
5255 /** Nobody can do anything with threads: a reader's view. */
112115 return <LiveEditor {...props} provider={provider} />;
113116 }
114117
115−function LiveEditor({ slug, pageId, role, me, mentionables, usercontent, suggestions, showComments, onPresence, renderSuggestion, onPageThreads, provider }: DocEditorProps & { provider: DocsProvider }) {
118+function LiveEditor({ slug, pageId, role, me, mentionables, usercontent, suggestions, showComments, onPresence, renderSuggestion, onPageThreads, projects, provider }: DocEditorProps & { provider: DocsProvider }) {
116119 const editable = canDo(role, "edit");
120+ const [citing, setCiting] = useState(false);
117121 const colour = cursorColour(me.name);
118122 // Below 1280px the comments sit under the page: opening them goes there.
119123 const comments = useRef<HTMLElement>(null);
253257 onItemClick: () => editor.insertInlineContent("[[" as never),
254258 },
255259 {
260+ title: "Cite code",
261+ subtext: "A file, folder, symbol, endpoint or variable this page describes",
262+ aliases: ["cite", "code", "citation", "path", "file", "symbol", "endpoint", "env", "variable"],
263+ group: "g1t",
264+ icon: <FileCode2 size={18} />,
265+ onItemClick: () => setCiting(true),
266+ },
267+ {
256268 title: "Date",
257269 subtext: "Today's date, which you can change",
258270 aliases: ["date", "today", "when"],
317329 <SuggestionMenuController triggerCharacter="@" getItems={async (query) => mentionItems(query)} />
318330 <SuggestionMenuController triggerCharacter="[[" getItems={pageItems} />
319331 </BlockNoteView>
332+ {editable && (
333+ <CiteDialog
334+ slug={slug}
335+ open={citing}
336+ onOpenChange={setCiting}
337+ projects={projects}
338+ onCite={(c) => {
339+ editor.focus();
340+ editor.insertInlineContent([{ type: "citation", props: c }, " "] as never);
341+ }}
342+ />
343+ )}
320344 </div>
321345 );
322346 }
+4−1
4343 <span className="flex size-9 items-center justify-center rounded-lg border border-line bg-bg text-lg">
4444 <PageIcon icon={page.icon} size={18} />
4545 </span>
46− <span className="mt-2 line-clamp-1 text-sm font-medium text-fg group-hover:text-accent">{page.title || "Untitled"}</span>
46+ <span className="mt-2 flex items-center gap-2">
47+ <span className="line-clamp-1 min-w-0 text-sm font-medium text-fg group-hover:text-accent">{page.title || "Untitled"}</span>
48+ {page.stale && <span className="shrink-0 rounded-full bg-warn/12 px-1.5 py-px text-[0.625rem] font-medium text-warn">Possibly stale</span>}
49+ </span>
4750 <span className="mt-1 line-clamp-2 min-h-[2.5em] text-xs leading-relaxed text-muted">{page.excerpt || "Nothing written yet."}</span>
4851 <span className="mt-3 flex items-center gap-1.5 text-[0.6875rem] text-faint">
4952 {page.updated_by && <Face who={page.updated_by} size={14} />}
+130−11
11 /**
22 * Docs mode's sidebar (beside the rail, docs/WORKSPACE.md "Shell"):
3− * search, Home, Templates and Trash; Favorites and Recent; then each
4− * space with its page tree, which opens to the page being read. Pages are
3+ * search, Home, Possibly stale, Templates and Trash; Favorites and
4+ * Recent; each space with its page tree, which opens to the page being
5+ * read; then projects' docs folders, read-only. Pages are
56 * dragged to reorder them or to put one inside another; the ⋯ menu moves
67 * them too, for keyboards and phones.
78 */
8−import type { DocsSidebarSpace } from "@g1t/contracts";
9−import { BookOpen, ChevronRight, Clock, FileText, Home, LayoutTemplate, Lock, Plus, Search, Settings, Star, Trash2, Users } from "lucide-react";
9+import type { DocRepoSpace, DocsSidebarSpace } from "@g1t/contracts";
10+import { AlertTriangle, BookOpen, ChevronRight, Clock, FileText, Folder, FolderGit2, Home, LayoutTemplate, Lock, Plus, Search, Settings, Star, Trash2, Users } from "lucide-react";
1011 import { type DragEvent, type ReactNode, useEffect, useMemo, useState } from "react";
11−import { NavLink, useLocation, useNavigate, useParams } from "react-router";
12+import { NavLink, useLocation, useNavigate, useParams, useRevalidator } from "react-router";
1213
13−import { buildTree, canDo, pageIdOf, pagePath, pathTo, type TreeItem } from "../../lib/docs";
14+import { buildTree, canDo, pageIdOf, pagePath, pathTo, repoFilePath, repoFolders, type RepoFolder, type TreeItem } from "../../lib/docs";
1415 import { Hint } from "../ui/hint";
1516 import { Skeleton } from "../ui/skeleton";
1617 import { docsRequest, useDocsAction, useDocsData } from "./actions";
18+import { RepoDocsDialog } from "./code";
1719
1820 const ROW = "group flex h-8 items-center gap-1.5 rounded-md pr-1 text-[0.8125rem] transition-colors";
1921
7577 setDrag,
7678 onDrop,
7779 onAdd,
80+ stale,
7881 }: {
7982 slug: string;
8083 space: DocsSidebarSpace;
84+ stale: Set<string>;
8185 item: TreeItem;
8286 current: string | null;
8387 open: Set<string>;
135139 <PageIcon icon={item.icon} size={14} />
136140 </span>
137141 <span className="min-w-0 truncate">{item.title || "Untitled"}</span>
142+ {stale.has(item.id) && (
143+ <Hint label="Possibly out of date: code it cites changed">
144+ <span className="ml-auto size-1.5 shrink-0 rounded-full bg-warn" aria-label="Possibly out of date" />
145+ </Hint>
146+ )}
138147 </NavLink>
139148 {editable && (
140149 <Hint label="Add a page inside">
142151 type="button"
143152 onClick={() => onAdd(item.id)}
144153 aria-label={`Add a page inside ${item.title || "Untitled"}`}
145− className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100"
154+ className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100 pointer-coarse:size-9 pointer-coarse:opacity-100"
146155 >
147156 <Plus size={13} />
148157 </button>
152161 {expanded && item.children.length > 0 && (
153162 <ul>
154163 {item.children.map((child) => (
155− <TreeRow key={child.id} slug={slug} space={space} item={child} current={current} open={open} toggle={toggle} drag={drag} setDrag={setDrag} onDrop={onDrop} onAdd={onAdd} />
164+ <TreeRow key={child.id} slug={slug} space={space} stale={stale} item={child} current={current} open={open} toggle={toggle} drag={drag} setDrag={setDrag} onDrop={onDrop} onAdd={onAdd} />
156165 ))}
157166 </ul>
158167 )}
164173 const navigate = useNavigate();
165174 const { send } = useDocsAction(slug);
166175 const tree = useMemo(() => buildTree(space.pages), [space.pages]);
176+ const stale = useMemo(() => new Set(space.pages.filter((p) => p.stale).map((p) => p.id)), [space.pages]);
167177 const [open, setOpen] = useState<Set<string>>(() => new Set(pathTo(space.pages, current)));
168178 const [collapsed, setCollapsed] = useState(false);
169179 // Opening a page opens the way to it.
221231 </NavLink>
222232 {canDo(space.viewer_role, "manage") && (
223233 <Hint label="Space settings">
224− <NavLink to={`/${slug}/-/docs/${space.slug}/settings`} aria-label={`${space.name} settings`} className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100">
234+ <NavLink to={`/${slug}/-/docs/${space.slug}/settings`} aria-label={`${space.name} settings`} className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100 pointer-coarse:size-9 pointer-coarse:opacity-100">
225235 <Settings size={13} />
226236 </NavLink>
227237 </Hint>
228238 )}
229239 {editable && (
230240 <Hint label={`New page in ${space.name}`}>
231− <button type="button" onClick={() => add(null)} aria-label={`New page in ${space.name}`} className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100">
241+ <button type="button" onClick={() => add(null)} aria-label={`New page in ${space.name}`} className="flex size-6 shrink-0 items-center justify-center rounded text-faint opacity-0 group-hover:opacity-100 hover:bg-line hover:text-fg focus-visible:opacity-100 pointer-coarse:size-9 pointer-coarse:opacity-100">
232242 <Plus size={13} />
233243 </button>
234244 </Hint>
237247 {!collapsed && (
238248 <ul>
239249 {tree.map((item) => (
240− <TreeRow key={item.id} slug={slug} space={space} item={item} current={current} open={open} toggle={toggle} drag={drag} setDrag={setDrag} onDrop={drop} onAdd={add} />
250+ <TreeRow key={item.id} slug={slug} space={space} stale={stale} item={item} current={current} open={open} toggle={toggle} drag={drag} setDrag={setDrag} onDrop={drop} onAdd={add} />
241251 ))}
242252 {tree.length === 0 && <li className="py-1 pr-2 pl-9 text-xs text-faint">No pages yet.</li>}
243253 </ul>
246256 );
247257 }
248258
259+function RepoFolderRows({ slug, repo, folder, depth, current }: { slug: string; repo: string; folder: RepoFolder; depth: number; current: string }) {
260+ const [open, setOpen] = useState<Set<string>>(() => new Set(folder.folders.filter((f) => current.includes(`/${f.path}/`)).map((f) => f.path)));
261+ return (
262+ <>
263+ {folder.files.map((f) => {
264+ const to = repoFilePath(slug, repo, f.path);
265+ return (
266+ <li key={f.path}>
267+ <NavLink to={to} prefetch="intent" className={({ isActive }) => `${ROW} ${isActive ? "bg-raised font-medium text-fg" : "text-muted hover:bg-raised/60 hover:text-fg"}`} style={{ paddingLeft: 26 + depth * 14 }}>
268+ <FileText size={14} className="shrink-0 text-faint" aria-hidden="true" />
269+ <span className="min-w-0 truncate">{f.title}</span>
270+ </NavLink>
271+ </li>
272+ );
273+ })}
274+ {folder.folders.map((sub) => {
275+ const expanded = open.has(sub.path);
276+ return (
277+ <li key={sub.path}>
278+ <button
279+ type="button"
280+ aria-expanded={expanded}
281+ onClick={() =>
282+ setOpen((was) => {
283+ const next = new Set(was);
284+ if (next.has(sub.path)) next.delete(sub.path);
285+ else next.add(sub.path);
286+ return next;
287+ })
288+ }
289+ className={`${ROW} w-full text-muted hover:bg-raised/60 hover:text-fg`}
290+ style={{ paddingLeft: 4 + (depth + 1) * 14 }}
291+ >
292+ <ChevronRight size={13} className={`shrink-0 text-faint transition-transform ${expanded ? "rotate-90" : ""}`} />
293+ <Folder size={14} className="shrink-0 text-faint" aria-hidden="true" />
294+ <span className="min-w-0 truncate">{sub.name}</span>
295+ </button>
296+ {expanded && (
297+ <ul>
298+ <RepoFolderRows slug={slug} repo={repo} folder={sub} depth={depth + 1} current={current} />
299+ </ul>
300+ )}
301+ </li>
302+ );
303+ })}
304+ </>
305+ );
306+}
307+
308+/** A project's docs folder: read-only, its files as folders, from the repository's default branch. */
309+function RepoTree({ slug, space }: { slug: string; space: DocRepoSpace }) {
310+ const { pathname } = useLocation();
311+ const base = `/${slug}/-/docs/repo/${space.repo}/`;
312+ const here = decodeURIComponent(pathname).startsWith(base);
313+ const [collapsed, setCollapsed] = useState(!here);
314+ const root = useMemo(() => repoFolders(space.files), [space.files]);
315+ return (
316+ <li className="mt-1">
317+ <div className={`${ROW} text-fg-soft`}>
318+ <button type="button" onClick={() => setCollapsed(!collapsed)} aria-expanded={!collapsed} aria-label={collapsed ? `Show ${space.repo}'s docs` : `Hide ${space.repo}'s docs`} className="flex size-5 shrink-0 items-center justify-center rounded text-faint hover:bg-line hover:text-fg">
319+ <ChevronRight size={13} className={`transition-transform ${collapsed ? "" : "rotate-90"}`} />
320+ </button>
321+ <span className="flex w-4 shrink-0 justify-center">
322+ <FolderGit2 size={14} className="text-faint" aria-hidden="true" />
323+ </span>
324+ <span className="min-w-0 grow truncate font-medium">{space.repo}</span>
325+ </div>
326+ {!collapsed && (
327+ <ul>
328+ <RepoFolderRows slug={slug} repo={space.repo} folder={root} depth={0} current={decodeURIComponent(pathname)} />
329+ {space.files.length === 0 && <li className="py-1 pr-2 pl-9 text-xs text-faint">{space.indexed_at ? "No docs folder or README." : "Reading its docs…"}</li>}
330+ </ul>
331+ )}
332+ </li>
333+ );
334+}
335+
249336 export function DocsSidebar({ slug, onClose }: { slug: string; onClose?: () => void }) {
250337 const data = useDocsData();
251338 const navigate = useNavigate();
254341 const current = pageIdOf(params.page);
255342 const [drag, setDrag] = useState<Drag>(null);
256343 const [query, setQuery] = useState("");
344+ const [addingRepo, setAddingRepo] = useState(false);
345+ const { revalidate } = useRevalidator();
257346 const sidebar = data?.sidebar;
258347 const general = sidebar?.spaces.find((s) => s.is_default) ?? sidebar?.spaces.find((s) => canDo(s.viewer_role, "edit"));
259348 const newPage = async () => {
297386 <SideLink to={`/${slug}/-/docs`} end icon={<Home size={15} />}>
298387 Home
299388 </SideLink>
389+ {!!sidebar?.stale_count && (
390+ <SideLink to={`/${slug}/-/docs/stale`} icon={<AlertTriangle size={15} className="text-warn" />} trailing={<span className="text-[0.6875rem] text-warn tabular-nums">{sidebar.stale_count}</span>}>
391+ Possibly stale
392+ </SideLink>
393+ )}
300394 <SideLink to={`/${slug}/-/docs/templates`} icon={<LayoutTemplate size={15} />}>
301395 Templates
302396 </SideLink>
361455 </ul>
362456 </Section>
363457 {pathname === `/${slug}/-/docs` && sidebar.spaces.length === 0 && <p className="mt-3 px-2 text-xs text-faint">No spaces you can read yet.</p>}
458+ <Section
459+ title="Projects' docs"
460+ open={(sidebar.repos ?? []).length > 0}
461+ action={
462+ <Hint label="Show a project's docs">
463+ <button type="button" onClick={() => setAddingRepo(true)} aria-label="Show a project's docs" className="flex size-6 items-center justify-center rounded text-faint hover:bg-raised hover:text-fg">
464+ <Plus size={13} />
465+ </button>
466+ </Hint>
467+ }
468+ >
469+ <ul>
470+ {(sidebar.repos ?? []).map((space) => (
471+ <RepoTree key={space.id} slug={slug} space={space} />
472+ ))}
473+ {(sidebar.repos ?? []).length === 0 && (
474+ <li>
475+ <button type="button" onClick={() => setAddingRepo(true)} className="px-2 py-1 text-left text-xs text-faint hover:text-fg">
476+ Show a repository&apos;s docs folder here, read-only.
477+ </button>
478+ </li>
479+ )}
480+ </ul>
481+ </Section>
482+ <RepoDocsDialog slug={slug} open={addingRepo} onOpenChange={setAddingRepo} shown={(sidebar.repos ?? []).map((r) => r.repo.toLowerCase())} onAdded={() => void revalidate()} />
364483 </>
365484 )}
366485 </nav>
+22−1
11 import assert from "node:assert/strict";
22 import { test } from "node:test";
33
4−import { buildTree, canDo, coverStyle, cursorColour, flatten, pageIdOf, pagePath, pathTo, readingTime, snippetParts } from "./docs.ts";
4+import { buildTree, canDo, citationHref, coverStyle, cursorColour, flatten, pageIdOf, pagePath, pathTo, readingTime, repoFilePath, repoFolders, snippetParts } from "./docs.ts";
55
66 const id = "pag_01jb2k7x9hfq0b3zj0f5s2m8ra";
77
5151 assert.equal(canDo("comment", "edit"), false);
5252 assert.deepEqual(readingTime("one two three"), { words: 3, minutes: 1 });
5353 });
54+
55+test("a citation links to its code, a glob to the folder it starts from", () => {
56+ assert.equal(citationHref({ repo: "acme/web", path: "src/export.ts", ref: "abc1234" }), "/acme/web/blob/abc1234/src/export.ts");
57+ assert.equal(citationHref({ repo: "acme/web", path: "src/jobs", ref: null }), "/acme/web/tree/HEAD/src/jobs");
58+ assert.equal(citationHref({ repo: "acme/web", path: "src/**/*.sql", ref: "abc" }), "/acme/web/tree/abc/src");
59+});
60+
61+test("a project's docs become folders under the space", () => {
62+ const root = repoFolders([
63+ { path: "README.md", title: "Acme" },
64+ { path: "docs/setup.md", title: "Setup" },
65+ { path: "docs/guides/deploy.md", title: "Deploy" },
66+ ]);
67+ assert.deepEqual(
68+ root.files.map((f) => f.path),
69+ ["README.md", "docs/setup.md"],
70+ );
71+ assert.equal(root.folders[0]!.name, "guides");
72+ assert.deepEqual(root.folders[0]!.files.map((f) => f.title), ["Deploy"]);
73+ assert.equal(repoFilePath("acme", "acme/web", "docs/a b.md"), "/acme/-/docs/repo/acme/web/docs/a%20b.md");
74+});
+48−0
135135 export function markdownFileName(title: string): string {
136136 return `${title.replace(/[\\/:*?"<>|]+/g, " ").trim() || "Untitled"}.md`;
137137 }
138+
139+/**
140+ * Where a citation links: the file or folder in Code at the commit it was
141+ * cited at, or the default branch (`HEAD`). A glob links to the folder it
142+ * starts from. Mirrors `citationHref` in services/docs src/citations.ts.
143+ */
144+export function citationHref(c: { repo: string; path: string; ref: string | null }): string {
145+ const parts = c.path.split("/").filter(Boolean);
146+ const globAt = parts.findIndex((p) => /[*?]/.test(p));
147+ const glob = globAt >= 0;
148+ const shown = (glob ? parts.slice(0, globAt) : parts).map(encodeURIComponent).join("/");
149+ const kind = glob || !/\.[A-Za-z0-9]{1,10}$/.test(c.path) ? "tree" : "blob";
150+ return `/${c.repo}/${kind}/${encodeURIComponent(c.ref || "HEAD")}${shown ? `/${shown}` : ""}`;
151+}
152+
153+/** A project's docs file's address in Docs. */
154+export function repoFilePath(workspace: string, repo: string, path: string): string {
155+ return `/${workspace}/-/docs/repo/${repo}/${path.split("/").map(encodeURIComponent).join("/")}`;
156+}
157+
158+/** A folder of a project's docs, as the sidebar shows it: files, then folders, each by name. */
159+export type RepoFolder = { name: string; path: string; files: { path: string; title: string }[]; folders: RepoFolder[] };
160+
161+/** A project's docs files as folders: README and `docs/` at the top, `docs/a/b.md` under `a`. */
162+export function repoFolders(files: { path: string; title: string }[]): RepoFolder {
163+ const root: RepoFolder = { name: "", path: "", files: [], folders: [] };
164+ for (const file of files) {
165+ // `docs/` is the space itself: its files sit at the top beside the README.
166+ const parts = file.path.replace(/^docs\//i, "").split("/");
167+ parts.pop();
168+ let at = root;
169+ for (const part of parts) {
170+ let next = at.folders.find((f) => f.name === part);
171+ if (!next) {
172+ next = { name: part, path: at.path ? `${at.path}/${part}` : part, files: [], folders: [] };
173+ at.folders.push(next);
174+ }
175+ at = next;
176+ }
177+ at.files.push(file);
178+ }
179+ const sort = (f: RepoFolder) => {
180+ f.folders.sort((a, b) => a.name.localeCompare(b.name));
181+ f.folders.forEach(sort);
182+ };
183+ sort(root);
184+ return root;
185+}
+3−0
175175 route("templates", "routes/workspace/docs/templates.tsx"),
176176 route("trash", "routes/workspace/docs/trash.tsx"),
177177 route("new", "routes/workspace/docs/new-space.tsx"),
178+ // Pages possibly out of date, and a project's docs folder, read-only.
179+ route("stale", "routes/workspace/docs/stale.tsx"),
180+ route("repo/:repoOwner/:repoName/*", "routes/workspace/docs/repo-file.tsx"),
178181 route(":space", "routes/workspace/docs/space.tsx"),
179182 route(":space/settings", "routes/workspace/docs/space-settings.tsx"),
180183 route(":space/:page", "routes/workspace/docs/page.tsx"),
+54−1
11 import { data } from "react-router";
22
3−import type { DocEditTarget, DocMove, DocPageChange, DocRole, DocSpaceChange, NewDocPage, NewDocSpace, Result } from "@g1t/contracts";
3+import type { DocEditTarget, DocMove, DocPageChange, DocRole, DocSpaceChange, NewDocPage, NewDocSpace, Result, User } from "@g1t/contracts";
44
55 import type { Route } from "./+types/api";
66 import { workspacePeople } from "../../../lib/chat.server";
3030 if (q.get("space")) return docs.space(slug, q.get("space")!, viewer);
3131 if (q.get("embed")) return embed(slug, q.get("embed")!, viewer, url.origin);
3232 if (q.has("templates")) return docs.templates(slug, viewer);
33+ if (q.has("stale")) return docs.stalePages(slug, viewer, { repo: q.get("stale") || null });
34+ // Repositories to cite, describe or show the docs of: the workspace's that the viewer can read.
35+ if (q.has("repos")) return repositories(slug, viewer);
36+ // A citation's commit: the default branch's head, once the path is there.
37+ if (q.get("cite")) return cite(viewer, q.get("cite")!, q.get("path") ?? "");
3338 // How member keys show (`user:<id>`, `agent:<id>`): comment authors, people on a page.
3439 if (q.get("who")) return who(slug, viewer, q.get("who")!.split(",").slice(0, 100));
3540 // A member's key (`user:<id>`) by username, for adding them to a space.
6166 name?: string;
6267 description?: string | null;
6368 target?: DocEditTarget;
69+ repo?: string;
70+ id?: string;
6471 };
6572
6673 export async function action({ params, context, request }: Route.ActionArgs) {
104111 return docs.updateSpace(slug, sent.space_id ?? "", viewer, sent.space_change ?? {});
105112 case "set_member":
106113 return docs.setSpaceMember(slug, sent.space_id ?? "", viewer, sent.member ?? "", sent.role ?? null);
114+ case "mark_current":
115+ return docs.markCurrent(slug, page, viewer);
116+ case "add_repo_space":
117+ return docs.addRepoSpace(slug, viewer, sent.repo ?? "");
118+ case "remove_repo_space":
119+ return docs.removeRepoSpace(slug, viewer, sent.id ?? "");
107120 default:
108121 return { ok: false, error: { code: "invalid", message: "Unknown request." } };
109122 }
171184 return { ok: true, value: { kind: "project", title: `${repo.namespace}/${repo.name}`, subtitle: (found.value as { description?: string | null }).description ?? "Project", state: null, href: `/${repo.namespace}/${repo.name}` } };
172185 }
173186
187+/** The workspace's repositories the viewer can read, newest first: `owner/name` and its default branch. */
188+async function repositories(slug: string, viewer: User): Promise<Result<{ repo: string; default_branch: string; private: boolean }[]>> {
189+ const found = await repos.list(viewer, { namespace: slug });
190+ return {
191+ ok: true,
192+ value: found
193+ .filter((r) => !r.forkOf)
194+ .slice(0, 300)
195+ .map((r) => ({ repo: `${r.namespace}/${r.name}`.toLowerCase(), default_branch: r.defaultBranch, private: r.isPrivate })),
196+ };
197+}
198+
199+/**
200+ * A citation for `path` in `repo`, as the viewer can read it: pinned to
201+ * the default branch's head commit, once the file or folder is there (a
202+ * glob is taken as written).
203+ */
204+async function cite(viewer: User, repo: string, rawPath: string): Promise<Result<{ repo: string; path: string; ref: string | null; kind: "file" | "folder" | "glob" }>> {
205+ const [namespace, name, ...rest] = repo.trim().toLowerCase().split("/");
206+ const missing = { ok: false as const, error: { code: "not_found" as const, message: "No such repository, or you can't read it." } };
207+ if (!namespace || !name || rest.length) return missing;
208+ const path = rawPath
209+ .trim()
210+ .replace(/\\/g, "/")
211+ .split("/")
212+ .filter((p) => p && p !== ".")
213+ .join("/");
214+ if (!path || path.split("/").includes("..")) return { ok: false, error: { code: "invalid", message: "Name a file, a folder or a pattern in the repository." } };
215+ const where = { namespace, name };
216+ const root = await repos.tree(where, viewer, null, "");
217+ if (!root.ok) return missing;
218+ const head = root.value.head?.hash ?? null;
219+ if (/[*?]/.test(path)) return { ok: true, value: { repo: `${namespace}/${name}`, path, ref: head, kind: "glob" } };
220+ if (!head) return { ok: false, error: { code: "not_found", message: "That repository has no commits yet." } };
221+ const [blob, tree] = await Promise.all([repos.blob(where, viewer, head, path).catch(() => null), repos.tree(where, viewer, head, path).catch(() => null)]);
222+ if (blob?.ok) return { ok: true, value: { repo: `${namespace}/${name}`, path, ref: head, kind: "file" } };
223+ if (tree?.ok) return { ok: true, value: { repo: `${namespace}/${name}`, path, ref: head, kind: "folder" } };
224+ return { ok: false, error: { code: "not_found", message: `There is no ${path} on ${root.value.repo.defaultBranch}.` } };
225+}
226+
174227 /** Names and faces for member keys, as the workspace knows them. */
175228 async function who(slug: string, viewer: Parameters<typeof docs.page>[2], keys: string[]): Promise<Result<unknown>> {
176229 const userIds = keys.filter((k) => k.startsWith("user:")).map((k) => k.slice(5));
+31−2
11 import type { DocTemplate, DocsHome } from "@g1t/contracts";
2−import { BookOpen, FilePlus2, Plus, Search } from "lucide-react";
2+import { AlertTriangle, BookOpen, FilePlus2, FolderGit2, Plus, Search } from "lucide-react";
33 import { useState } from "react";
4−import { Form, Link, data, useNavigate, useSubmit } from "react-router";
4+import { Form, Link, data, useNavigate, useRevalidator, useSubmit } from "react-router";
55
66 import type { Route } from "./+types/home";
77 import { docsRequest, useDocsData } from "../../../components/docs/actions";
8+import { RepoDocsDialog } from "../../../components/docs/code";
89 import { PageCard, SectionTitle, SpaceCard, TemplateCard } from "../../../components/docs/parts";
910 import { DocsSidebar } from "../../../components/docs/sidebar";
1011 import { ButtonLink, EmptyState, ErrorText } from "../../../components/ui";
4546 const navigate = useNavigate();
4647 const submit = useSubmit();
4748 const [error, setError] = useState<string | null>(null);
49+ const [addingRepo, setAddingRepo] = useState(false);
50+ const { revalidate } = useRevalidator();
4851 const general = layout?.sidebar?.spaces.find((s) => s.is_default) ?? layout?.sidebar?.spaces.find((s) => canDo(s.viewer_role, "edit"));
4952 const spacesById = new Map((home?.spaces ?? []).map((s) => [s.id, s]));
5053 const create = async (template?: string) => {
7982 <p className="mt-1 max-w-xl text-sm text-muted">Specs, runbooks, decisions and onboarding, written together with your agents. Everything here is searchable, and agents read it before they work.</p>
8083 </div>
8184 <div className="flex items-center gap-2">
85+ <button type="button" onClick={() => setAddingRepo(true)} className="inline-flex h-9 items-center gap-1.5 rounded-md border border-line px-3 text-sm text-fg/80 transition-colors hover:border-line-strong hover:bg-surface hover:text-fg">
86+ <FolderGit2 size={15} /> Show a project&apos;s docs
87+ </button>
8288 <ButtonLink to={`/${slug}/-/docs/new`} variant="quiet">
8389 <Plus size={15} /> New space
8490 </ButtonLink>
115121 )}
116122 </div>
117123
124+ {home.stale.length > 0 && (
125+ <section className="mt-8">
126+ <SectionTitle
127+ action={
128+ <Link to={`/${slug}/-/docs/stale`} className="text-xs text-muted hover:text-fg">
129+ All of them
130+ </Link>
131+ }
132+ >
133+ <span className="inline-flex items-center gap-1.5">
134+ <AlertTriangle size={13} className="text-warn" /> Possibly out of date
135+ </span>
136+ </SectionTitle>
137+ <div className="grid gap-3 sm:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4">
138+ {home.stale.map((p) => (
139+ <PageCard key={p.id} page={p} space={spacesById.get(p.space_id)} />
140+ ))}
141+ </div>
142+ </section>
143+ )}
144+
118145 <section className="mt-8">
119146 <SectionTitle>{home.project ? `Recently edited · ${home.project}` : "Recently edited"}</SectionTitle>
120147 {home.recent.length ? (
164191 </div>
165192 </section>
166193
194+ <RepoDocsDialog slug={slug} open={addingRepo} onOpenChange={setAddingRepo} shown={(layout?.sidebar?.repos ?? []).map((r) => r.repo.toLowerCase())} onAdded={() => void revalidate()} />
195+
167196 {general && featured.length > 0 && (
168197 <section className="mt-10 mb-6">
169198 <SectionTitle
+5−1
2525 import { docsRequest, useDocsAction, useDocsData } from "../../../components/docs/actions";
2626 import type { DocEditorProps, Presence } from "../../../components/docs/editor";
2727 import { EditorSkeleton } from "../../../components/docs/editor-skeleton";
28+import { Describes, StaleBanner } from "../../../components/docs/code";
2829 import { CoverPicker, Discussion, HistoryDialog, IconPicker, MoveDialog, SuggestionCard, TemplateDialog, type PageThread } from "../../../components/docs/page-parts";
2930 import { Crumbs, Face, Faces } from "../../../components/docs/parts";
3031 import type { LiveStatus } from "../../../components/docs/provider";
136137 else navigate(`/${slug}/-/docs`);
137138 } else if (event.type === "suggestion.created") setSuggestions((was) => [...was.filter((s) => s.id !== event.suggestion.id), event.suggestion]);
138139 else if (event.type === "suggestion.updated") setSuggestions((was) => (event.suggestion.status === "open" ? was.map((s) => (s.id === event.suggestion.id ? event.suggestion : s)) : was.filter((s) => s.id !== event.suggestion.id)));
139− else if (event.type === "version.created") void revalidator.revalidate();
140+ else if (event.type === "version.created" || event.type === "page.staleness") void revalidator.revalidate();
140141 },
141142 [navigate, slug, revalidator],
142143 );
418419 Owners <Faces people={live.owners} size={18} />
419420 </span>
420421 )}
422+ <Describes slug={slug} describes={detail.describes} editable={editable} onSave={(describes) => send("update_page", { page_id: page.id, change: { describes } })} />
421423 {[...new Set([...live.projects, ...space.projects])].map((p) => (
422424 <Link key={p} to={`/${p}`} className="rounded-full border border-line px-2 py-0.5 font-mono text-[0.6875rem] text-muted hover:border-line-strong hover:text-fg">
423425 {p}
433435 <ErrorText>{error}</ErrorText>
434436 </div>
435437 )}
438+ {detail.staleness && <StaleBanner staleness={detail.staleness} canMark={editable} onMark={() => send("mark_current", { page_id: page.id })} />}
436439
437440 {/* Agents' suggestions, all together on a narrower screen (beside their blocks on a wide one). */}
438441 {suggestions.length > 0 && (
483486 onEvent={onEvent}
484487 onPageThreads={setThreads}
485488 renderSuggestion={(s) => <SuggestionCard suggestion={s} canDecide={editable} onDecide={decide} />}
489+ projects={[...new Set([...live.projects, ...space.projects, ...detail.describes.map((d) => d.repo)])]}
486490 />
487491 </Suspense>
488492 ) : (
+101−0
1+import type { DocRepoPage } from "@g1t/contracts";
2+import { FolderGit2, GitPullRequestArrow, PencilLine, Trash2 } from "lucide-react";
3+import { Link, data, useNavigate } from "react-router";
4+
5+import type { Route } from "./+types/repo-file";
6+import { useDocsAction } from "../../../components/docs/actions";
7+import { Crumbs } from "../../../components/docs/parts";
8+import { Markdown } from "../../../components/markdown";
9+import { ErrorText, TimeAgo } from "../../../components/ui";
10+import { Hint } from "../../../components/ui/hint";
11+import { page as pageMeta } from "../../../lib/meta";
12+import { docs } from "../../../lib/services.server";
13+import { requireUser, roleIn } from "../../../lib/session.server";
14+
15+export function meta({ loaderData: loaded, params, ...args }: Route.MetaArgs) {
16+ const file = loaded?.found.file;
17+ return pageMeta(args, { title: `${file?.title ?? "Docs"} · ${params.repoOwner}/${params.repoName} · ${params.owner} · g1t` });
18+}
19+
20+export async function loader({ params, context, request }: Route.LoaderArgs): Promise<{ found: DocRepoPage }> {
21+ const viewer = requireUser(context, request);
22+ if (!roleIn(viewer, params.owner)) throw data(null, { status: 404 });
23+ const repo = `${params.repoOwner}/${params.repoName}`;
24+ const found = await docs.repoPage(params.owner.toLowerCase(), viewer, repo, params["*"] ?? "").catch(() => null);
25+ if (!found?.ok) throw data(null, { status: 404 });
26+ return { found: found.value };
27+}
28+
29+/**
30+ * One file of a project's docs, read-only: rendered as Code renders it,
31+ * with its links and pictures pointing into the repository. Changes go
32+ * through the repository: Edit in Code opens the file there.
33+ */
34+export default function RepoDocFile({ loaderData, params }: Route.ComponentProps) {
35+ const slug = params.owner.toLowerCase();
36+ const { space, file } = loaderData.found;
37+ const navigate = useNavigate();
38+ const { send, error } = useDocsAction(slug);
39+ const [namespace, name] = space.repo.split("/") as [string, string];
40+ const folder = file.path.includes("/") ? file.path.slice(0, file.path.lastIndexOf("/")) : "";
41+ const encode = (path: string) => path.split("/").map(encodeURIComponent).join("/");
42+ return (
43+ <div className="-mt-6">
44+ <div className="sticky top-(--topbar-h) z-20 -mx-4 flex h-12 items-center gap-3 border-b border-line bg-bg/85 px-4 backdrop-blur sm:-mx-6 sm:px-6 lg:-mx-8 lg:px-8">
45+ <div className="min-w-0 grow">
46+ <Crumbs items={[{ label: "Docs", to: `/${slug}/-/docs` }, { label: space.repo }, ...file.path.split("/").map((part) => ({ label: part }))]} />
47+ </div>
48+ <Link to={file.code_href} className="inline-flex h-8 items-center gap-1.5 rounded-md border border-line px-2.5 text-xs text-fg/85 hover:bg-raised">
49+ <PencilLine size={13} /> Edit in Code
50+ </Link>
51+ {space.can_remove && (
52+ <Hint label={`Stop showing ${space.repo}'s docs in Docs`}>
53+ <button
54+ type="button"
55+ aria-label={`Stop showing ${space.repo}'s docs`}
56+ onClick={async () => {
57+ const done = await send("remove_repo_space", { id: space.id });
58+ if (done.ok) navigate(`/${slug}/-/docs`);
59+ }}
60+ className="flex size-8 items-center justify-center rounded-md text-faint hover:bg-raised hover:text-danger"
61+ >
62+ <Trash2 size={15} />
63+ </button>
64+ </Hint>
65+ )}
66+ </div>
67+ <article className="mx-auto max-w-3xl pt-10">
68+ <p className="flex flex-wrap items-center gap-x-3 gap-y-1 text-xs text-faint">
69+ <span className="inline-flex items-center gap-1.5">
70+ <FolderGit2 size={13} /> {space.repo} · {space.default_branch}
71+ </span>
72+ <span className="font-mono">{file.path}</span>
73+ {space.indexed_at && (
74+ <span>
75+ Read <TimeAgo at={space.indexed_at} />
76+ </span>
77+ )}
78+ </p>
79+ <p className="mt-3 flex items-start gap-2 rounded-lg border border-line bg-surface px-3 py-2 text-xs leading-relaxed text-muted">
80+ <GitPullRequestArrow size={14} className="mt-px shrink-0 text-faint" aria-hidden="true" />
81+ This page lives in the repository and changes through pull requests. Edit it in Code, or ask an agent to open a pull request for it.
82+ </p>
83+ {error && (
84+ <div className="mt-3">
85+ <ErrorText>{error}</ErrorText>
86+ </div>
87+ )}
88+ <div className="docs-read mt-6">
89+ <Markdown
90+ source={file.markdown}
91+ repo={{ namespace, name }}
92+ // Relative links point into the repository at its default
93+ // branch, and pictures at its files there.
94+ base={`/${space.repo}/blob/${encodeURIComponent(space.default_branch)}${folder ? `/${encode(folder)}` : ""}`}
95+ rawBase={`/${space.repo}/raw/${encodeURIComponent(space.commit ?? space.default_branch)}${folder ? `/${encode(folder)}` : ""}`}
96+ />
97+ </div>
98+ </article>
99+ </div>
100+ );
101+}
+3−3
11 import type { DocSearchHit } from "@g1t/contracts";
2−import { Search } from "lucide-react";
2+import { FolderGit2, Search } from "lucide-react";
33 import { Form, Link, data, useSubmit } from "react-router";
44
55 import type { Route } from "./+types/search";
6262 {hits === null ? (
6363 <EmptyState title="Search didn't answer">Try again in a moment.</EmptyState>
6464 ) : !q.trim() ? (
65− <p className="text-sm text-muted">Search titles and text across every space you can read. Agents search the same way, and only see what the people they answer can see.</p>
65+ <p className="text-sm text-muted">Search titles and text across every space you can read, and the projects&apos; docs shown in Docs. Agents search the same way, and only see what the people they answer can see.</p>
6666 ) : hits.length === 0 ? (
6767 <EmptyState title="Nothing found">No page you can read matches &ldquo;{q}&rdquo;.</EmptyState>
6868 ) : (
7171 <li key={hit.id}>
7272 <Link to={hit.path} className="block px-4 py-3 transition-colors hover:bg-raised">
7373 <span className="flex items-center gap-2">
74− <PageIcon icon={hit.icon} />
74+ {hit.repo_file ? <FolderGit2 size={15} className="shrink-0 text-faint" aria-hidden="true" /> : <PageIcon icon={hit.icon} />}
7575 <span className="truncate text-sm font-medium">{hit.title || "Untitled"}</span>
7676 <span className="ml-auto shrink-0 text-xs text-faint">
7777 {hit.space_name} · <TimeAgo at={hit.updated_at} />
+63−0
1+import type { DocPage } from "@g1t/contracts";
2+import { AlertTriangle } from "lucide-react";
3+import { Link, data } from "react-router";
4+
5+import type { Route } from "./+types/stale";
6+import { useDocsData } from "../../../components/docs/actions";
7+import { Faces } from "../../../components/docs/parts";
8+import { PageIcon } from "../../../components/docs/sidebar";
9+import { EmptyState, TimeAgo } from "../../../components/ui";
10+import { page } from "../../../lib/meta";
11+import { docs } from "../../../lib/services.server";
12+import { requireUser, roleIn } from "../../../lib/session.server";
13+
14+export function meta({ params, ...args }: Route.MetaArgs) {
15+ return page(args, { title: `Possibly stale · Docs · ${params.owner} · g1t` });
16+}
17+
18+export async function loader({ params, context, request }: Route.LoaderArgs): Promise<{ pages: DocPage[] | null; repo: string | null }> {
19+ const viewer = requireUser(context, request);
20+ if (!roleIn(viewer, params.owner)) throw data(null, { status: 404 });
21+ const repo = new URL(request.url).searchParams.get("repo") || null;
22+ const found = await docs.stalePages(params.owner.toLowerCase(), viewer, { repo }).catch(() => null);
23+ return { pages: found?.ok ? found.value : null, repo };
24+}
25+
26+/** Pages whose cited code changed since someone last marked them current, most recently flagged first. */
27+export default function DocsStale({ loaderData }: Route.ComponentProps) {
28+ const { pages, repo } = loaderData;
29+ const layout = useDocsData();
30+ const spaces = new Map((layout?.sidebar?.spaces ?? []).map((s) => [s.id, s.name]));
31+ return (
32+ <div className="mx-auto max-w-3xl">
33+ <h1 className="flex items-center gap-2 text-xl font-semibold tracking-tight">
34+ <AlertTriangle size={18} className="text-warn" /> Possibly stale{repo ? ` · ${repo}` : ""}
35+ </h1>
36+ <p className="mt-1 text-sm text-muted">A merged pull request or a push to a default branch changed code these pages cite. Read what changed, update the page (or ask an agent to), then mark it current.</p>
37+ <div className="mt-6">
38+ {pages === null ? (
39+ <EmptyState title="Docs didn't answer">Try again in a moment.</EmptyState>
40+ ) : pages.length === 0 ? (
41+ <EmptyState title="Everything is current">No page you can read cites code that changed since it was last checked.</EmptyState>
42+ ) : (
43+ <ul className="divide-y divide-line overflow-hidden rounded-xl border border-line bg-surface">
44+ {pages.map((p) => (
45+ <li key={p.id}>
46+ <Link to={p.path} prefetch="intent" className="flex items-center gap-3 px-4 py-2.5 transition-colors hover:bg-raised">
47+ <PageIcon icon={p.icon} />
48+ <span className="min-w-0 grow">
49+ <span className="block truncate text-sm">{p.title || "Untitled"}</span>
50+ <span className="block text-xs text-faint">
51+ {spaces.get(p.space_id) ?? p.space_slug} · edited <TimeAgo at={p.updated_at} />
52+ </span>
53+ </span>
54+ {p.owners.length > 0 && <Faces people={p.owners} size={18} />}
55+ </Link>
56+ </li>
57+ ))}
58+ </ul>
59+ )}
60+ </div>
61+ </div>
62+ );
63+}
+51−0
971971 pub repo_id: String,
972972 }
973973
974+/// The `doc.page.*` types the docs service (services/docs, TypeScript)
975+/// publishes, with no `repoId` on the event: a page may be in a private
976+/// space, so it never reaches a repository's timeline or webhooks.
977+pub const DOC_PAGE_EVENTS: [&str; 4] = ["doc.page.created", "doc.page.updated", "doc.page.archived", "doc.page.stale"];
978+
979+/// What every `doc.page.*` event carries (`DocPageEventData` in
980+/// events.ts). `doc.page.updated` adds `versionId`, `kind` and `authors`;
981+/// `doc.page.stale` adds `repoId`, `repo`, `commit`, `pull`, `paths` and
982+/// `owners` ([`DocPageStale`]).
983+#[derive(Clone, Debug, Serialize, serde::Deserialize)]
984+#[serde(rename_all = "camelCase")]
985+pub struct DocPageEvent {
986+ pub workspace: String,
987+ pub workspace_id: String,
988+ pub page_id: String,
989+ pub space_id: String,
990+ pub title: String,
991+ /// The page's address on the site.
992+ pub path: String,
993+}
994+
995+/// `doc.page.stale`: code a page cites changed.
996+#[derive(Clone, Debug, Serialize, serde::Deserialize)]
997+#[serde(rename_all = "camelCase")]
998+pub struct DocPageStale {
999+ #[serde(flatten)]
1000+ pub page: DocPageEvent,
1001+ pub repo_id: String,
1002+ /// `owner/name`.
1003+ pub repo: String,
1004+ pub commit: String,
1005+ /// The merged pull request's number, when a pull request made the change.
1006+ pub pull: Option<u32>,
1007+ pub paths: Vec<String>,
1008+ /// Member keys: `user:<id>`, `agent:<id>`.
1009+ pub owners: Vec<String>,
1010+}
1011+
9741012 #[cfg(test)]
9751013 mod tests {
9761014 use super::*;
11451183 let event: WorkspaceRenamed = serde_json::from_value(data).unwrap();
11461184 assert_eq!((event.from.as_str(), event.to.as_str()), ("a", "b"));
11471185 }
1186+
1187+ #[test]
1188+ fn a_stale_page_reads_as_published() {
1189+ let data = serde_json::json!({
1190+ "workspace": "acme", "workspaceId": "wsp_1", "pageId": "pag_1", "spaceId": "spc_1",
1191+ "title": "Exports", "path": "/acme/-/docs/general/exports-pag_1",
1192+ "repoId": "rep_1", "repo": "acme/web", "commit": "abc", "pull": 431,
1193+ "paths": ["src/export.ts"], "owners": ["user:usr_1"]
1194+ });
1195+ let event: DocPageStale = serde_json::from_value(data).unwrap();
1196+ assert_eq!((event.page.page_id.as_str(), event.pull), ("pag_1", Some(431)));
1197+ assert!(DOC_PAGE_EVENTS.contains(&"doc.page.stale"));
1198+ }
11481199 }
+13−2
189189 "team.deleted",
190190 ],
191191 ),
192+ // Pages that cite code a merge or a push to the default branch
193+ // changed become possibly out of date, and projects' docs folders shown
194+ // in Docs are read again (docs/src/staleness.ts, repo-spaces.ts).
195+ ("SUBSCRIBER_DOCS", &["git.push", "pull.merged"]),
192196 // Agents' routines that run on events: a pull request ready for
193197 // review or merged, checks or a deploy failing, an issue opened
194198 // (agents/src/triggers.ts).
228232 use super::*;
229233
230234 /// Types published on the bus that hooks are not offered.
231− const UNOFFERED: [&str; 18] = [
235+ const UNOFFERED: [&str; 22] = [
232236 "pull.mergecheck",
233237 "pull.mergeability",
234238 "deployment.review_requested",
247251 "workspace.deleting",
248252 "workspace.restored",
249253 "workspace.deleted",
254+ // Docs pages: no repository, so no repository's hooks.
255+ "doc.page.created",
256+ "doc.page.updated",
257+ "doc.page.archived",
258+ "doc.page.stale",
250259 ];
251260
252261 fn published(kind: &str) -> bool {
279288 names.sort_unstable();
280289 names.dedup();
281290 assert_eq!(names.len(), count);
282− assert_eq!(count, 14);
291+ assert_eq!(count, 15);
283292 }
284293
285294 #[test]
296305 assert!(routed("SUBSCRIBER_SEARCH", "user.updated"));
297306 assert!(routed("SUBSCRIBER_REPOS", "user.deleting"));
298307 assert!(!routed("SUBSCRIBER_REPOS", "git.push"));
308+ assert!(routed("SUBSCRIBER_DOCS", "pull.merged"));
309+ assert!(!routed("SUBSCRIBER_DOCS", "pull.opened"));
299310 // Every subscriber follows what moves or removes a repository.
300311 for (binding, _) in ROUTES {
301312 for kind in LIFECYCLE {
+17−3
1010 // which keeps repositories in the git store (gitstore/server.mjs);
1111 // - EMAIL (Email Sending) becomes a service binding to workers/mail;
1212 // - the packages service keeps files in S3-compatible storage (RustFS)
13−// instead of R2, and the repos service its nightly backups (a bucket of
14−// their own, BACKUP_S3_BUCKET);
13+// instead of R2, the repos service its nightly backups (a bucket of
14+// their own, BACKUP_S3_BUCKET), and the docs service the files in pages
15+// (DOCS_S3_BUCKET);
1516 // - services that are off in this phase (agents, the context hub, the
1617 // g1t.page dispatcher, model proxy) are bound to workers/off instead, and
1718 // events stop queueing work for them;
2122 // Environment: PUBLIC_URL, GITSTORE_URL, GITSTORE_SECRET, MAIL_URL,
2223 // ACTIONS_KEY, INTEGRATIONS_KEY, WEBHOOKS_KEY, IDENTITY_KEY, USERCONTENT_KEY,
2324 // USERCONTENT_URL,
24−// PACKAGES_TOKEN_SECRET, S3_ENDPOINT, S3_BUCKET, BACKUP_S3_BUCKET, S3_REGION,
25+// PACKAGES_TOKEN_SECRET, S3_ENDPOINT, S3_BUCKET, BACKUP_S3_BUCKET, DOCS_S3_BUCKET, S3_REGION,
2526 // S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_PUBLIC_ENDPOINT, and optionally
2627 // your own GitHub App: GITHUB_APP_ID, GITHUB_APP_SLUG, GITHUB_APP_CLIENT_ID,
2728 // GITHUB_APP_CLIENT_SECRET, GITHUB_APP_PRIVATE_KEY, GITHUB_APP_WEBHOOK_SECRET.
265266 S3_SECRET_ACCESS_KEY: process.env.S3_SECRET_ACCESS_KEY ?? "",
266267 });
267268 }
269+ // Files people put in Docs pages go to a bucket of their own on the same
270+ // S3-compatible store, instead of the FILES R2 bucket (services/docs
271+ // src/files.ts, `s3FileStore`).
272+ if (hosted.name === "g1t-docs-service") {
273+ Object.assign(config.vars, {
274+ DOCS_FILES: "s3",
275+ DOCS_S3_ENDPOINT: process.env.S3_ENDPOINT ?? "http://rustfs:9000",
276+ DOCS_S3_BUCKET: process.env.DOCS_S3_BUCKET ?? "g1t-docs-files",
277+ DOCS_S3_REGION: process.env.S3_REGION ?? "us-east-1",
278+ DOCS_S3_ACCESS_KEY_ID: process.env.S3_ACCESS_KEY_ID ?? "",
279+ DOCS_S3_SECRET_ACCESS_KEY: process.env.S3_SECRET_ACCESS_KEY ?? "",
280+ });
281+ }
268282 // Nothing to deploy to: deployments are off (no Cloudflare API token).
269283 if (hosted.name === "g1t-deployments") delete config.vars.CUSTOM_HOSTNAMES_ZONE_ID;
270284
+6−3
7070 # Packs kept for fresh clones, so the next clone of the same commit
7171 # does not rebuild one; storage-setup expires them after 7 days.
7272 PACK_S3_BUCKET: ${PACK_S3_BUCKET:-g1t-git-packs}
73+ # Images and files people put in Docs pages.
74+ DOCS_S3_BUCKET: ${DOCS_S3_BUCKET:-g1t-docs-files}
7375 volumes:
7476 - g1t-data:/data
7577 - g1t-secrets:/secrets:ro
132134 retries: 20
133135 restart: unless-stopped
134136
135− # Makes the buckets once, then exits: packages' files, backups, and
136− # clone packs, with a lifecycle rule on the packs' bucket that deletes
137+ # Makes the buckets once, then exits: packages' files, backups, clone
138+ # packs and Docs pages' files, with a lifecycle rule on the packs' bucket that deletes
137139 # packs 7 days old and uploads left unfinished after a day.
138140 storage-setup:
139141 image: ${AWS_CLI_IMAGE:-amazon/aws-cli:2.37.10}
145147 - -c
146148 - >-
147149 set -e;
148− for bucket in "$$S3_BUCKET" "$$BACKUP_S3_BUCKET" "$$PACK_S3_BUCKET"; do
150+ for bucket in "$$S3_BUCKET" "$$BACKUP_S3_BUCKET" "$$PACK_S3_BUCKET" "$$DOCS_S3_BUCKET"; do
149151 aws s3api head-bucket --bucket "$$bucket" >/dev/null 2>&1 || aws s3api create-bucket --bucket "$$bucket" >/dev/null;
150152 done;
151153 aws s3api put-bucket-lifecycle-configuration --bucket "$$PACK_S3_BUCKET" --lifecycle-configuration
159161 S3_BUCKET: ${S3_BUCKET:-g1t-packages}
160162 BACKUP_S3_BUCKET: ${BACKUP_S3_BUCKET:-g1t-backups}
161163 PACK_S3_BUCKET: ${PACK_S3_BUCKET:-g1t-git-packs}
164+ DOCS_S3_BUCKET: ${DOCS_S3_BUCKET:-g1t-docs-files}
162165
163166 mailpit:
164167 image: axllent/mailpit:latest
+2−1
138138 "secrets": [],
139139 "setup": [
140140 "The D1 database, before the first deploy: npx wrangler d1 create g1t-docs, then put its id in services/docs/wrangler.jsonc",
141− "The R2 bucket for files in pages: npx wrangler r2 bucket create g1t-docs-files"
141+ "The R2 bucket for files in pages: npx wrangler r2 bucket create g1t-docs-files",
142+ "The queue the events service sends it merges and pushes on (pages whose cited code changed, projects' docs): npx wrangler queues create g1t-events-docs"
142143 ],
143144 "self_host": "run"
144145 },
+25−1
112112 | `services/events` | Rust | Queues (producer and fan-out) | Runs unchanged; the off services' queues are not produced to |
113113 | `services/projects` | TS | Queue consumer | Runs unchanged |
114114 | `services/chat` | TS | Durable Objects (one room per channel, WebSocket hibernation), KV `AVATARS` (custom emoji images, under `emoji/`) | Runs unchanged; workerd runs its Durable Objects, and the site serves emoji images from the same KV |
115−| `services/docs` | TS | Durable Objects (one room per page: the Yjs document, WebSocket hibernation, SQLite storage, alarms), **R2** (`FILES`, files in pages, behind the `FileStore` interface in `src/files.ts`), D1 with FTS5 | Runs unchanged; workerd runs its Durable Objects, and `FILES` is the local R2 bucket Wrangler keeps on disk (an S3 `FileStore` for RustFS is the next step) |
115+| `services/docs` | TS | Durable Objects (one room per page: the Yjs document, WebSocket hibernation, SQLite storage, alarms), **R2** (`FILES`, files in pages, behind the `FileStore` interface in `src/files.ts`), D1 with FTS5, a queue (`g1t-events-docs`: merges and pushes, for pages whose cited code changed and projects' docs) | Runs unchanged; workerd runs its Durable Objects, and files in pages go to RustFS (`DOCS_FILES=s3`, the `g1t-docs-files` bucket; see "Files in Docs pages") |
116116 | `services/notify` | TS | Durable Objects (one feed per person: WebSocket hibernation, SQLite storage); outbound HTTPS to browsers' push services | Runs unchanged; browser push needs a VAPID key pair (`node scripts/ops/vapid-keys.mjs`), else notifications are live in open tabs only |
117117 | `services/agents` | TS | Durable Objects (one desk per agent, alarms) | Runs unchanged; replies reach a model through the `MODELS` binding (the model proxy), which is off, so an agent answers with a short apology |
118118 | `services/search` | Rust | Queues (events and its own jobs); FTS5 | Runs unchanged |
316316 `VECTORS` is missing (`index.ts:919`), so phase 3 can first run context
317317 with neither bound.
318318
319+### Files in Docs pages
320+
321+Images and files people put in Docs pages are kept behind the `FileStore`
322+interface (`services/docs/src/files.ts`). Hosted g1t uses R2 (`FILES`). A
323+self-hosted g1t keeps them in any S3-compatible store (RustFS in the compose
324+file, MinIO, Ceph, Garage, AWS S3) with `s3FileStore`: requests are signed
325+with AWS Signature Version 4 by hand over WebCrypto (`src/sigv4.ts`, checked
326+against AWS's published test vectors), with no SDK. It is chosen by settings
327+on the docs service:
328+
329+| Setting | What it is |
330+| --- | --- |
331+| `DOCS_FILES` | `s3` to use an S3-compatible store; anything else (or unset) uses the `FILES` R2 binding. |
332+| `DOCS_S3_ENDPOINT` | The store's address, e.g. `http://rustfs:9000` or `https://s3.eu-west-1.amazonaws.com`. |
333+| `DOCS_S3_BUCKET` | The bucket; `g1t-docs-files` in the compose file, which `storage-setup` makes. |
334+| `DOCS_S3_REGION` | The region it signs for; `us-east-1` by default (RustFS and MinIO accept it). |
335+| `DOCS_S3_ACCESS_KEY_ID`, `DOCS_S3_SECRET_ACCESS_KEY` | The keys. Keep them as secrets (`wrangler secret put`, or the compose file's environment). |
336+| `DOCS_S3_VIRTUAL_HOSTED` | `true` for `https://<bucket>.<host>/<key>` addresses; path style (`<endpoint>/<bucket>/<key>`) otherwise, which every compatible store takes. |
337+
338+`deploy/self-host/configs.mjs` sets them from the compose file's `S3_*`
339+settings and `DOCS_S3_BUCKET`. With `DOCS_FILES=s3` and a setting missing,
340+the first upload says which. Files are served the same way either way: the
341+site's usercontent origin asks the docs service for `/files/<key>`.
342+
319343 ### Models
320344
321345 Already portable. The model proxy (`services/models`) and the runner's
+60−6
521521 which checks the role; a passage's comment is a mark on its text.
522522 - **Files** go to R2 (`g1t-docs-files`) behind a small store interface and
523523 are served from the usercontent origin at `/docs-files/<key>`, 256
524− random bits per file.
524+ random bits per file. Self-hosted, the same interface keeps them in any
525+ S3-compatible store (`DOCS_FILES=s3`, SigV4 by hand: `src/sigv4.ts`).
526+- **Citations and staleness** (`src/citations.ts`, `src/staleness.ts`,
527+ tables `citations` and `page_changes`). A page cites code from its text
528+ (the editor's `citation` chips: a repository, a path or glob, what kind
529+ of thing (path, symbol, endpoint, env var) and the commit it was cited
530+ at; and any link to `/<owner>/<repo>/blob|tree/<ref>/<path>`), rebuilt
531+ on each save, and from its header's "Describes" list. The docs service
532+ consumes `g1t-events-docs` (`SUBSCRIBER_DOCS`: `git.push`,
533+ `pull.merged`). For a repository some live page cites, it asks what
534+ changed as g1t itself (a merged pull request's files from work; a push
535+ to the default branch by comparing `before..after` in repos), matches the
536+ cited paths, and records one row per page and commit (the push and the
537+ pull request of one merge land on the same row; the pull request names
538+ it). A new row notifies the page's owners (naming the change only to
539+ those who can read the repository), tells open editors to reload, and
540+ publishes `doc.page.stale`. The page shows a banner with the newest
541+ change the reader can read ("a change you can't see" otherwise); Mark as
542+ current (edit role) clears every open row; the sidebar, home and cards
543+ show it. Agents get `stalePagesForAgent` (changes only in repositories
544+ their person can read) and mark a page current with an edit or
545+ suggestion carrying `marks_current`.
546+- **Projects' docs** (`src/repo-spaces.ts`, tables `repo_spaces`,
547+ `repo_files`, `repo_files_fts`). A member adds a repository they can
548+ read; its README and `docs/**/*.md` on the default branch are read with
549+ `listFiles` and `rawBlobs` (only blobs whose hash changed), kept as
550+ Markdown with FTS, and read again on every push to the default branch.
551+ Each reader sees the ones `ReposApi.readable` says they can read. Shown
552+ read-only at `/<workspace>/-/docs/repo/<owner>/<name>/<path>`; "Edit in
553+ Code" opens the file (Code has no file editor yet; when it has one, or a
554+ "propose a change" flow, the button opens that instead).
555+- **Events.** `doc.page.created`, `doc.page.updated` (when history
556+ records a version: at most every ten minutes of editing, and every agent
557+ edit, accepted suggestion and restore, with its authors),
558+ `doc.page.archived` and `doc.page.stale`, published with no `repoId` so a
559+ page never reaches a repository's timeline or hooks (types in
560+ `packages/contracts/src/events.ts` and `g1t_contracts::events`). Not
561+ offered to webhooks yet.
525562
526563 Decided: **not git, for now.** Spaces were planned as git repositories.
527564 D1 + Durable Objects ships live collaboration, comments, suggestions and
528565 search without a git write per keystroke burst, and Markdown export (a
529566 page, or a space as a zip in its tree's folders) keeps the content
530−portable. Repository-backed spaces (a project's `docs/` as a read-only
531−space, edits as pull requests) remain the next step for repository docs.
567+portable. A project's `docs/` is shown as a read-only space (below);
568+editing it from Docs as a pull request is the next step.
532569 For self-hosting, the Durable Object, R2 and D1 sit behind the room,
533570 `FileStore` and SQL; nothing above them depends on Cloudflare.
534571
543580 doing; the thread link it cites is `<conversation path>?thread=<id>`, which
544581 **Copy link to thread** also gives.
545582
546−Not built yet: citations and staleness, the documenter agent, repository docs as spaces, `doc.page.*` events and
547−indexing pages in `services/context`.
583+Not built yet: the documenter agent (it reads `stalePagesForAgent` and
584+updates with `marks_current`; the routine and its `doc.page.stale` trigger
585+are the agents service's), editing a project's docs from Docs as a pull
586+request, and indexing pages in `services/context`.
548587
588+**Pages in `services/context`: what it needs.** The context hub indexes
589+items per project, and decides who sees one by the project's privacy
590+(`private` and membership). A Docs page's access is per space (private and
591+team spaces, listed members), which that model can't express, so indexing
592+pages there as they are would show private spaces' pages to every member.
593+Doing it right needs: an ingest RPC on context for documents with an
594+access key (`docs:<space id>`), a check at query time that asks the docs
595+service which spaces the reader (and audience) can read
596+(`spacesForAgent` already answers that), and a `doc.page.updated`
597+consumer that re-embeds the page's Markdown (`SUBSCRIBER_CONTEXT` would
598+route `doc.page.*`). Until then agents reach pages through the docs tools
599+(`searchForAgent`, `pageMarkdown`), which apply exactly those rules.
600+Projects' docs folders are already in context: it reads each project's
601+`docs/` on every push.
602+
549603 ## Chat
550604
551605 Channels, direct messages, threads, reactions, read state and the
11361190 | `services/notify` (new, TS) | One feed per person: live notifications and unread counts over each tab's socket, browser push (VAPID), preferences, presence, status and Do Not Disturb; one presence room per workspace. Durable Object SQLite storage, no D1. |
11371191 | `services/chat` (new, TS) | Channels, members, messages, threads, reactions, read state; one Durable Object per channel for live delivery with WebSocket hibernation. |
11381192 | `services/agents` (new, TS) | Agent definitions and versions, the desk Durable Object per agent, the coordinator Durable Object per workspace (claims), replies (the no-sandbox model loop over g1t MCP). |
1139−| `services/docs` (new, TS) | Spaces, pages, the page Durable Object (Yjs), history, suggestions, comments, templates, search (FTS5), files (R2); citations and staleness to come. |
1193+| `services/docs` (new, TS) | Spaces, pages, the page Durable Object (Yjs), history, suggestions, comments, templates, search (FTS5), files (R2, or S3 self-hosted), citations and staleness (queue `g1t-events-docs`), projects' docs folders, `doc.page.*` events. |
11401194 | `services/work` | Tasks and task links beside `agent_runs` (which gains `task_id`); `agent_messages` widened to task addresses. |
11411195 | `services/runner` | Resumable sessions: transcript save and restore in R2, `--resume`, the steer hook reading task threads, claims checked at start and widened from diffs. |
11421196 | `services/context` | Indexes doc pages and channel decisions; serves them to replies and sessions. |
+192−2
139139 owners: MemberProfile[];
140140 /** The first lines of its text, for cards. */
141141 excerpt: string;
142+ /** Whether code it cites changed since someone last marked it current: possibly out of date. */
143+ stale: boolean;
144+};
145+
146+// ── Citations and staleness ───────────────────────────────────────────────
147+//
148+// A page can cite code: a path (a file, a folder, or a glob like
149+// `src/export/**`) in a repository, optionally naming what at that path it
150+// describes (a symbol, an endpoint, an environment variable). Citations
151+// come from the page's text (the editor's citation chips, and links to
152+// files in a repository: `/<owner>/<repo>/blob/<ref>/<path>`) and from
153+// the page's header ("Describes"). When a merged pull request or a push
154+// to a repository's default branch changes a cited path, the page is
155+// marked possibly out of date with that change, until someone with edit
156+// access marks it current again (or an agent updates it).
157+
158+/** What a citation names at its path. */
159+export type DocCitationKind = "path" | "symbol" | "endpoint" | "env";
160+export const DOC_CITATION_KINDS: readonly DocCitationKind[] = ["path", "symbol", "endpoint", "env"];
161+
162+export const DOC_CITATION_KIND_LABELS: Record<DocCitationKind, string> = {
163+ path: "A file or folder",
164+ symbol: "A symbol",
165+ endpoint: "An endpoint",
166+ env: "An environment variable",
167+};
168+
169+export type DocCitation = {
170+ /** `owner/name`, lowercased. */
171+ repo: string;
172+ /** A file, a folder (everything under it), or a glob (`*`, `**`, `?`). */
173+ path: string;
174+ kind: DocCitationKind;
175+ /** The symbol, endpoint (`POST /v1/export`) or variable (`EXPORT_BUCKET`), for those kinds. */
176+ label: string | null;
177+ /** The commit it was cited at, when known. */
178+ ref: string | null;
179+ /** From the page's text, or from its header's "Describes". */
180+ source: "body" | "header";
181+};
182+
183+/** One entry of a page's "Describes": a repository and a path in it. */
184+export type DocDescribes = { repo: string; path: string };
185+
186+/** A change that made a page possibly out of date. */
187+export type DocStaleChange = {
188+ /**
189+ * False when the viewer can't read the repository: then `repo`, `pull`,
190+ * `commit` and `paths` are blank, and the page only says that a change
191+ * they can't see touched code it cites.
192+ */
193+ visible: boolean;
194+ repo: string | null;
195+ commit: string | null;
196+ /** The merged pull request, when the change came from one. */
197+ pull: { number: number; title: string | null } | null;
198+ /** The cited paths it changed (the changed files, at most 20). */
199+ paths: string[];
200+ /** When it was noticed. RFC 3339. */
201+ at: string;
202+};
203+
204+/** Why a page is possibly out of date: every change since it was last marked current, newest first. */
205+export type DocStaleness = { since: string; changes: DocStaleChange[] };
206+
207+/** A page an agent may bring up to date, with what changed. */
208+export type DocStalePage = {
209+ page: DocPageRef & { updated_at: string };
210+ space: { id: string; slug: string; name: string; agent_mode: DocAgentMode };
211+ can: DocAgentAbilities;
212+ owners: MemberProfile[];
213+ citations: DocCitation[];
214+ /** Changes in repositories the viewer (and audience) can read; newest first. */
215+ changes: DocStaleChange[];
216+ since: string;
217+};
218+
219+// ── A project's docs ──────────────────────────────────────────────────────
220+//
221+// A repository's `docs/` folder (and README.md), shown read-only in Docs
222+// next to the workspace's spaces and found by the same search. It is read
223+// from the default branch and kept up to date on every push to it. Each
224+// reader sees only the repositories they can read. Changes go through the
225+// repository: "Edit in Code" opens the file.
226+
227+export type DocRepoFile = {
228+ /** From the repository's root: `docs/guide/setup.md`, `README.md`. */
229+ path: string;
230+ /** Its first heading, or its file name. */
231+ title: string;
232+};
233+
234+export type DocRepoSpace = {
235+ id: string;
236+ /** `owner/name`, as the repository is named now. */
237+ repo: string;
238+ default_branch: string;
239+ /** The commit it was read at; null until it has been. */
240+ commit: string | null;
241+ indexed_at: string | null;
242+ added_by: MemberProfile;
243+ files: DocRepoFile[];
244+ /** Whether the viewer may stop showing it (whoever added it, or an owner). */
245+ can_remove: boolean;
142246 };
143247
248+export type DocRepoPage = {
249+ space: DocRepoSpace;
250+ file: DocRepoFile & {
251+ markdown: string;
252+ /** `/<workspace>/-/docs/repo/<owner>/<name>/<path>`. */
253+ href: string;
254+ /** The file in Code, on the default branch. */
255+ code_href: string;
256+ };
257+};
258+
144259 /** A page as the sidebar's tree lists it: flat, ordered by `position` within each parent. */
145260 export type DocTreeNode = {
146261 id: string;
149264 title: string;
150265 icon: string | null;
151266 slug: string;
267+ /** Possibly out of date (see `DocPage.stale`). */
268+ stale?: boolean;
152269 };
153270
154271 export type DocsSidebarSpace = DocSpace & { pages: DocTreeNode[] };
161278 /** Whether the viewer may make spaces (members of the workspace may). */
162279 can_create_space: boolean;
163280 trash_count: number;
281+ /** Pages the viewer can read that are possibly out of date. */
282+ stale_count: number;
283+ /** Projects' docs folders shown in Docs, those whose repository the viewer can read. */
284+ repos: DocRepoSpace[];
164285 };
165286
166287 export type DocsHome = {
168289 recent: DocPage[];
169290 /** Pages the viewer made or owns. */
170291 mine: DocPage[];
292+ /** Pages possibly out of date, most recently flagged first. */
293+ stale: DocPage[];
171294 spaces: DocSpace[];
172295 /** Every project some space or page is linked to, for the filter. */
173296 projects: string[];
245368 /** When the viewer last opened it, before now. */
246369 last_viewed_at: string | null;
247370 suggestions: DocSuggestion[];
371+ /** Code the page cites: from its text and its header. */
372+ citations: DocCitation[];
373+ /** The header's "Describes" list. */
374+ describes: DocDescribes[];
375+ /** Why it is possibly out of date; null when it isn't. */
376+ staleness: DocStaleness | null;
248377 };
249378
250379 export type DocSearchHit = DocPageRef & {
253382 snippet: string;
254383 updated_at: string;
255384 projects: string[];
385+ /** Set for a file from a project's docs folder (then `id` is `repo:<space>:<path>` and `path` its address in Docs). */
386+ repo_file?: { repo: string; path: string } | null;
256387 };
257388
258389 export type DocTemplate = {
295426 /** Everyone in the workspace, e.g. a public channel: only workspace-wide spaces. */
296427 | { kind: "workspace" };
297428
429+/** An agent's edit to a page. */
430+export type DocAgentEdit = {
431+ target: DocEditTarget;
432+ markdown: string;
433+ note?: string | null;
434+ /** The edit brings the page up to date with the code it cites: once applied, the page is no longer possibly out of date. */
435+ marks_current?: boolean | null;
436+};
437+
298438 /** What `apply_edit` did: applied it, or filed a suggestion because it may not edit there. */
299439 export type DocAgentEditResult =
300440 | { mode: "applied"; version_id: string | null; page: DocPageRef }
333473 projects?: string[];
334474 /** Member keys (`user:<id>`, `agent:<id>`). */
335475 owners?: string[];
476+ /** Replaces the header's "Describes" list (at most 20). */
477+ describes?: DocDescribes[];
336478 };
337479
338480 /** Where a page goes: under `parent_id` (null for the top of the space), before `before_id` (null for the end). */
383525 | { type: "page.archived"; page_id: string }
384526 | { type: "suggestion.created" | "suggestion.updated"; suggestion: DocSuggestion }
385527 | { type: "version.created"; version: DocVersion }
528+ /** Whether it is possibly out of date changed: ask for the page again (what each reader sees of why depends on what they can read). */
529+ | { type: "page.staleness" }
386530 /** The viewer's role changed, or their access ended (`role` null). */
387531 | { type: "access"; role: DocRole | null };
388532
454598 thread(workspace: string, pageId: string, viewer: User, action: DocThreadAction): Promise<Result<unknown>>;
455599 threads(workspace: string, pageId: string, viewer: User): Promise<Result<DocThread[]>>;
456600
601+ /** Clears "possibly out of date": the page says what the code does now. Edit role. */
602+ markCurrent(workspace: string, pageId: string, viewer: User): Promise<Result<boolean>>;
603+ /** Pages the viewer can read that are possibly out of date, most recently flagged first; `repo` (`owner/name`) narrows to changes there. */
604+ stalePages(workspace: string, viewer: User, options?: { repo?: string | null }): Promise<Result<DocPage[]>>;
605+
606+ /**
607+ * Shows a repository's `docs/` folder and README.md in Docs, read from
608+ * its default branch. Any member who can read the repository may; the
609+ * files are read at once and again on every push to the default branch.
610+ */
611+ addRepoSpace(workspace: string, viewer: User, repo: string): Promise<Result<DocRepoSpace>>;
612+ /** Stops showing it. Whoever added it, or a workspace owner. */
613+ removeRepoSpace(workspace: string, viewer: User, id: string): Promise<Result<boolean>>;
614+ /** One file of a project's docs, for a viewer who can read the repository; not found otherwise. */
615+ repoPage(workspace: string, viewer: User, repo: string, path: string): Promise<Result<DocRepoPage>>;
616+
457617 // ── Agents (services/agents) ─────────────────────────────────────────
458618 //
459619 // Each takes the agent and the person it acts for (`viewer`, the asker).
469629 /** Full-text search over pages every reader can read; at most 20, best first. */
470630 searchForAgent(workspace: string, agentId: string, viewer: User, query: DocSearchQuery, audience?: DocAudience | null): Promise<Result<DocSearchHit[]>>;
471631 /** Files a tracked suggestion; people with edit access accept or reject it inline. Needs the viewer's comment role. Notifies the page's owners. */
472− suggestEdit(workspace: string, agentId: string, viewer: User, pageId: string, edit: { target: DocEditTarget; markdown: string; note?: string | null }): Promise<Result<DocSuggestion>>;
632+ suggestEdit(workspace: string, agentId: string, viewer: User, pageId: string, edit: DocAgentEdit): Promise<Result<DocSuggestion>>;
473633 /**
474634 * Applies an edit to the live document when the space lets agents edit
475635 * and the viewer can edit; otherwise files it as a suggestion (and says
476636 * so in `mode`). Attributed to the agent in the page's history.
477637 */
478− applyEdit(workspace: string, agentId: string, viewer: User, pageId: string, edit: { target: DocEditTarget; markdown: string; note?: string | null }): Promise<Result<DocAgentEditResult>>;
638+ applyEdit(workspace: string, agentId: string, viewer: User, pageId: string, edit: DocAgentEdit): Promise<Result<DocAgentEditResult>>;
479639 /**
480640 * A new page, written by the agent: in `space_id` (the General space
481641 * when null), under `parent_id`. Needs the viewer's edit role there
490650 ): Promise<Result<DocPageRef>>;
491651 /** A page's comment threads, for an agent asked about them. */
492652 threadsForAgent(workspace: string, agentId: string, viewer: User, pageId: string, audience?: DocAudience | null): Promise<Result<DocThread[]>>;
653+ /**
654+ * Pages possibly out of date that the agent may read for the viewer (and
655+ * audience), each with the changes that made it so: for a documenter
656+ * routine to bring them up to date. At most 50, most recently flagged
657+ * first.
658+ *
659+ * - `repo` (`owner/name`): only pages made stale by a change there.
660+ * - `since` (RFC 3339): only pages flagged at or after it.
661+ *
662+ * Changes in repositories the viewer can't read are left out, and a page
663+ * whose every change is one of those is left out too: an agent never
664+ * learns of code its person can't see. To update a page, read it
665+ * (`pageMarkdown`), then `applyEdit` or `suggestEdit` with
666+ * `marks_current: true`: the page is marked current when the edit is
667+ * applied (at once, or when a person accepts the suggestion).
668+ */
669+ stalePagesForAgent(
670+ workspace: string,
671+ agentId: string,
672+ viewer: User,
673+ options?: { repo?: string | null; since?: string | null },
674+ audience?: DocAudience | null,
675+ ): Promise<Result<DocStalePage[]>>;
493676 };
494677
495678 async function rpc<T>(service: ServiceBinding, method: string, args: object): Promise<T> {
535718 acceptAll: (workspace, pageId, viewer) => call("accept_all", { workspace, page_id: pageId, viewer }),
536719 thread: (workspace, pageId, viewer, action) => call("thread", { workspace, page_id: pageId, viewer, action }),
537720 threads: (workspace, pageId, viewer) => call("threads", { workspace, page_id: pageId, viewer }),
721+ markCurrent: (workspace, pageId, viewer) => call("mark_current", { workspace, page_id: pageId, viewer }),
722+ stalePages: (workspace, viewer, options) => call("stale_pages", { workspace, viewer, repo: options?.repo ?? null }),
723+ addRepoSpace: (workspace, viewer, repo) => call("add_repo_space", { workspace, viewer, repo }),
724+ removeRepoSpace: (workspace, viewer, id) => call("remove_repo_space", { workspace, viewer, id }),
725+ repoPage: (workspace, viewer, repo, path) => call("repo_page", { workspace, viewer, repo, path }),
538726 spacesForAgent: (workspace, agentId, viewer, audience) => call("spaces_for_agent", { workspace, agent_id: agentId, viewer, audience: audience ?? null }),
539727 pageMarkdown: (workspace, agentId, viewer, pageId, audience) =>
540728 call("page_markdown", { workspace, agent_id: agentId, viewer, page_id: pageId, audience: audience ?? null }),
544732 createPageAsAgent: (workspace, agentId, viewer, input) => call("create_page_as_agent", { workspace, agent_id: agentId, viewer, input }),
545733 threadsForAgent: (workspace, agentId, viewer, pageId, audience) =>
546734 call("threads_for_agent", { workspace, agent_id: agentId, viewer, page_id: pageId, audience: audience ?? null }),
735+ stalePagesForAgent: (workspace, agentId, viewer, options, audience) =>
736+ call("stale_pages_for_agent", { workspace, agent_id: agentId, viewer, repo: options?.repo ?? null, since: options?.since ?? null, audience: audience ?? null }),
547737 };
548738 }
+35−0
464464 sandbox: string;
465465 metrics: Record<string, unknown> | null;
466466 };
467+ /**
468+ * Docs (services/docs): a page was made. Published with no `repoId`, so
469+ * a page (which may be in a private space) never reaches a repository's
470+ * timeline or webhooks; `actor` is the user's or agent's id. Readers
471+ * check access with the docs service before showing anything of it.
472+ */
473+ "doc.page.created": DocPageEventData;
474+ /**
475+ * A page's content changed: at most once per page every ten minutes of
476+ * editing (when its history records a version), and for every agent
477+ * edit, accepted suggestion and restore. `authors` are member keys
478+ * (`user:<id>`, `agent:<id>`) of everyone whose changes are in it.
479+ */
480+ "doc.page.updated": DocPageEventData & { versionId: string; kind: "edit" | "agent" | "suggestion" | "restore"; authors: string[] };
481+ /** A page went to the trash (with every page under it; one event for the page asked about). */
482+ "doc.page.archived": DocPageEventData;
483+ /**
484+ * A page became possibly out of date: a merged pull request or a push to
485+ * a repository's default branch changed code it cites. `repoId` is in
486+ * `data`, not on the event, for the same reason as above. `owners` are
487+ * member keys; an agent that owns the page can update it
488+ * (`stalePagesForAgent` in docs.ts).
489+ */
490+ "doc.page.stale": DocPageEventData & { repoId: string; repo: string; commit: string; pull: number | null; paths: string[]; owners: string[] };
491+};
492+
493+/** What every `doc.page.*` event carries. */
494+export type DocPageEventData = {
495+ workspace: string;
496+ workspaceId: string;
497+ pageId: string;
498+ spaceId: string;
499+ title: string;
500+ /** The page's address on the site. */
501+ path: string;
467502 };
468503
469504 export type EventType = keyof EventPayloads;
+1−1
11 const ALPHABET = "0123456789abcdefghjkmnpqrstvwxyz";
22
3−export type IdPrefix = "usr" | "ses" | "tok" | "key" | "rep" | "int" | "att" | "evt" | "dpl" | "prj" | "dom" | "dep" | "dst" | "chn" | "msg" | "agt" | "arp" | "asn" | "mem" | "rtn" | "drf" | "spc" | "pag" | "ver" | "thr" | "cmt" | "sug" | "tpl" | "fil";
3+export type IdPrefix = "usr" | "ses" | "tok" | "key" | "rep" | "int" | "att" | "evt" | "dpl" | "prj" | "dom" | "dep" | "dst" | "chn" | "msg" | "agt" | "arp" | "asn" | "mem" | "rtn" | "drf" | "spc" | "pag" | "ver" | "thr" | "cmt" | "sug" | "tpl" | "fil" | "rds";
44
55 let lastMs = 0;
66 let lastCounter = 0;
+16−0
191191 const blocks = p.blocks.map((b) => `${b.id} ${b.type}${b.level ? ` ${b.level}` : ""}`).join(", ");
192192 return `# ${where(p.page)}\nSpace: ${p.space.name}; ${can}. Updated ${p.page.updated_at.slice(0, 16)}.\nTop-level blocks: ${blocks}\n\n${p.markdown}`;
193193 },
194+ async stale(viewer, audience, repo) {
195+ const found = await docs.stalePagesForAgent(workspace, agentId, viewer, { repo }, audience);
196+ if (!found.ok) return null;
197+ if (!found.value.length) return "No pages are marked possibly out of date.";
198+ return found.value
199+ .map((s) => {
200+ const changes = s.changes
201+ .filter((c) => c.visible)
202+ .slice(0, 3)
203+ .map((c) => `${c.repo}${c.pull ? `#${c.pull.number}${c.pull.title ? ` (${c.pull.title})` : ""}` : `@${String(c.commit ?? "").slice(0, 8)}`} changed ${c.paths.slice(0, 5).join(", ")}`)
204+ .join("; ");
205+ const can = s.can.edit ? "you can edit" : s.can.suggest ? "you can suggest" : "read only";
206+ return `- ${where(s.page)} (${can}), since ${s.since.slice(0, 10)}: ${changes}`;
207+ })
208+ .join("\n");
209+ },
194210 async edit(viewer, pageId, edit, suggestOnly) {
195211 const done = suggestOnly
196212 ? await docs.suggestEdit(workspace, agentId, viewer, pageId, edit).then((r) => (r.ok ? { ok: true as const, value: { mode: "suggested" as const, suggestion: r.value, page: null } } : r))
+1−1
3535 match: /\b(docs|documentation|pages|runbooks?)\b.*\b(change|changes|merge|merges|wrong|current|up to date|stale)\b/i,
3636 routine: (duty) => ({
3737 name: "Keep the docs current",
38− instructions: `When a pull request is merged, keep the docs true (${trimmed(duty)}). Find the pages it makes wrong or incomplete: pages marked possibly out of date by this change first, then search_docs for what it changed. Update each with edit_page (it becomes a suggestion where you can't edit), citing the pull request. Post a short list of what you changed here; say so if nothing needed changing.`,
38+ instructions: `When a pull request is merged, keep the docs true (${trimmed(duty)}). Find the pages it makes wrong or incomplete: stale_pages for this repository first (pages citing code it changed), then search_docs for what it changed. Update each with edit_page (it becomes a suggestion where you can't edit), citing the pull request, with marks_current when it brings a stale page up to date. Post a short list of what you changed here; say so if nothing needed changing.`,
3939 schedule: null,
4040 events: ["pull_merged"],
4141 repos: [],
+13−2
5454 spaces(viewer: User, audience: DocAudience): Promise<string | null>;
5555 search(viewer: User, audience: DocAudience, query: string, project: string | null): Promise<string | null>;
5656 read(viewer: User, audience: DocAudience, pageId: string): Promise<string | null>;
57− edit(viewer: User, pageId: string, edit: { target: DocEditTarget; markdown: string; note: string | null }, suggestOnly: boolean): Promise<{ ok: boolean; message: string }>;
57+ /** Pages possibly out of date since code they cite changed, with the change. */
58+ stale(viewer: User, audience: DocAudience, repo: string | null): Promise<string | null>;
59+ edit(viewer: User, pageId: string, edit: { target: DocEditTarget; markdown: string; note: string | null; marks_current: boolean }, suggestOnly: boolean): Promise<{ ok: boolean; message: string }>;
5860 create(viewer: User, input: { space_id: string | null; parent_id: string | null; title: string; markdown: string; source: { title: string; href: string } | null }): Promise<{ ok: boolean; message: string }>;
5961 }
6062
291293 input_schema: { type: "object", properties: { page: { type: "string" } }, required: ["page"] },
292294 },
293295 {
296+ name: "stale_pages",
297+ description:
298+ "Docs pages possibly out of date because code they cite changed, each with the change (pull request or commit) and the paths. Optionally only for one repository (`workspace/name`). Start here when keeping the docs current.",
299+ input_schema: { type: "object", properties: { repo: { type: "string" } } },
300+ },
301+ {
294302 name: "list_doc_spaces",
295303 description: "The Docs spaces you can read here, with what you may do in each (read, suggest, edit).",
296304 input_schema: { type: "object", properties: {} },
312320 to_block: { type: "string" },
313321 markdown: { type: "string" },
314322 note: { type: "string", description: "Why, in a line, for the history or the suggestion." },
323+ marks_current: { type: "boolean", description: "This edit brings a page marked possibly out of date up to date: it clears the mark when it applies or is accepted." },
315324 suggest_only: { type: "boolean", description: "Suggest even where you could edit." },
316325 },
317326 required: ["page", "target", "markdown"],
508517 const project = text("project", 200).toLowerCase() || null;
509518 return read(`search_docs "${query}"`, await docs.search(asker, audience, query, project));
510519 }
520+ case "stale_pages":
521+ return read("stale_pages", await docs.stale(asker, audience, text("repo", 200).toLowerCase() || null));
511522 case "read_page": {
512523 const page = pageId(text("page", 300));
513524 if (!page) return { text: "Give the page's id or link.", outcome: "refused" };
529540 ? { kind: "blocks", from_block: text("from_block", 100), to_block: text("to_block", 100) }
530541 : null;
531542 if (!target) return { text: "Say what to change: append, a section by its heading, blocks by their ids, or the whole document.", outcome: "refused" };
532− const done = await docs.edit(asker, page, { target, markdown, note: text("note", 300) || null }, input.suggest_only === true);
543+ const done = await docs.edit(asker, page, { target, markdown, note: text("note", 300) || null, marks_current: input.marks_current === true }, input.suggest_only === true);
533544 return { text: done.message, outcome: done.ok ? "allowed" : "refused" };
534545 }
535546 case "create_page": {
+74−0
1+-- Pages know what they describe, and projects' docs folders in Docs
2+-- (docs/WORKSPACE.md, "Docs"; services/docs src/staleness.ts and
3+-- src/repo-spaces.ts).
4+
5+-- Code a page cites: from its text (the editor's citation chips and links
6+-- to files in a repository), rebuilt on each save; and from its header's
7+-- "Describes" list. `repo` is `owner/name`, lowercased; `path` a file, a
8+-- folder or a glob.
9+CREATE TABLE citations (
10+ page_id TEXT NOT NULL REFERENCES pages (id) ON DELETE CASCADE,
11+ repo TEXT NOT NULL,
12+ path TEXT NOT NULL,
13+ kind TEXT NOT NULL CHECK (kind IN ('path', 'symbol', 'endpoint', 'env')),
14+ label TEXT NOT NULL DEFAULT '',
15+ ref TEXT,
16+ source TEXT NOT NULL CHECK (source IN ('body', 'header')),
17+ PRIMARY KEY (page_id, source, repo, path, kind, label)
18+);
19+CREATE INDEX citations_repo ON citations (repo);
20+
21+-- Changes that touched code a page cites: one row per page and commit (a
22+-- merge is told twice, as `git.push` and as `pull.merged`; both land on
23+-- the same row, and the pull request names it). Open until someone marks
24+-- the page current (`cleared_at`).
25+CREATE TABLE page_changes (
26+ page_id TEXT NOT NULL REFERENCES pages (id) ON DELETE CASCADE,
27+ repo TEXT NOT NULL,
28+ repo_id TEXT NOT NULL,
29+ commit_sha TEXT NOT NULL,
30+ pull_number INTEGER,
31+ pull_title TEXT,
32+ -- The cited paths it changed, as JSON.
33+ paths TEXT NOT NULL DEFAULT '[]',
34+ detected_at TEXT NOT NULL,
35+ cleared_at TEXT,
36+ cleared_by TEXT,
37+ PRIMARY KEY (page_id, repo, commit_sha)
38+);
39+CREATE INDEX page_changes_open ON page_changes (page_id) WHERE cleared_at IS NULL;
40+CREATE INDEX page_changes_recent ON page_changes (detected_at) WHERE cleared_at IS NULL;
41+
42+-- An agent's suggestion that, once accepted, marks the page current.
43+ALTER TABLE suggestions ADD COLUMN marks_current INTEGER NOT NULL DEFAULT 0;
44+
45+-- A repository's docs folder shown in a workspace's Docs.
46+CREATE TABLE repo_spaces (
47+ id TEXT PRIMARY KEY,
48+ workspace_id TEXT NOT NULL,
49+ repo_id TEXT NOT NULL,
50+ -- `owner/name`, lowercased, as it is named now.
51+ repo TEXT NOT NULL,
52+ default_branch TEXT NOT NULL,
53+ -- The commit it was last read at.
54+ commit_sha TEXT,
55+ indexed_at TEXT,
56+ added_by TEXT NOT NULL,
57+ added_at TEXT NOT NULL,
58+ UNIQUE (workspace_id, repo_id)
59+);
60+CREATE INDEX repo_spaces_repo ON repo_spaces (repo_id);
61+
62+-- Its Markdown files as last read. `hash` is the blob's, so a push reads
63+-- only what changed.
64+CREATE TABLE repo_files (
65+ space_id TEXT NOT NULL REFERENCES repo_spaces (id) ON DELETE CASCADE,
66+ path TEXT NOT NULL,
67+ hash TEXT NOT NULL,
68+ title TEXT NOT NULL,
69+ markdown TEXT NOT NULL,
70+ PRIMARY KEY (space_id, path)
71+);
72+
73+-- Full text over them, beside pages_fts.
74+CREATE VIRTUAL TABLE repo_files_fts USING fts5 (space_id UNINDEXED, path UNINDEXED, title, body, tokenize = 'unicode61 remove_diacritics 2');
+75−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+
4+import { bodyCitations, citationHref, citationMarkdown, citationsFromMarkdown, cleanCitation, cleanDescribes, cleanPath, touchedPaths, touches } from "./citations.ts";
5+
6+test("paths are kept tidy, and unsafe ones refused", () => {
7+ assert.equal(cleanPath("/src//export.ts/"), "src/export.ts");
8+ assert.equal(cleanPath("./docs/a.md"), "docs/a.md");
9+ assert.equal(cleanPath("src/../secrets"), null);
10+ assert.equal(cleanPath(""), null);
11+});
12+
13+test("a citation names a repository and a path; a label only for what it names", () => {
14+ assert.deepEqual(cleanCitation({ repo: "Acme/Web", path: "src/export.ts", kind: "symbol", label: " exportCsv ", ref: "ABCDEF1" }, "body"), {
15+ repo: "acme/web",
16+ path: "src/export.ts",
17+ kind: "symbol",
18+ label: "exportCsv",
19+ ref: "abcdef1",
20+ source: "body",
21+ });
22+ assert.equal(cleanCitation({ repo: "acme/web", path: "src/a.ts", kind: "path", label: "ignored" }, "body")?.label, null);
23+ assert.equal(cleanCitation({ repo: "nope", path: "src/a.ts" }, "body"), null);
24+ assert.equal(cleanCitation({ repo: "acme/web", path: "src/a.ts", kind: "weird" as never }, "body")?.kind, "path");
25+});
26+
27+test("a change touches a cited file, anything under a cited folder, and what a glob matches", () => {
28+ assert.ok(touches("src/export.ts", "src/export.ts"));
29+ assert.ok(!touches("src/export.ts", "src/export.tsx"));
30+ assert.ok(touches("src/export", "src/export/csv.ts"));
31+ assert.ok(!touches("src/export", "src/exports/csv.ts"));
32+ assert.ok(touches("src/*.ts", "src/a.ts"));
33+ assert.ok(!touches("src/*.ts", "src/deep/a.ts"));
34+ assert.ok(touches("src/**/*.sql", "src/a.sql"));
35+ assert.ok(touches("src/**/*.sql", "src/db/migrations/0001.sql"));
36+ assert.ok(touches("api/**", "api/v1/routes.rs"));
37+ assert.ok(touches("docs/?.md", "docs/a.md"));
38+ assert.ok(!touches("docs/?.md", "docs/ab.md"));
39+ assert.ok(!touches("a.b/*.ts", "aXb/x.ts"));
40+ assert.deepEqual(touchedPaths([{ path: "src/export" }, { path: "*.toml" }], ["README.md", "src/export/csv.ts", "Cargo.toml"]), ["src/export/csv.ts", "Cargo.toml"]);
41+});
42+
43+test("a citation links to the code at the commit it was cited at", () => {
44+ assert.equal(citationHref({ repo: "acme/web", path: "src/export.ts", ref: "abc1234" }), "/acme/web/blob/abc1234/src/export.ts");
45+ assert.equal(citationHref({ repo: "acme/web", path: "src/export", ref: null }), "/acme/web/tree/HEAD/src/export");
46+ assert.equal(citationHref({ repo: "acme/web", path: "src/**/*.sql", ref: null }), "/acme/web/tree/HEAD/src");
47+ assert.equal(citationMarkdown({ repo: "acme/web", path: "src/export.ts", ref: "abc1234", kind: "env", label: "EXPORT_BUCKET" }), "[`EXPORT_BUCKET`](/acme/web/blob/abc1234/src/export.ts)");
48+});
49+
50+test("links to code in a page are citations too, a chip's own link once", () => {
51+ const md = "See [`exportCsv`](/acme/web/blob/abc1234/src/export.ts) and [the folder](/acme/web/tree/main/src/jobs) and [elsewhere](https://example.com/a/b/blob/x/y.ts) and [on g1t](https://g1t.sh/acme/api/blob/main/src/lib.rs#L10).";
52+ assert.deepEqual(
53+ citationsFromMarkdown(md).map((c) => [c.repo, c.path, c.ref]),
54+ [
55+ ["acme/web", "src/export.ts", "abc1234"],
56+ ["acme/web", "src/jobs", "main"],
57+ ["acme/api", "src/lib.rs", "main"],
58+ ],
59+ );
60+ const chips = [{ repo: "acme/web", path: "src/export.ts", kind: "symbol" as const, label: "exportCsv", ref: "abc1234" }];
61+ const all = bodyCitations(chips, md);
62+ assert.deepEqual(
63+ all.map((c) => [c.path, c.kind]),
64+ [
65+ ["src/export.ts", "symbol"],
66+ ["src/jobs", "path"],
67+ ["src/lib.rs", "path"],
68+ ],
69+ );
70+});
71+
72+test("the header's Describes list is tidy and short", () => {
73+ assert.deepEqual(cleanDescribes([{ repo: "Acme/Web", path: "/src/export/" }, { repo: "acme/web", path: "src/export" }, { repo: "x", path: "y" }, "nope"]), [{ repo: "acme/web", path: "src/export" }]);
74+ assert.equal(cleanDescribes(Array.from({ length: 40 }, (_, i) => ({ repo: "acme/web", path: `f${i}` }))).length, 20);
75+});
+192−0
1+/**
2+ * Code a page cites, and whether a change touched it. Pure.
3+ *
4+ * A citation names a repository (`owner/name`) and a path in it: a file
5+ * (`src/export.ts`), a folder (`src/export`, everything under it) or a
6+ * glob (`src/**\/*.sql`, `*` within a folder, `**` across folders, `?` one
7+ * character). Its kind says what at that path the page describes: the
8+ * path itself, a symbol, an endpoint or an environment variable, named by
9+ * its label. A page's citations come from its text (the editor's citation
10+ * chips, and links to files in a repository) and from its header.
11+ */
12+import type { DocCitation, DocCitationKind, DocDescribes } from "@g1t/contracts";
13+
14+import { projectRef } from "./search.ts";
15+
16+const KINDS = new Set<DocCitationKind>(["path", "symbol", "endpoint", "env"]);
17+const MAX_PATH = 400;
18+const MAX_LABEL = 200;
19+/** Citations a page keeps from its text, and in its header. */
20+export const MAX_CITATIONS = 100;
21+export const MAX_DESCRIBES = 20;
22+
23+/** A path as kept: no leading or trailing slash, no `.`/`..` parts, no doubled slashes; null when empty or unsafe. */
24+export function cleanPath(path: unknown): string | null {
25+ const parts = String(path ?? "")
26+ .trim()
27+ .replace(/\\/g, "/")
28+ .split("/")
29+ .filter((p) => p && p !== ".");
30+ if (!parts.length || parts.some((p) => p === ".." || /[\u0000-\u001f]/.test(p))) return null;
31+ const out = parts.join("/");
32+ return out.length <= MAX_PATH ? out : null;
33+}
34+
35+function isGlob(path: string): boolean {
36+ return /[*?]/.test(path);
37+}
38+
39+/** A citation as stored, or null when it names no repository or path. */
40+export function cleanCitation(raw: Partial<DocCitation> | null | undefined, source: DocCitation["source"]): DocCitation | null {
41+ if (!raw || typeof raw !== "object") return null;
42+ const repo = projectRef(String(raw.repo ?? ""));
43+ const path = cleanPath(raw.path);
44+ if (!repo || !path) return null;
45+ const kind = KINDS.has(raw.kind as DocCitationKind) ? (raw.kind as DocCitationKind) : "path";
46+ const label = kind === "path" ? null : String(raw.label ?? "").replace(/\s+/g, " ").trim().slice(0, MAX_LABEL) || null;
47+ const ref = /^[0-9a-f]{7,64}$/i.test(String(raw.ref ?? "")) ? String(raw.ref).toLowerCase() : /^[A-Za-z0-9._/-]{1,100}$/.test(String(raw.ref ?? "")) ? String(raw.ref) : null;
48+ return { repo, path, kind, label, ref, source };
49+}
50+
51+/** The header's "Describes" list as stored: valid entries, once each, at most MAX_DESCRIBES. */
52+export function cleanDescribes(list: unknown): DocDescribes[] {
53+ if (!Array.isArray(list)) return [];
54+ const seen = new Set<string>();
55+ const out: DocDescribes[] = [];
56+ for (const item of list) {
57+ const c = cleanCitation({ ...(item as object), kind: "path" }, "header");
58+ if (!c || seen.has(`${c.repo}\n${c.path}`)) continue;
59+ seen.add(`${c.repo}\n${c.path}`);
60+ out.push({ repo: c.repo, path: c.path });
61+ if (out.length >= MAX_DESCRIBES) break;
62+ }
63+ return out;
64+}
65+
66+/** The folder a glob starts from: its parts before the first with a wildcard. */
67+export function globRoot(path: string): string {
68+ const parts = path.split("/");
69+ const i = parts.findIndex((p) => isGlob(p));
70+ return (i < 0 ? parts : parts.slice(0, i)).join("/");
71+}
72+
73+/**
74+ * Where a citation links: the file (or folder) in Code at the commit it
75+ * was cited at, or the default branch (`HEAD`) when none is known. A
76+ * glob links to the folder it starts from.
77+ */
78+export function citationHref(c: Pick<DocCitation, "repo" | "path" | "ref">): string {
79+ const ref = encodeURIComponent(c.ref || "HEAD");
80+ const glob = isGlob(c.path);
81+ const path = (glob ? globRoot(c.path) : c.path)
82+ .split("/")
83+ .filter(Boolean)
84+ .map(encodeURIComponent)
85+ .join("/");
86+ const kind = glob || !/\.[A-Za-z0-9]{1,10}$/.test(c.path) ? "tree" : "blob";
87+ return `/${c.repo}/${kind}/${ref}${path ? `/${path}` : ""}`;
88+}
89+
90+/** A citation's text in a page's Markdown: a link to the code. */
91+export function citationMarkdown(c: Pick<DocCitation, "repo" | "path" | "ref" | "kind" | "label">): string {
92+ const shown = c.kind !== "path" && c.label ? c.label : c.path;
93+ const ticks = shown.includes("`") ? "``" : "`";
94+ return `[${ticks}${shown}${ticks}](${citationHref(c)})`;
95+}
96+
97+/**
98+ * Links in Markdown to files and folders in a repository on this site:
99+ * `/<owner>/<repo>/blob|tree/<ref>/<path>`, absolute or on this origin.
100+ * The ref is one segment (a commit or a branch without a slash).
101+ */
102+export function citationsFromMarkdown(markdown: string): DocCitation[] {
103+ const out: DocCitation[] = [];
104+ const seen = new Set<string>();
105+ const link = /\]\(\s*<?((?:https?:\/\/[^/\s)]+)?\/([A-Za-z0-9][A-Za-z0-9._-]*)\/([A-Za-z0-9._-]+)\/(?:blob|tree)\/([^/\s)#?]+)\/([^\s)#?>]+))[^)]*\)/g;
106+ for (const m of String(markdown ?? "").matchAll(link)) {
107+ if (m[1]!.startsWith("http") && !/^https?:\/\/([a-z0-9-]+\.)*g1t\.(sh|dev)(:\d+)?\//i.test(m[1]!)) continue;
108+ let path: string;
109+ let ref: string;
110+ try {
111+ path = m[5]!
112+ .split("/")
113+ .map((p) => decodeURIComponent(p))
114+ .join("/");
115+ ref = decodeURIComponent(m[4]!);
116+ } catch {
117+ continue;
118+ }
119+ const c = cleanCitation({ repo: `${m[2]}/${m[3]}`, path, ref: ref === "HEAD" ? null : ref, kind: "path" }, "body");
120+ if (!c || seen.has(`${c.repo}\n${c.path}`)) continue;
121+ seen.add(`${c.repo}\n${c.path}`);
122+ out.push(c);
123+ }
124+ return out;
125+}
126+
127+/**
128+ * A page's citations from its text: the citation chips (`nodes`), then
129+ * links to code not already a chip's own. At most MAX_CITATIONS.
130+ */
131+export function bodyCitations(nodes: Partial<DocCitation>[], markdown: string): DocCitation[] {
132+ const out: DocCitation[] = [];
133+ const seen = new Set<string>();
134+ const key = (c: DocCitation) => `${c.repo}\n${c.path}\n${c.kind}\n${c.label ?? ""}`;
135+ const chips = new Set<string>();
136+ for (const n of nodes) {
137+ const c = cleanCitation(n, "body");
138+ if (!c || seen.has(key(c))) continue;
139+ seen.add(key(c));
140+ chips.add(citationHref(c));
141+ out.push(c);
142+ }
143+ for (const c of citationsFromMarkdown(markdown)) {
144+ // A chip's own link is the chip.
145+ if (chips.has(citationHref(c)) || out.some((o) => o.repo === c.repo && o.path === c.path)) continue;
146+ if (seen.has(key(c))) continue;
147+ seen.add(key(c));
148+ out.push(c);
149+ }
150+ return out.slice(0, MAX_CITATIONS);
151+}
152+
153+/** A glob as a regular expression over a whole path. */
154+function globExpression(glob: string): RegExp {
155+ let out = "";
156+ for (let i = 0; i < glob.length; i++) {
157+ const ch = glob[i]!;
158+ if (ch === "*") {
159+ if (glob[i + 1] === "*") {
160+ // `**/` is any folders, or none; a trailing `**` anything below.
161+ if (glob[i + 2] === "/") {
162+ out += "(?:.*/)?";
163+ i += 2;
164+ } else {
165+ out += ".*";
166+ i += 1;
167+ }
168+ } else out += "[^/]*";
169+ } else if (ch === "?") out += "[^/]";
170+ else out += ch.replace(/[.+^${}()|[\]\\]/g, "\\$&");
171+ }
172+ return new RegExp(`^${out}$`);
173+}
174+
175+/**
176+ * Whether a change to `changed` touches what `cited` names: the same file,
177+ * anything under a cited folder, or a path a glob matches.
178+ */
179+export function touches(cited: string, changed: string): boolean {
180+ if (isGlob(cited)) return globExpression(cited).test(changed);
181+ return changed === cited || changed.startsWith(`${cited}/`);
182+}
183+
184+/** Of `changed` paths, those any of the citations names; at most `max`. */
185+export function touchedPaths(citations: Pick<DocCitation, "path">[], changed: string[], max = 20): string[] {
186+ const out: string[] = [];
187+ for (const path of changed) {
188+ if (citations.some((c) => touches(c.path, path))) out.push(path);
189+ if (out.length >= max) break;
190+ }
191+ return out;
192+}
+26−0
1+/**
2+ * What the docs service tells the rest of g1t: `doc.page.*` events on the
3+ * bus (packages/contracts events.ts), through the EVENTS binding when it
4+ * has one. Published with no `repoId`, so a page never reaches a
5+ * repository's timeline or webhooks. Never throws: an event that can't be
6+ * told is logged, and the change it is about still happened.
7+ */
8+import { eventsClient, type DocPageEventData, type EventPayloads, type ServiceBinding } from "@g1t/contracts";
9+
10+type DocEventType = "doc.page.created" | "doc.page.updated" | "doc.page.archived" | "doc.page.stale";
11+
12+/** The id in a member key (`user:<id>`, `agent:<id>`), as an event's actor. */
13+export function actorOf(key: string | null | undefined): string | null {
14+ if (!key) return null;
15+ const at = key.indexOf(":");
16+ return at > 0 ? key.slice(at + 1) || null : null;
17+}
18+
19+export async function publishDocEvent<T extends DocEventType>(events: ServiceBinding | undefined, type: T, data: EventPayloads[T] & DocPageEventData, actor: string | null): Promise<void> {
20+ if (!events) return;
21+ try {
22+ await eventsClient(events).publish([{ type, source: "docs", repoId: null, actor: actorOf(actor), data } as never]);
23+ } catch (error) {
24+ console.error("docs could not publish", type, String(error));
25+ }
26+}
+56−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+
4+import { fileStore, s3FileStore } from "./files.ts";
5+
6+type Call = { url: string; method: string; headers: Record<string, string>; body: Uint8Array | null };
7+
8+function fake(responses: Response[] = []) {
9+ const calls: Call[] = [];
10+ const fetcher = (async (url: string, init: RequestInit) => {
11+ calls.push({ url, method: String(init.method), headers: init.headers as Record<string, string>, body: (init.body as Uint8Array | null) ?? null });
12+ return responses.shift() ?? new Response(null, { status: 200 });
13+ }) as unknown as typeof fetch;
14+ return { calls, fetcher };
15+}
16+
17+const config = { endpoint: "https://s3.example.com", bucket: "g1t-docs", region: "us-east-1", access_key_id: "AKID", secret_access_key: "secret" };
18+
19+test("puts, gets and deletes a file by its key, path style, signed", async () => {
20+ const { calls, fetcher } = fake([new Response(null, { status: 200 }), new Response("hello", { status: 200, headers: { "content-type": "image/png", "content-length": "5", etag: '"e"' } }), new Response(null, { status: 204 })]);
21+ const store = s3FileStore(config, fetcher);
22+ await store.put("docs/ab cd", new TextEncoder().encode("hello"), "image/png");
23+ const got = await store.get("docs/ab cd");
24+ await store.delete("docs/ab cd");
25+ assert.deepEqual(
26+ calls.map((c) => [c.method, c.url]),
27+ [
28+ ["PUT", "https://s3.example.com/g1t-docs/docs/ab%20cd"],
29+ ["GET", "https://s3.example.com/g1t-docs/docs/ab%20cd"],
30+ ["DELETE", "https://s3.example.com/g1t-docs/docs/ab%20cd"],
31+ ],
32+ );
33+ assert.match(calls[0]!.headers.authorization!, /^AWS4-HMAC-SHA256 Credential=AKID\/\d{8}\/us-east-1\/s3\/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=[0-9a-f]{64}$/);
34+ assert.equal(calls[0]!.headers["x-amz-content-sha256"], "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824");
35+ assert.equal(calls[0]!.headers.host, undefined);
36+ assert.equal(got?.content_type, "image/png");
37+ assert.equal(got?.bytes, 5);
38+ assert.equal(await new Response(got!.body).text(), "hello");
39+});
40+
41+test("a missing file is null", async () => {
42+ const { fetcher } = fake([new Response("", { status: 404 })]);
43+ assert.equal(await s3FileStore(config, fetcher).get("docs/x"), null);
44+});
45+
46+test("virtual-hosted addresses put the bucket in the host", async () => {
47+ const { calls, fetcher } = fake();
48+ await s3FileStore({ ...config, virtual_hosted: true }, fetcher).delete("docs/x");
49+ assert.equal(calls[0]!.url, "https://g1t-docs.s3.example.com/docs/x");
50+});
51+
52+test("the store follows the settings, and says what is missing", () => {
53+ assert.throws(() => fileStore({ DOCS_FILES: "s3", DOCS_S3_BUCKET: "b" }), /DOCS_S3_ENDPOINT, DOCS_S3_ACCESS_KEY_ID, DOCS_S3_SECRET_ACCESS_KEY/);
54+ assert.throws(() => fileStore({}), /No file store/);
55+ assert.ok(fileStore({ DOCS_FILES: "s3", DOCS_S3_ENDPOINT: "http://minio:9000", DOCS_S3_BUCKET: "b", DOCS_S3_ACCESS_KEY_ID: "k", DOCS_S3_SECRET_ACCESS_KEY: "s" }));
56+});
+101−2
11 /**
22 * Where files put in pages are kept: behind this small interface, so a
3− * self-hosted g1t can keep them on disk or in any S3-compatible store.
4− * The managed service uses R2 (`r2FileStore`).
3+ * self-hosted g1t can keep them in any S3-compatible store. The managed
4+ * service uses R2 (`r2FileStore`); `DOCS_FILES=s3` picks `s3FileStore`
5+ * (docs/SELF_HOSTING.md, "Files in Docs pages").
56 */
7+import { sha256Hex, sign } from "./sigv4.ts";
68
79 export type StoredFile = { body: ReadableStream; content_type: string; bytes: number; etag: string };
810
3335 };
3436 }
3537
38+/** An S3-compatible store: the endpoint, the bucket and the keys, from the Worker's settings and secrets. */
39+export type S3Config = {
40+ /** `https://s3.example.com` (or `http://minio:9000` inside a private network). */
41+ endpoint: string;
42+ bucket: string;
43+ /** `us-east-1` unless the store says otherwise; R2 and MinIO accept `auto`/`us-east-1`. */
44+ region: string;
45+ access_key_id: string;
46+ secret_access_key: string;
47+ /** `https://<bucket>.<endpoint host>/<key>` instead of `<endpoint>/<bucket>/<key>`. Path style by default: every compatible store takes it. */
48+ virtual_hosted?: boolean;
49+};
50+
51+/** Each part of a key encoded, its slashes kept. */
52+function keyPath(key: string): string {
53+ return key
54+ .split("/")
55+ .map((part) => encodeURIComponent(part))
56+ .join("/");
57+}
58+
59+export function s3FileStore(config: S3Config, fetcher: typeof fetch = fetch): FileStore {
60+ const base = new URL(config.endpoint);
61+ const urlOf = (key: string) => {
62+ const root = base.pathname.replace(/\/+$/, "");
63+ if (config.virtual_hosted) return `${base.protocol}//${config.bucket}.${base.host}${root}/${keyPath(key)}`;
64+ return `${base.protocol}//${base.host}${root}/${encodeURIComponent(config.bucket)}/${keyPath(key)}`;
65+ };
66+ const send = async (method: string, key: string, body: Uint8Array | null, headers: Record<string, string> = {}) => {
67+ const payloadHash = await sha256Hex(body ?? new Uint8Array());
68+ const signed = await sign({
69+ method,
70+ url: urlOf(key),
71+ headers: { ...headers, "x-amz-content-sha256": payloadHash },
72+ payload_hash: payloadHash,
73+ region: config.region || "us-east-1",
74+ service: "s3",
75+ credentials: { access_key_id: config.access_key_id, secret_access_key: config.secret_access_key },
76+ });
77+ const { host: _host, ...send } = signed.headers;
78+ return fetcher(urlOf(key), { method, headers: send, body: body as BodyInit | null });
79+ };
80+ return {
81+ async put(key, body, contentType) {
82+ // Signed over its bytes: a page's file is at most DOC_MAX_FILE_BYTES.
83+ const bytes = body instanceof ReadableStream ? new Uint8Array(await new Response(body).arrayBuffer()) : body instanceof Uint8Array ? body : new Uint8Array(body);
84+ const response = await send("PUT", key, bytes, { "content-type": contentType });
85+ if (!response.ok) throw new Error(`The file store refused the file (${response.status}).`);
86+ },
87+ async get(key) {
88+ const response = await send("GET", key, null);
89+ if (response.status === 404) return null;
90+ if (!response.ok || !response.body) throw new Error(`The file store didn't answer (${response.status}).`);
91+ return {
92+ body: response.body,
93+ content_type: response.headers.get("content-type") ?? "application/octet-stream",
94+ bytes: Number(response.headers.get("content-length") ?? "0"),
95+ etag: response.headers.get("etag") ?? `"${key}"`,
96+ };
97+ },
98+ async delete(key) {
99+ const response = await send("DELETE", key, null);
100+ if (!response.ok && response.status !== 404) throw new Error(`The file store didn't delete the file (${response.status}).`);
101+ },
102+ };
103+}
104+
105+/** What picks the store: the R2 binding, or `DOCS_FILES=s3` and its settings. */
106+export type FileStoreEnv = {
107+ FILES?: R2Bucket;
108+ DOCS_FILES?: string;
109+ DOCS_S3_ENDPOINT?: string;
110+ DOCS_S3_BUCKET?: string;
111+ DOCS_S3_REGION?: string;
112+ DOCS_S3_ACCESS_KEY_ID?: string;
113+ DOCS_S3_SECRET_ACCESS_KEY?: string;
114+ DOCS_S3_VIRTUAL_HOSTED?: string;
115+};
116+
117+/** The store this deployment keeps files in. Throws when it is set up halfway, so a misconfiguration shows at once. */
118+export function fileStore(env: FileStoreEnv): FileStore {
119+ if ((env.DOCS_FILES ?? "").toLowerCase() === "s3") {
120+ const missing = (["DOCS_S3_ENDPOINT", "DOCS_S3_BUCKET", "DOCS_S3_ACCESS_KEY_ID", "DOCS_S3_SECRET_ACCESS_KEY"] as const).filter((k) => !env[k]);
121+ if (missing.length) throw new Error(`DOCS_FILES=s3 needs ${missing.join(", ")}.`);
122+ return s3FileStore({
123+ endpoint: env.DOCS_S3_ENDPOINT!,
124+ bucket: env.DOCS_S3_BUCKET!,
125+ region: env.DOCS_S3_REGION || "us-east-1",
126+ access_key_id: env.DOCS_S3_ACCESS_KEY_ID!,
127+ secret_access_key: env.DOCS_S3_SECRET_ACCESS_KEY!,
128+ virtual_hosted: env.DOCS_S3_VIRTUAL_HOSTED === "true",
129+ });
130+ }
131+ if (!env.FILES) throw new Error("No file store: bind FILES (R2) or set DOCS_FILES=s3.");
132+ return r2FileStore(env.FILES);
133+}
134+
36135 /** Types a page's file is served as; anything else is served as a download. */
37136 const INLINE = new Set(["image/png", "image/jpeg", "image/gif", "image/webp", "image/avif", "video/mp4", "video/webm", "audio/mpeg", "audio/ogg", "audio/wav", "application/pdf"]);
38137
+483−21
2424 openD1,
2525 parsePrincipalKey,
2626 principalKey,
27+ reposClient,
2728 workspaceAgentsClient,
2829 type DocAgentAbilities,
30+ type DocCitation,
31+ type DocDescribes,
32+ type DocRepoPage,
33+ type DocRepoSpace,
34+ type DocStaleChange,
35+ type DocStalePage,
36+ type DocStaleness,
37+ type G1tEvent,
38+ type Repo,
2939 type DocAgentEditResult,
3040 type DocAgentMode,
3141 type DocAgentPage,
6979
7080 import { RANK, agentAbilities, atLeast, isRole, leavesNoManager, memberKey, readableByAll, readableByWorkspace, roleOf, type Person, type SpaceRules } from "./access.ts";
7181 import { diffLines } from "./diff.ts";
72−import { r2FileStore, safeName, servedType } from "./files.ts";
82+import { cleanDescribes } from "./citations.ts";
83+import { publishDocEvent } from "./events.ts";
84+import { fileStore, safeName, servedType, type FileStoreEnv } from "./files.ts";
7385 import { excerpt, searchText } from "./markdown.ts";
7486 import { ROOM_MEMBER_HEADER, type Origin, type PageRoom, type RoomMember } from "./room.ts";
87+import { indexRepoSpace, reindexRepo, type RepoSpaceRow } from "./repo-spaces.ts";
7588 import { ftsQuery, inProject, projectRef, searchSpaces } from "./search.ts";
7689 import { freeSlug, pageSlug, validSpaceSlug } from "./slugs.ts";
7790 import { BUILTIN_TEMPLATES, builtinTemplate } from "./templates.ts";
7891 import type { ThreadResult } from "./threads.ts";
92+import { onEvent } from "./staleness.ts";
7993 import { descendants, exportPaths, lastPosition, placeBefore, wouldCycle, ancestors } from "./tree.ts";
8094
8195 export { PageRoom } from "./room.ts";
8296
83−type Env = {
97+type Env = FileStoreEnv & {
8498 DB: D1Database;
8599 IDENTITY: ServiceBinding;
86100 AGENTS: ServiceBinding;
87101 NOTIFY?: ServiceBinding;
102+ /** Repositories: who may read one, what a change touched, a project's docs (src/staleness.ts, src/repo-spaces.ts). */
103+ REPOS?: ServiceBinding;
104+ /** Pull requests: what a merged one changed. */
105+ WORK?: ServiceBinding;
106+ /** The bus: `doc.page.*` events (src/events.ts). */
107+ EVENTS?: ServiceBinding;
88108 PAGES: DurableObjectNamespace<PageRoom>;
89− FILES: R2Bucket;
90109 };
91110
92111 type SpaceRow = {
137156 created_at: string;
138157 decided_by: string | null;
139158 decided_at: string | null;
159+ marks_current?: number;
140160 };
141161
162+type ChangeRow = {
163+ page_id: string;
164+ repo: string;
165+ repo_id: string;
166+ commit_sha: string;
167+ pull_number: number | null;
168+ pull_title: string | null;
169+ paths: string;
170+ detected_at: string;
171+ cleared_at: string | null;
172+ cleared_by: string | null;
173+};
174+
142175 type VersionRow = { id: string; page_id: string; created_at: string; kind: DocVersion["kind"]; authors: string; note: string | null; markdown: string; state: ArrayBuffer | null };
143176
144177 /** A space, with who is in it and the viewer's role. */
426459 if (!rows.length) return [];
427460 const ids = rows.map((r) => r.id);
428461 const marks = ids.map(() => "?").join(",");
429− const [owners, projects, kids] = await Promise.all([
462+ const [owners, projects, kids, stale] = await Promise.all([
430463 this.db.prepare(`SELECT page_id, principal FROM page_owners WHERE page_id IN (${marks})`).bind(...ids).all<{ page_id: string; principal: string }>(),
431464 this.db.prepare(`SELECT page_id, repo FROM page_projects WHERE page_id IN (${marks})`).bind(...ids).all<{ page_id: string; repo: string }>(),
432465 this.db
433466 .prepare(`SELECT DISTINCT parent_id FROM pages WHERE parent_id IN (${marks}) AND archived_at IS NULL`)
434467 .bind(...ids)
435468 .all<{ parent_id: string }>(),
469+ this.staleIds(ids),
436470 ]);
437471 const keys = [...rows.flatMap((r) => [r.created_by, r.updated_by ?? r.created_by]), ...owners.results.map((o) => o.principal)];
438472 const people = await this.profiles(workspace, keys);
453487 projects: projects.results.filter((p) => p.page_id === row.id).map((p) => p.repo),
454488 owners: owners.results.filter((o) => o.page_id === row.id).map((o) => people.get(o.principal)!),
455489 excerpt: excerpt(row.markdown ?? ""),
490+ stale: stale.has(row.id),
456491 };
457492 });
458493 }
459494
495+ /** Of these pages, those possibly out of date. */
496+ private async staleIds(ids: string[]): Promise<Set<string>> {
497+ if (!ids.length) return new Set();
498+ const found = new Set<string>();
499+ for (let i = 0; i < ids.length; i += 90) {
500+ const part = ids.slice(i, i + 90);
501+ const rows = await this.db
502+ .prepare(`SELECT DISTINCT page_id FROM page_changes WHERE cleared_at IS NULL AND page_id IN (${part.map(() => "?").join(",")})`)
503+ .bind(...part)
504+ .all<{ page_id: string }>();
505+ for (const r of rows.results) found.add(r.page_id);
506+ }
507+ return found;
508+ }
509+
460510 /** A page and its space, with the viewer's role; not found when they can't read it. */
461511 private async pageFor(slug: string, pageId: string, viewer: Viewer, need: DocRole): Promise<Result<{ workspace: Workspace; page: PageRow; space: Space; spaces: Space[] }>> {
462512 const found = await this.viewerWorkspace(slug, viewer);
497547 await this.ensureDefault(workspace, viewer);
498548 const spaces = (await this.spacesFor(workspace, viewer)).filter((s) => s.role);
499549 const ids = spaces.map((s) => s.row.id);
500− if (!ids.length) return ok({ spaces: [], favorites: [], recent: [], can_create_space: true, trash_count: 0 });
550+ const repos = await this.repoSpacesFor(workspace, viewer).catch((error: unknown) => {
551+ console.error("docs could not list projects' docs", String(error));
552+ return [] as DocRepoSpace[];
553+ });
554+ if (!ids.length) return ok({ spaces: [], favorites: [], recent: [], can_create_space: true, trash_count: 0, stale_count: 0, repos });
501555 const marks = ids.map(() => "?").join(",");
502− const [pages, favorites, recent, trash] = await Promise.all([
556+ const [pages, favorites, recent, trash, stale] = await Promise.all([
503557 this.db
504558 .prepare(`SELECT id, space_id, parent_id, position, title, icon FROM pages WHERE space_id IN (${marks}) AND archived_at IS NULL ORDER BY position`)
505559 .bind(...ids)
516570 .prepare(`SELECT COUNT(*) AS n FROM pages WHERE space_id IN (${marks}) AND archived_at IS NOT NULL`)
517571 .bind(...ids)
518572 .first<{ n: number }>(),
573+ this.db
574+ .prepare(`SELECT DISTINCT c.page_id FROM page_changes c JOIN pages p ON p.id = c.page_id WHERE c.cleared_at IS NULL AND p.space_id IN (${marks}) AND p.archived_at IS NULL`)
575+ .bind(...ids)
576+ .all<{ page_id: string }>(),
519577 ]);
520578 const bySpace = new Map(spaces.map((s) => [s.row.id, s.row]));
521579 const ref = (r: Pick<PageRow, "id" | "space_id" | "title" | "icon">) => this.ref(workspace.slug, bySpace.get(r.space_id)!, r);
580+ const staleSet = new Set(stale.results.map((r) => r.page_id));
522581 return ok({
523582 spaces: spaces.map((s) => {
524583 const mine = pages.results.filter((p) => p.space_id === s.row.id);
525584 return {
526585 ...this.toSpace(s, mine.length),
527− pages: mine.map((p): DocTreeNode => ({ id: p.id, parent_id: p.parent_id, position: p.position, title: p.title, icon: p.icon, slug: pageSlug(p.title, p.id) })),
586+ pages: mine.map((p): DocTreeNode => ({ id: p.id, parent_id: p.parent_id, position: p.position, title: p.title, icon: p.icon, slug: pageSlug(p.title, p.id), stale: staleSet.has(p.id) })),
528587 };
529588 }),
530589 favorites: favorites.results.map(ref),
531590 recent: recent.results.map(ref),
532591 can_create_space: true,
533592 trash_count: trash?.n ?? 0,
593+ stale_count: staleSet.size,
594+ repos,
595+ });
596+ }
597+
598+ // ── A project's docs ────────────────────────────────────────────────────
599+
600+ /** The repository docs shown in the workspace that the viewer can read, with the repositories as they are now. */
601+ private async readableRepoSpaces(workspace: Workspace, viewer: User): Promise<{ row: RepoSpaceRow; repo: Repo }[]> {
602+ const rows = (await this.db.prepare("SELECT * FROM repo_spaces WHERE workspace_id = ? ORDER BY repo").bind(workspace.id).all<RepoSpaceRow>()).results;
603+ if (!rows.length || !this.env.REPOS) return [];
604+ const readable = await reposClient(this.env.REPOS).readable(
605+ rows.map((r) => r.repo_id),
606+ viewer,
607+ );
608+ const byId = new Map(readable.map((r) => [r.id, r]));
609+ return rows.filter((r) => byId.has(r.repo_id)).map((row) => ({ row, repo: byId.get(row.repo_id)! }));
610+ }
611+
612+ private async toRepoSpaces(workspace: Workspace, viewer: User, found: { row: RepoSpaceRow; repo: Repo }[]): Promise<DocRepoSpace[]> {
613+ if (!found.length) return [];
614+ const ids = found.map((f) => f.row.id);
615+ const [files, people] = await Promise.all([
616+ this.db
617+ .prepare(`SELECT space_id, path, title FROM repo_files WHERE space_id IN (${ids.map(() => "?").join(",")})`)
618+ .bind(...ids)
619+ .all<{ space_id: string; path: string; title: string }>(),
620+ this.profiles(
621+ workspace,
622+ found.map((f) => f.row.added_by),
623+ ),
624+ ]);
625+ const me = this.userKey(viewer);
626+ const owner = this.viewerOwner(viewer, workspace.slug);
627+ const readme = (path: string) => (/^readme\./i.test(path) ? 0 : 1);
628+ return found.map(({ row, repo }) => ({
629+ id: row.id,
630+ repo: `${repo.namespace}/${repo.name}`,
631+ default_branch: repo.defaultBranch,
632+ commit: row.commit_sha,
633+ indexed_at: row.indexed_at,
634+ added_by: people.get(row.added_by)!,
635+ files: files.results
636+ .filter((f) => f.space_id === row.id)
637+ .sort((a, b) => readme(a.path) - readme(b.path) || a.path.localeCompare(b.path))
638+ .map((f) => ({ path: f.path, title: f.title })),
639+ can_remove: row.added_by === me || owner,
640+ }));
641+ }
642+
643+ private async repoSpacesFor(workspace: Workspace, viewer: User): Promise<DocRepoSpace[]> {
644+ return this.toRepoSpaces(workspace, viewer, await this.readableRepoSpaces(workspace, viewer));
645+ }
646+
647+ async addRepoSpace(a: { workspace: string; viewer: Viewer; repo: string }): Promise<Result<DocRepoSpace>> {
648+ const found = await this.viewerWorkspace(a.workspace, a.viewer);
649+ if (!found.ok) return found;
650+ const workspace = found.value;
651+ const viewer = a.viewer!;
652+ if (!this.env.REPOS) return fail("conflict", "Projects' docs aren't available here.");
653+ const ref = projectRef(String(a.repo ?? ""));
654+ if (!ref) return fail("invalid", "Choose a repository: owner/name.");
655+ const [namespace, name] = ref.split("/") as [string, string];
656+ const repo = await reposClient(this.env.REPOS).get({ namespace, name }, viewer);
657+ if (!repo.ok) return fail("not_found", "No such repository, or you can't read it.");
658+ const id = newId("rds");
659+ const row: RepoSpaceRow = {
660+ id,
661+ workspace_id: workspace.id,
662+ repo_id: repo.value.id,
663+ repo: `${repo.value.namespace}/${repo.value.name}`.toLowerCase(),
664+ default_branch: repo.value.defaultBranch,
665+ commit_sha: null,
666+ indexed_at: null,
667+ added_by: this.userKey(viewer),
668+ added_at: now(),
669+ };
670+ const inserted = await this.db
671+ .prepare("INSERT INTO repo_spaces (id, workspace_id, repo_id, repo, default_branch, added_by, added_at) VALUES (?, ?, ?, ?, ?, ?, ?) ON CONFLICT (workspace_id, repo_id) DO NOTHING RETURNING id")
672+ .bind(row.id, row.workspace_id, row.repo_id, row.repo, row.default_branch, row.added_by, row.added_at)
673+ .first<{ id: string }>();
674+ if (!inserted) return fail("conflict", `${ref}'s docs are already in Docs.`);
675+ try {
676+ await indexRepoSpace({ DB: this.db, REPOS: this.env.REPOS }, row);
677+ } catch (error) {
678+ console.error("docs could not read a project's docs", row.repo, String(error));
679+ }
680+ const fresh = (await this.db.prepare("SELECT * FROM repo_spaces WHERE id = ?").bind(id).first<RepoSpaceRow>()) ?? row;
681+ const [space] = await this.toRepoSpaces(workspace, viewer, [{ row: fresh, repo: repo.value }]);
682+ return ok(space!);
683+ }
684+
685+ async removeRepoSpace(a: { workspace: string; viewer: Viewer; id: string }): Promise<Result<boolean>> {
686+ const found = await this.viewerWorkspace(a.workspace, a.viewer);
687+ if (!found.ok) return found;
688+ const row = await this.db.prepare("SELECT * FROM repo_spaces WHERE id = ? AND workspace_id = ?").bind(String(a.id ?? ""), found.value.id).first<RepoSpaceRow>();
689+ if (!row) return fail("not_found", "No such project's docs.");
690+ if (row.added_by !== this.userKey(a.viewer!) && !this.viewerOwner(a.viewer!, a.workspace)) return fail("forbidden", "Only whoever added a project's docs, or an owner, can remove them.");
691+ await this.db.batch([this.db.prepare("DELETE FROM repo_files_fts WHERE space_id = ?").bind(row.id), this.db.prepare("DELETE FROM repo_spaces WHERE id = ?").bind(row.id)]);
692+ return ok(true);
693+ }
694+
695+ async repoPage(a: { workspace: string; viewer: Viewer; repo: string; path: string }): Promise<Result<DocRepoPage>> {
696+ const found = await this.viewerWorkspace(a.workspace, a.viewer);
697+ if (!found.ok) return found;
698+ const workspace = found.value;
699+ const ref = projectRef(String(a.repo ?? ""));
700+ if (!ref) return fail("not_found", "No such file.");
701+ const spaces = await this.readableRepoSpaces(workspace, a.viewer!);
702+ const match = spaces.find((s) => `${s.repo.namespace}/${s.repo.name}`.toLowerCase() === ref || s.row.repo === ref);
703+ if (!match) return fail("not_found", "No such file.");
704+ const path = String(a.path ?? "").replace(/^\/+/, "");
705+ const file = await this.db.prepare("SELECT path, title, markdown FROM repo_files WHERE space_id = ? AND path = ?").bind(match.row.id, path).first<{ path: string; title: string; markdown: string }>();
706+ if (!file) return fail("not_found", "No such file.");
707+ const [space] = await this.toRepoSpaces(workspace, a.viewer!, [match]);
708+ const repoPath = `${match.repo.namespace}/${match.repo.name}`;
709+ const encoded = file.path.split("/").map(encodeURIComponent).join("/");
710+ return ok({
711+ space: space!,
712+ file: {
713+ path: file.path,
714+ title: file.title,
715+ markdown: file.markdown,
716+ href: `/${workspace.slug}/-/docs/repo/${repoPath}/${encoded}`,
717+ code_href: `/${repoPath}/blob/${encodeURIComponent(match.repo.defaultBranch)}/${encoded}`,
718+ },
534719 });
535720 }
536721
544729 const project = a.project ? projectRef(a.project) : null;
545730 const ids = spaces.map((s) => s.row.id);
546731 const allProjects = new Set(spaces.flatMap((s) => s.projects));
547− if (!ids.length) return ok({ recent: [], mine: [], spaces: [], projects: [...allProjects].sort(), project });
732+ if (!ids.length) return ok({ recent: [], mine: [], stale: [], spaces: [], projects: [...allProjects].sort(), project });
548733 const marks = ids.map(() => "?").join(",");
549− const [recentRows, mineRows, pageProjects, counts] = await Promise.all([
734+ const [recentRows, mineRows, pageProjects, counts, staleRows] = await Promise.all([
550735 this.db
551736 .prepare(`SELECT id, workspace_id, space_id, parent_id, position, title, icon, cover, substr(markdown, 1, 600) AS markdown, created_by, created_at, updated_by, updated_at, archived_at, archived_by FROM pages WHERE space_id IN (${marks}) AND archived_at IS NULL ORDER BY updated_at DESC LIMIT 60`)
552737 .bind(...ids)
565750 .prepare(`SELECT space_id, COUNT(*) AS n FROM pages WHERE space_id IN (${marks}) AND archived_at IS NULL GROUP BY space_id`)
566751 .bind(...ids)
567752 .all<{ space_id: string; n: number }>(),
753+ this.staleRows(ids, null, 24),
568754 ]);
569755 for (const p of pageProjects.results) allProjects.add(p.repo);
570756 const projectsOf = (pageId: string) => pageProjects.results.filter((p) => p.page_id === pageId).map((p) => p.repo);
571757 const spaceProjects = new Map(spaces.map((s) => [s.row.id, s.projects]));
572758 const keep = (r: PageRow) => inProject(project, projectsOf(r.id), spaceProjects.get(r.space_id) ?? []);
573759 const bySpace = new Map(spaces.map((s) => [s.row.id, s.row]));
574− const [recent, mine] = await Promise.all([this.toPages(workspace, bySpace, recentRows.results.filter(keep).slice(0, 12)), this.toPages(workspace, bySpace, mineRows.results.filter(keep).slice(0, 8))]);
760+ const [recent, mine, stale] = await Promise.all([
761+ this.toPages(workspace, bySpace, recentRows.results.filter(keep).slice(0, 12)),
762+ this.toPages(workspace, bySpace, mineRows.results.filter(keep).slice(0, 8)),
763+ this.toPages(workspace, bySpace, staleRows.filter(keep).slice(0, 8)),
764+ ]);
575765 const count = new Map(counts.results.map((c) => [c.space_id, c.n]));
576766 return ok({
577767 recent,
578768 mine,
769+ stale,
579770 spaces: spaces.filter((s) => !project || s.projects.includes(project) || recentRows.results.some((r) => r.space_id === s.row.id && keep(r))).map((s) => this.toSpace(s, count.get(s.row.id) ?? 0)),
580771 projects: [...allProjects].sort(),
581772 project,
769960 const viewer = a.viewer!;
770961 const bySpace = new Map(spaces.map((s) => [s.row.id, s.row]));
771962 const readable = new Set(spaces.filter((s) => s.role).map((s) => s.row.id));
772− const [tree, backlinks, children, favorite, viewed, suggestions] = await Promise.all([
963+ const [tree, backlinks, children, favorite, viewed, suggestions, cited, staleness] = await Promise.all([
773964 this.db.prepare("SELECT id, space_id, parent_id, position, title, icon FROM pages WHERE space_id = ?").bind(page.space_id).all<PageRow>(),
774965 this.db
775966 .prepare("SELECT p.id, p.space_id, p.title, p.icon FROM page_links l JOIN pages p ON p.id = l.from_page WHERE l.to_page = ? AND p.archived_at IS NULL LIMIT 50")
779970 this.db.prepare("SELECT 1 AS yes FROM favorites WHERE user_id = ? AND page_id = ?").bind(viewer.id, page.id).first<{ yes: number }>(),
780971 this.db.prepare("SELECT viewed_at FROM page_views WHERE user_id = ? AND page_id = ?").bind(viewer.id, page.id).first<{ viewed_at: string }>(),
781972 this.openSuggestions(workspace, page.id),
973+ this.citationsOf([page.id]),
974+ this.stalenessFor(page.id, viewer),
782975 ]);
783976 this.defer(
784977 this.db
798991 favorite: !!favorite,
799992 last_viewed_at: viewed?.viewed_at ?? null,
800993 suggestions,
994+ citations: cited.get(page.id) ?? [],
995+ describes: (cited.get(page.id) ?? []).filter((c) => c.source === "header").map((c) => ({ repo: c.repo, path: c.path })),
996+ staleness,
801997 });
802998 }
803999
1000+ // ── Citations and staleness ─────────────────────────────────────────────
1001+
1002+ /** Each page's citations, by page. */
1003+ private async citationsOf(pageIds: string[]): Promise<Map<string, DocCitation[]>> {
1004+ const out = new Map<string, DocCitation[]>();
1005+ if (!pageIds.length) return out;
1006+ const rows = await this.db
1007+ .prepare(`SELECT page_id, repo, path, kind, label, ref, source FROM citations WHERE page_id IN (${pageIds.map(() => "?").join(",")}) ORDER BY source DESC, repo, path`)
1008+ .bind(...pageIds)
1009+ .all<Omit<DocCitation, "label"> & { page_id: string; label: string }>();
1010+ for (const r of rows.results) {
1011+ const list = out.get(r.page_id) ?? [];
1012+ list.push({ repo: r.repo, path: r.path, kind: r.kind, label: r.label || null, ref: r.ref, source: r.source });
1013+ out.set(r.page_id, list);
1014+ }
1015+ return out;
1016+ }
1017+
1018+ /** Of these repositories (`owner/name`), those the viewer can read. */
1019+ private async readableRepos(viewer: User, repos: string[]): Promise<Set<string>> {
1020+ const out = new Set<string>();
1021+ if (!this.env.REPOS) return out;
1022+ const client = reposClient(this.env.REPOS);
1023+ await Promise.all(
1024+ [...new Set(repos)].slice(0, 25).map(async (repo) => {
1025+ const [namespace, name] = repo.split("/") as [string, string];
1026+ const found = await client.get({ namespace, name }, viewer).catch(() => null);
1027+ if (found?.ok) out.add(repo);
1028+ }),
1029+ );
1030+ return out;
1031+ }
1032+
1033+ /** Open changes on these pages, newest first. */
1034+ private async openChanges(pageIds: string[]): Promise<ChangeRow[]> {
1035+ if (!pageIds.length) return [];
1036+ const out: ChangeRow[] = [];
1037+ for (let i = 0; i < pageIds.length; i += 90) {
1038+ const part = pageIds.slice(i, i + 90);
1039+ const rows = await this.db
1040+ .prepare(`SELECT * FROM page_changes WHERE cleared_at IS NULL AND page_id IN (${part.map(() => "?").join(",")}) ORDER BY detected_at DESC`)
1041+ .bind(...part)
1042+ .all<ChangeRow>();
1043+ out.push(...rows.results);
1044+ }
1045+ return out.sort((a, b) => b.detected_at.localeCompare(a.detected_at));
1046+ }
1047+
1048+ /** A change as a reader sees it: named only when they can read its repository. */
1049+ private toChange(row: ChangeRow, readable: Set<string>): DocStaleChange {
1050+ if (!readable.has(row.repo)) return { visible: false, repo: null, commit: null, pull: null, paths: [], at: row.detected_at };
1051+ let paths: string[] = [];
1052+ try {
1053+ paths = JSON.parse(row.paths) as string[];
1054+ } catch {
1055+ paths = [];
1056+ }
1057+ return {
1058+ visible: true,
1059+ repo: row.repo,
1060+ commit: row.commit_sha,
1061+ pull: row.pull_number ? { number: row.pull_number, title: row.pull_title } : null,
1062+ paths,
1063+ at: row.detected_at,
1064+ };
1065+ }
1066+
1067+ /** Why a page is possibly out of date, as this viewer may see it; null when it isn't. */
1068+ private async stalenessFor(pageId: string, viewer: User): Promise<DocStaleness | null> {
1069+ const rows = await this.openChanges([pageId]);
1070+ if (!rows.length) return null;
1071+ const readable = await this.readableRepos(
1072+ viewer,
1073+ rows.map((r) => r.repo),
1074+ );
1075+ const changes = rows.slice(0, 20).map((r) => this.toChange(r, readable));
1076+ return { since: rows[rows.length - 1]!.detected_at, changes };
1077+ }
1078+
1079+ /** Live pages in these spaces that are possibly out of date, most recently flagged first; `repo` narrows to changes there. */
1080+ private async staleRows(spaceIds: string[], repo: string | null, limit: number): Promise<PageRow[]> {
1081+ if (!spaceIds.length) return [];
1082+ const marks = spaceIds.map(() => "?").join(",");
1083+ return (
1084+ await this.db
1085+ .prepare(
1086+ `SELECT p.id, p.workspace_id, p.space_id, p.parent_id, p.position, p.title, p.icon, p.cover, substr(p.markdown, 1, 600) AS markdown, p.created_by, p.created_at, p.updated_by, p.updated_at, p.archived_at, p.archived_by
1087+ FROM pages p JOIN (SELECT page_id, MAX(detected_at) AS flagged FROM page_changes WHERE cleared_at IS NULL ${repo ? "AND repo = ?" : ""} GROUP BY page_id) c ON c.page_id = p.id
1088+ WHERE p.space_id IN (${marks}) AND p.archived_at IS NULL ORDER BY c.flagged DESC LIMIT ?`,
1089+ )
1090+ .bind(...(repo ? [repo] : []), ...spaceIds, limit)
1091+ .all<PageRow>()
1092+ ).results;
1093+ }
1094+
1095+ async stalePages(a: { workspace: string; viewer: Viewer; repo?: string | null }): Promise<Result<DocPage[]>> {
1096+ const found = await this.viewerWorkspace(a.workspace, a.viewer);
1097+ if (!found.ok) return found;
1098+ const workspace = found.value;
1099+ const spaces = (await this.spacesFor(workspace, a.viewer!)).filter((s) => s.role);
1100+ const rows = await this.staleRows(
1101+ spaces.map((s) => s.row.id),
1102+ a.repo ? projectRef(a.repo) : null,
1103+ 200,
1104+ );
1105+ return ok(await this.toPages(workspace, new Map(spaces.map((s) => [s.row.id, s.row])), rows));
1106+ }
1107+
1108+ /** Clears every open change on a page. */
1109+ private async clearStale(pageId: string, by: string): Promise<boolean> {
1110+ const done = await this.db.prepare("UPDATE page_changes SET cleared_at = ?, cleared_by = ? WHERE page_id = ? AND cleared_at IS NULL").bind(now(), by, pageId).run();
1111+ const cleared = (done.meta?.changes ?? 0) > 0;
1112+ if (cleared) this.tell(pageId, { type: "page.staleness" });
1113+ return cleared;
1114+ }
1115+
1116+ async markCurrent(a: { workspace: string; page_id: string; viewer: Viewer }): Promise<Result<boolean>> {
1117+ const found = await this.pageFor(a.workspace, a.page_id, a.viewer, "edit");
1118+ if (!found.ok) return found;
1119+ await this.clearStale(found.value.page.id, this.userKey(a.viewer!));
1120+ return ok(true);
1121+ }
1122+
1123+ async stalePagesForAgent(a: { workspace: string; agent_id: string; viewer: Viewer; repo?: string | null; since?: string | null; audience: DocAudience | null }): Promise<Result<DocStalePage[]>> {
1124+ const found = await this.agentSpaces(a.workspace, a.agent_id, a.viewer, a.audience);
1125+ if (!found.ok) return found;
1126+ const { workspace, spaces } = found.value;
1127+ const repo = a.repo ? projectRef(a.repo) : null;
1128+ if (a.repo && !repo) return fail("invalid", "Name the repository as owner/name.");
1129+ const since = a.since && !Number.isNaN(Date.parse(a.since)) ? new Date(a.since).toISOString() : null;
1130+ const rows = await this.staleRows(
1131+ spaces.map((s) => s.row.id),
1132+ repo,
1133+ 200,
1134+ );
1135+ const [changes, cited, pages] = await Promise.all([
1136+ this.openChanges(rows.map((r) => r.id)),
1137+ this.citationsOf(rows.map((r) => r.id)),
1138+ this.toPages(workspace, new Map(spaces.map((s) => [s.row.id, s.row])), rows),
1139+ ]);
1140+ // The agent learns only of code its person can read.
1141+ const readable = await this.readableRepos(a.viewer!, [...changes.map((c) => c.repo), ...[...cited.values()].flat().map((c) => c.repo)]);
1142+ const bySpace = new Map(spaces.map((s) => [s.row.id, s]));
1143+ const out: DocStalePage[] = [];
1144+ for (const row of rows) {
1145+ const mine = changes.filter((c) => c.page_id === row.id && readable.has(c.repo) && (!repo || c.repo === repo));
1146+ if (!mine.length) continue;
1147+ const newest = mine[0]!.detected_at;
1148+ if (since && newest < since) continue;
1149+ const space = bySpace.get(row.space_id)!;
1150+ const page = pages.find((p) => p.id === row.id)!;
1151+ out.push({
1152+ page: { ...this.ref(workspace.slug, space.row, row), updated_at: row.updated_at },
1153+ space: { id: space.row.id, slug: space.row.slug, name: space.row.name, agent_mode: space.row.agent_mode },
1154+ can: space.can,
1155+ owners: page.owners,
1156+ citations: (cited.get(row.id) ?? []).filter((c) => readable.has(c.repo)),
1157+ changes: mine.slice(0, 20).map((c) => this.toChange(c, readable)),
1158+ since: mine[mine.length - 1]!.detected_at,
1159+ });
1160+ if (out.length >= 50) break;
1161+ }
1162+ return ok(out);
1163+ }
1164+
8041165 /** Where a new page in `space` from `input` starts: its Markdown and title. */
8051166 private async startingPoint(workspace: Workspace, input: NewDocPage): Promise<{ markdown: string; title: string; icon: string | null }> {
8061167 let markdown = String(input.markdown ?? "").slice(0, MAX_MARKDOWN);
8691230 .bind(newId("ver"), id, at, JSON.stringify([author]), input.markdown),
8701231 ]);
8711232 await this.room(id).ensure({ page_id: id, workspace_slug: workspace.slug, markdown: input.markdown, state: input.state ?? null });
1233+ this.defer(publishDocEvent(this.env.EVENTS, "doc.page.created", this.eventData(workspace, space, row), author));
8721234 return row;
8731235 }
8741236
1237+ /** What every `doc.page.*` event says of a page. */
1238+ private eventData(workspace: Workspace, space: Pick<SpaceRow, "id" | "slug">, row: Pick<PageRow, "id" | "title" | "icon">) {
1239+ return { workspace: workspace.slug, workspaceId: workspace.id, pageId: row.id, spaceId: space.id, title: row.title, path: this.ref(workspace.slug, space, row).path };
1240+ }
1241+
8751242 async createPage(a: { workspace: string; viewer: Viewer; input: NewDocPage }): Promise<Result<DocPage>> {
8761243 const found = await this.viewerWorkspace(a.workspace, a.viewer);
8771244 if (!found.ok) return found;
9271294 statements.push(this.db.prepare("DELETE FROM page_projects WHERE page_id = ?").bind(page.id));
9281295 for (const repo of cleanProjects(c.projects)) statements.push(this.db.prepare("INSERT INTO page_projects (page_id, repo) VALUES (?, ?)").bind(page.id, repo));
9291296 }
1297+ if (c.describes !== undefined) {
1298+ statements.push(this.db.prepare("DELETE FROM citations WHERE page_id = ? AND source = 'header'").bind(page.id));
1299+ for (const d of cleanDescribes(c.describes)) {
1300+ statements.push(this.db.prepare("INSERT OR IGNORE INTO citations (page_id, repo, path, kind, label, ref, source) VALUES (?, ?, ?, 'path', '', NULL, 'header')").bind(page.id, d.repo, d.path));
1301+ }
1302+ }
9301303 if (c.owners !== undefined) {
9311304 const owners = [...new Set((Array.isArray(c.owners) ? c.owners : []).map(String).filter((k) => memberKey(k)?.kind === "user" || memberKey(k)?.kind === "agent"))].slice(0, 20);
9321305 statements.push(this.db.prepare("DELETE FROM page_owners WHERE page_id = ?").bind(page.id));
9991372 const at = now();
10001373 await this.db.batch(ids.map((id) => this.db.prepare("UPDATE pages SET archived_at = ?, archived_by = ? WHERE id = ? AND archived_at IS NULL").bind(at, this.userKey(a.viewer!), id)));
10011374 for (const id of ids) this.defer(this.room(id).closeAll("Moved to the trash").catch(() => undefined));
1375+ this.defer(publishDocEvent(this.env.EVENTS, "doc.page.archived", this.eventData(workspace, space.row, page), this.userKey(a.viewer!)));
10021376 const after = await this.db.prepare("SELECT * FROM pages WHERE id = ?").bind(page.id).first<PageRow>();
10031377 const [detail] = await this.toPages(workspace, new Map([[space.row.id, space.row]]), [after!]);
10041378 return ok(detail!);
11261500 const found = await this.viewerWorkspace(a.workspace, a.viewer);
11271501 if (!found.ok) return found;
11281502 const workspace = found.value;
1503+ const query = a.query ?? { query: "" };
11291504 const spaces = (await this.spacesFor(workspace, a.viewer!)).filter((s) => s.role);
1130− return ok(await this.searchIn(workspace, spaces, a.query ?? { query: "" }));
1505+ const [pages, files] = await Promise.all([
1506+ this.searchIn(workspace, spaces, query),
1507+ // A project's docs, when the search isn't narrowed to one of the workspace's spaces.
1508+ query.space_id
1509+ ? Promise.resolve([] as DocSearchHit[])
1510+ : this.searchRepoFiles(workspace, a.viewer!, query).catch((error: unknown) => {
1511+ console.error("docs could not search projects' docs", String(error));
1512+ return [] as DocSearchHit[];
1513+ }),
1514+ ]);
1515+ const limit = Math.min(Math.max(Number(query.limit) || 20, 1), 50);
1516+ // Pages first, then files, as many as asked for.
1517+ return ok([...pages, ...files].slice(0, limit));
1518+ }
1519+
1520+ /** Full text over the projects' docs the viewer can read. */
1521+ private async searchRepoFiles(workspace: Workspace, viewer: User, query: DocSearchQuery): Promise<DocSearchHit[]> {
1522+ const q = ftsQuery(query.query);
1523+ if (!q) return [];
1524+ let spaces = await this.readableRepoSpaces(workspace, viewer);
1525+ const project = query.project ? projectRef(query.project) : null;
1526+ if (project) spaces = spaces.filter((s) => `${s.repo.namespace}/${s.repo.name}`.toLowerCase() === project);
1527+ if (!spaces.length) return [];
1528+ const limit = Math.min(Math.max(Number(query.limit) || 20, 1), 50);
1529+ const rows = (
1530+ await this.db
1531+ .prepare(
1532+ `SELECT space_id, path, title, snippet(repo_files_fts, 3, '[[', ']]', '…', 16) AS snippet FROM repo_files_fts
1533+ WHERE repo_files_fts MATCH ? AND space_id IN (${spaces.map(() => "?").join(",")}) ORDER BY bm25(repo_files_fts, 0, 0, 8.0, 1.0) LIMIT ?`,
1534+ )
1535+ .bind(q, ...spaces.map((s) => s.row.id), limit)
1536+ .all<{ space_id: string; path: string; title: string; snippet: string }>()
1537+ ).results;
1538+ const byId = new Map(spaces.map((s) => [s.row.id, s]));
1539+ return rows.map((r) => {
1540+ const s = byId.get(r.space_id)!;
1541+ const repo = `${s.repo.namespace}/${s.repo.name}`;
1542+ return {
1543+ id: `repo:${r.space_id}:${r.path}`,
1544+ space_id: r.space_id,
1545+ space_slug: "repo",
1546+ title: r.title,
1547+ icon: null,
1548+ slug: r.path,
1549+ path: `/${workspace.slug}/-/docs/repo/${repo}/${r.path.split("/").map(encodeURIComponent).join("/")}`,
1550+ space_name: repo,
1551+ snippet: r.snippet,
1552+ updated_at: s.row.indexed_at ?? s.row.added_at,
1553+ projects: [repo.toLowerCase()],
1554+ repo_file: { repo, path: r.path },
1555+ };
1556+ });
11311557 }
11321558
11331559 // ── History ─────────────────────────────────────────────────────────────
13401766 authors: [row.author, me],
13411767 });
13421768 if (!result.applied) status = "stale";
1769+ else if (row.marks_current) await this.clearStale(page.id, row.author);
13431770 }
13441771 await this.db.prepare("UPDATE suggestions SET status = ?, decided_by = ?, decided_at = ? WHERE id = ?").bind(status, me, now(), row.id).run();
13451772 const [after] = await this.toSuggestions(workspace, [{ ...row, status, decided_by: me, decided_at: now() }]);
15261953 space: SpaceRow,
15271954 agent: WorkspaceAgent,
15281955 viewer: User,
1529− edit: { target: DocEditTarget; markdown: string; note: string | null },
1956+ edit: { target: DocEditTarget; markdown: string; note: string | null; marks_current: boolean },
15301957 ): Promise<Result<DocSuggestion>> {
15311958 const room = this.room(page.id);
15321959 await room.ensure({ page_id: page.id, workspace_slug: workspace.slug, markdown: page.markdown });
15461973 created_at: now(),
15471974 decided_by: null,
15481975 decided_at: null,
1976+ marks_current: edit.marks_current ? 1 : 0,
15491977 };
15501978 await this.db
1551− .prepare("INSERT INTO suggestions (id, page_id, author, asked_by, target, before_markdown, after_markdown, note, status, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'open', ?)")
1552− .bind(row.id, row.page_id, row.author, row.asked_by, row.target, row.before_markdown, row.after_markdown, row.note, row.created_at)
1979+ .prepare("INSERT INTO suggestions (id, page_id, author, asked_by, target, before_markdown, after_markdown, note, status, created_at, marks_current) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'open', ?, ?)")
1980+ .bind(row.id, row.page_id, row.author, row.asked_by, row.target, row.before_markdown, row.after_markdown, row.note, row.created_at, row.marks_current)
15531981 .run();
15541982 const [suggestion] = await this.toSuggestions(workspace, [row], [current.block_ids]);
15551983 this.tell(page.id, { type: "suggestion.created", suggestion: suggestion! });
15892017 );
15902018 }
15912019
1592− private cleanEdit(edit: unknown): Result<{ target: DocEditTarget; markdown: string; note: string | null }> {
1593− const e = (edit ?? {}) as { target?: unknown; markdown?: unknown; note?: unknown };
2020+ private cleanEdit(edit: unknown): Result<{ target: DocEditTarget; markdown: string; note: string | null; marks_current: boolean }> {
2021+ const e = (edit ?? {}) as { target?: unknown; markdown?: unknown; note?: unknown; marks_current?: unknown };
15942022 const target = cleanTarget(e.target);
15952023 if (!target) return fail("invalid", "Say what to change: append, document, a section by its heading, or blocks by id.");
15962024 const markdown = String(e.markdown ?? "");
15972025 if (markdown.length > MAX_MARKDOWN) return fail("invalid", "That edit is too long.");
15982026 if (target.kind === "append" && !markdown.trim()) return fail("invalid", "Nothing to add.");
1599− return ok({ target, markdown, note: e.note ? String(e.note).trim().slice(0, MAX_NOTE) || null : null });
2027+ return ok({ target, markdown, note: e.note ? String(e.note).trim().slice(0, MAX_NOTE) || null : null, marks_current: e.marks_current === true });
16002028 }
16012029
16022030 async suggestEdit(a: { workspace: string; agent_id: string; viewer: Viewer; page_id: string; edit: unknown }): Promise<Result<DocSuggestion>> {
16302058 authors: [principalKey({ kind: "agent", id: agent.id })],
16312059 });
16322060 if (!result.applied) return fail("not_found", "That part of the page isn't there. Read the page again and target what is there now.");
2061+ if (edit.value.marks_current) await this.clearStale(page.id, principalKey({ kind: "agent", id: agent.id }));
16332062 this.defer(room.announce(principalKey({ kind: "agent", id: agent.id }), agent.display_name).catch(() => undefined));
16342063 return ok({ mode: "applied", version_id: result.version_id, page: ref });
16352064 }
17222151 const random = crypto.getRandomValues(new Uint8Array(32));
17232152 const key = [...random].map((b) => b.toString(16).padStart(2, "0")).join("");
17242153 const id = newId("fil");
1725− await r2FileStore(this.env.FILES).put(`docs/${key}`, request.body ?? new Uint8Array(), contentType);
2154+ await fileStore(this.env).put(`docs/${key}`, request.body ?? new Uint8Array(), contentType);
17262155 await this.db
17272156 .prepare("INSERT INTO files (id, workspace_id, page_id, key, name, content_type, bytes, created_by, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)")
17282157 .bind(id, found.value.workspace.id, found.value.page.id, key, name, contentType, bytes, this.userKey(viewer!), now())
17392168 async file(key: string): Promise<Response> {
17402169 const row = await this.db.prepare("SELECT name, content_type FROM files WHERE key = ?").bind(key).first<{ name: string; content_type: string }>();
17412170 if (!row) return new Response("Not found\n", { status: 404 });
1742− const stored = await r2FileStore(this.env.FILES).get(`docs/${key}`);
2171+ const stored = await fileStore(this.env).get(`docs/${key}`);
17432172 if (!stored) return new Response("Not found\n", { status: 404 });
17442173 const inline = row.content_type !== "application/octet-stream";
17452174 return new Response(stored.body, {
18312260 return Response.json(await service.createPageAsAgent(args));
18322261 case "threads_for_agent":
18332262 return Response.json(await service.threadsForAgent(args));
2263+ case "stale_pages_for_agent":
2264+ return Response.json(await service.stalePagesForAgent(args));
2265+ case "mark_current":
2266+ return Response.json(await service.markCurrent(args));
2267+ case "stale_pages":
2268+ return Response.json(await service.stalePages(args));
2269+ case "add_repo_space":
2270+ return Response.json(await service.addRepoSpace(args));
2271+ case "remove_repo_space":
2272+ return Response.json(await service.removeRepoSpace(args));
2273+ case "repo_page":
2274+ return Response.json(await service.repoPage(args));
18342275 default:
18352276 return new Response("Unknown method\n", { status: 404 });
18362277 }
18572298 return opened.finish(Response.json(fail("conflict", "Docs couldn't do that just now. Try again.")));
18582299 }
18592300 },
1860−} satisfies ExportedHandler<Env>;
2301+
2302+ /**
2303+ * Events from the events service (SUBSCRIBER_DOCS): pages whose cited
2304+ * code changed become possibly out of date, and projects' docs are read
2305+ * again after a push (src/staleness.ts). One failing event is retried on
2306+ * its own.
2307+ */
2308+ async queue(batch: MessageBatch<G1tEvent>, env: Env): Promise<void> {
2309+ const reindex = async (repoId: string) => {
2310+ if (env.REPOS) await reindexRepo({ DB: env.DB, REPOS: env.REPOS }, repoId);
2311+ };
2312+ for (const message of batch.messages) {
2313+ try {
2314+ await onEvent(env, message.body, reindex);
2315+ message.ack();
2316+ } catch (error) {
2317+ console.error("docs could not handle", message.body?.type, String(error));
2318+ message.retry();
2319+ }
2320+ }
2321+ },
2322+} satisfies ExportedHandler<Env, G1tEvent>;
+25−1
44 import * as Y from "yjs";
55
66 import { parseInline, parseMarkdown, seed } from "./blocks.ts";
7−import { documentMarkdown, excerpt, mentionedIds, outline, searchText, topContainers } from "./markdown.ts";
7+import { bodyCitations } from "./citations.ts";
8+import { citationNodes, documentMarkdown, excerpt, mentionedIds, outline, searchText, topContainers } from "./markdown.ts";
89
910 function docFrom(markdown: string) {
1011 const doc = new Y.Doc();
126127 assert.equal(searchText("## Steps\n\n- [x] **Ship** [it](https://x)"), "Steps\n Ship it");
127128 assert.equal(excerpt("# T\n\n" + "word ".repeat(100), 20).length, 20);
128129 });
130+
131+test("a citation chip is a link to the code, and is found as a citation", () => {
132+ const doc = new Y.Doc();
133+ const fragment = doc.getXmlFragment("document-store");
134+ seed(doc, fragment, "Exports run in ");
135+ doc.transact(() => {
136+ const paragraph = topContainers(fragment)[0]!.get(0) as Y.XmlElement;
137+ const chip = new Y.XmlElement("citation");
138+ chip.setAttribute("repo", "acme/web");
139+ chip.setAttribute("path", "src/export.ts");
140+ chip.setAttribute("kind", "symbol");
141+ chip.setAttribute("label", "exportCsv");
142+ chip.setAttribute("ref", "abc1234");
143+ paragraph.insert(paragraph.length, [chip]);
144+ });
145+ const markdown = documentMarkdown(fragment);
146+ assert.equal(markdown, "Exports run in [`exportCsv`](/acme/web/blob/abc1234/src/export.ts)\n");
147+ const found = bodyCitations(citationNodes(fragment), markdown);
148+ assert.deepEqual(
149+ found.map((c) => [c.repo, c.path, c.kind, c.label]),
150+ [["acme/web", "src/export.ts", "symbol", "exportCsv"]],
151+ );
152+});
+24−2
1414 * `callout` (`kind`: info | warning | success | danger), `mermaid`
1515 * (`code`), `math` (`expression`), `embed` (`kind`, `title`, `url`), and
1616 * inline `mention` (`kind`: user | agent | page; `id`: a person's
17− * username, an agent's id or a page's id; `name`; `href` for a page) and
18− * `date` (`date`).
17+ * username, an agent's id or a page's id; `name`; `href` for a page),
18+ * `date` (`date`) and `citation` (`repo`, `path`, `kind`, `label`, `ref`:
19+ * code the page cites, src/citations.ts).
1920 */
21+import type { DocCitation } from "@g1t/contracts";
2022 import * as Y from "yjs";
2123
24+import { citationMarkdown, cleanCitation } from "./citations.ts";
25+
2226 export type Outline = { id: string; type: string; level: number | null; markdown: string };
2327
2428 /** The callout kinds, as GitHub's alert names. */
7478 return `@${a.name ?? ""}`;
7579 }
7680 if (node.nodeName === "date") return a.date ?? "";
81+ if (node.nodeName === "citation") {
82+ const c = cleanCitation(a as Partial<DocCitation>, "body");
83+ return c ? citationMarkdown(c) : "";
84+ }
7785 // An unknown inline node: its text, if any.
7886 return node.toArray().map((c) => (c instanceof Y.XmlText ? c.toString() : "")).join("");
7987 }
316324 walk(fragment);
317325 return { users: [...users], agents: [...agents] };
318326 }
327+
328+/** The citation chips in a document, as their attributes say (src/citations.ts cleans them). */
329+export function citationNodes(fragment: Y.XmlFragment): Partial<DocCitation>[] {
330+ const out: Partial<DocCitation>[] = [];
331+ const walk = (node: Y.XmlElement | Y.XmlFragment) => {
332+ for (const child of node.toArray()) {
333+ if (!(child instanceof Y.XmlElement)) continue;
334+ if (child.nodeName === "citation") out.push(child.getAttributes() as Partial<DocCitation>);
335+ else walk(child);
336+ }
337+ };
338+ walk(fragment);
339+ return out;
340+}
+32−2
44 * newly mentioned in the page. Run by the room (src/room.ts), which owns
55 * the live document; nothing here reads the document itself.
66 */
7−import { newId, notifyClient, type DocVersionKind, type FeedNotification, type ServiceBinding } from "@g1t/contracts";
7+import { newId, notifyClient, type DocCitation, type DocVersionKind, type FeedNotification, type ServiceBinding } from "@g1t/contracts";
88
9+import { publishDocEvent } from "./events.ts";
910 import { excerpt, searchText } from "./markdown.ts";
1011 import { linkedPageIds, pageSlug } from "./slugs.ts";
1112
1415 /** The largest Yjs state a version keeps; past it, only its Markdown. */
1516 const MAX_VERSION_STATE = 1_500_000;
1617
17−export type SaveEnv = { DB: D1Database; NOTIFY?: ServiceBinding };
18+export type SaveEnv = { DB: D1Database; NOTIFY?: ServiceBinding; EVENTS?: ServiceBinding };
1819
1920 export type Save = {
2021 page_id: string;
2122 markdown: string;
23+ /** Code the document cites (src/citations.ts `bodyCitations`). */
24+ citations: DocCitation[];
2225 /** The editors since the last save, member keys, last one last. */
2326 editors: string[];
2427 /** People mentioned in the document now (usernames, lowercased). */
6366 for (const to of linkedPageIds(input.markdown).filter((id) => id !== page.id).slice(0, 200)) {
6467 statements.push(env.DB.prepare("INSERT OR IGNORE INTO page_links (from_page, to_page) VALUES (?, ?)").bind(page.id, to));
6568 }
69+ // What it cites, from its text; the header's own stay.
70+ statements.push(env.DB.prepare("DELETE FROM citations WHERE page_id = ? AND source = 'body'").bind(page.id));
71+ for (const c of input.citations) {
72+ statements.push(
73+ env.DB.prepare("INSERT OR IGNORE INTO citations (page_id, repo, path, kind, label, ref, source) VALUES (?, ?, ?, ?, ?, ?, 'body')").bind(page.id, c.repo, c.path, c.kind, c.label ?? "", c.ref),
74+ );
75+ }
6676 }
6777 // A version: asked for (an agent's edit, a suggestion, a restore), or
6878 // the first save after enough time since the last one.
98108 statements.push(env.DB.prepare("UPDATE pages SET mentioned = ? WHERE id = ?").bind(JSON.stringify([...new Set([...told.filter((id) => input.mentioned.includes(id)), ...fresh])]), page.id));
99109 }
100110 if (statements.length) await env.DB.batch(statements);
111+ // A version is what the rest of g1t hears of: at most every ten minutes of editing, and each agent edit, suggestion and restore.
112+ const kind = input.version?.kind ?? "edit";
113+ if (versionId && kind !== "created" && input.workspace_slug && !page.archived_at) {
114+ await publishDocEvent(
115+ env.EVENTS,
116+ "doc.page.updated",
117+ {
118+ workspace: input.workspace_slug,
119+ workspaceId: page.workspace_id,
120+ pageId: page.id,
121+ spaceId: page.space_id,
122+ title: page.title,
123+ path: `/${input.workspace_slug}/-/docs/${page.slug}/${pageSlug(page.title, page.id)}`,
124+ versionId,
125+ kind,
126+ authors: [...new Set(input.version?.authors.length ? input.version.authors : input.pending_authors)],
127+ },
128+ last,
129+ );
130+ }
101131 if (fresh.length && env.NOTIFY && !page.archived_at) {
102132 const slug = input.workspace_slug;
103133 if (slug) {
+25−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+
4+import { isRepoDoc, pickRepoDocs, repoDocTitle } from "./repo-docs.ts";
5+
6+test("a project's docs are Markdown under docs/ and the README at the root", () => {
7+ assert.ok(isRepoDoc("README.md"));
8+ assert.ok(isRepoDoc("docs/guide/setup.md"));
9+ assert.ok(isRepoDoc("docs/a.mdx"));
10+ assert.ok(!isRepoDoc("src/README.md"));
11+ assert.ok(!isRepoDoc("docs/logo.png"));
12+ assert.ok(!isRepoDoc("documentation/a.md"));
13+ assert.deepEqual(
14+ pickRepoDocs([{ path: "docs/b.md" }, { path: "src/x.ts" }, { path: "docs/a.md" }, { path: "README.md" }]).map((f) => f.path),
15+ ["README.md", "docs/a.md", "docs/b.md"],
16+ );
17+});
18+
19+test("a file's title is its front matter's, its first heading, or its name", () => {
20+ assert.equal(repoDocTitle("docs/a.md", "---\ntitle: \"Setting up\"\n---\n# Other"), "Setting up");
21+ assert.equal(repoDocTitle("docs/a.md", "Intro\n\n# The **real** title\n"), "The real title");
22+ assert.equal(repoDocTitle("docs/getting-started.md", "no heading"), "Getting started");
23+ assert.equal(repoDocTitle("README.md", "text"), "README");
24+ assert.equal(repoDocTitle("docs/deploy/README.md", "text"), "deploy");
25+});
+36−0
1+/** Which of a repository's files are its docs in Docs, and their titles (src/repo-spaces.ts). Pure. */
2+
3+/** Files kept per repository: the first this many Markdown files, README first, then by path. */
4+export const MAX_REPO_FILES = 300;
5+
6+/** Whether a path is one of a project's docs: Markdown under `docs/`, or the README at the root. */
7+export function isRepoDoc(path: string): boolean {
8+ if (/^readme\.(md|markdown|mdx)$/i.test(path)) return true;
9+ return /^docs\/.+\.(md|markdown|mdx)$/i.test(path);
10+}
11+
12+/** A file's title: its first heading, the `title:` of its front matter, or its name. */
13+export function repoDocTitle(path: string, markdown: string): string {
14+ const front = /^---\r?\n([\s\S]*?)\r?\n---/.exec(markdown);
15+ const fromFront = front ? /^title:\s*["']?(.+?)["']?\s*$/m.exec(front[1]!)?.[1] : null;
16+ if (fromFront) return fromFront.slice(0, 200);
17+ const body = front ? markdown.slice(front[0].length) : markdown;
18+ const heading = /^\s{0,3}#\s+(.+?)\s*#*\s*$/m.exec(body)?.[1];
19+ if (heading) return heading.replace(/[*_`]/g, "").slice(0, 200);
20+ const name = path.split("/").pop()!.replace(/\.(md|markdown|mdx)$/i, "");
21+ if (/^readme$/i.test(name)) return path.includes("/") ? path.split("/").slice(-2, -1)[0]! : "README";
22+ return name.replace(/[-_]+/g, " ").replace(/^\w/, (c) => c.toUpperCase());
23+}
24+
25+/** The docs files in a listing, README first, then by path; at most MAX_REPO_FILES. */
26+export function pickRepoDocs<T extends { path: string }>(files: T[]): T[] {
27+ return files
28+ .filter((f) => isRepoDoc(f.path))
29+ .sort((a, b) => {
30+ const ra = a.path.toLowerCase().startsWith("readme.") ? 0 : 1;
31+ const rb = b.path.toLowerCase().startsWith("readme.") ? 0 : 1;
32+ return ra - rb || a.path.localeCompare(b.path);
33+ })
34+ .slice(0, MAX_REPO_FILES);
35+}
36+
+94−0
1+/**
2+ * A project's docs in Docs: a repository's `docs/` folder and its
3+ * README.md, read from the default branch into D1 so the sidebar lists
4+ * them and search finds them beside the workspace's pages. Read when the
5+ * folder is added, and again on every push to the default branch
6+ * (src/staleness.ts `onEvent`); only files whose blob changed are read
7+ * again. Read-only here: changes go through the repository.
8+ *
9+ * Who sees one is decided when it is read: the viewer must be able to read
10+ * the repository (`ReposApi.readable`), whoever added it.
11+ */
12+import { reposClient, type ServiceBinding } from "@g1t/contracts";
13+
14+import { searchText } from "./markdown.ts";
15+import { pickRepoDocs, repoDocTitle } from "./repo-docs.ts";
16+
17+export { isRepoDoc, pickRepoDocs, repoDocTitle } from "./repo-docs.ts";
18+
19+/** A file larger than this is listed but not read. */
20+const MAX_FILE_BYTES = 512 * 1024;
21+
22+export type RepoSpaceRow = {
23+ id: string;
24+ workspace_id: string;
25+ repo_id: string;
26+ repo: string;
27+ default_branch: string;
28+ commit_sha: string | null;
29+ indexed_at: string | null;
30+ added_by: string;
31+ added_at: string;
32+};
33+
34+function decodeBase64(data: string): string {
35+ const binary = atob(data);
36+ const bytes = new Uint8Array(binary.length);
37+ for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
38+ return new TextDecoder().decode(bytes);
39+}
40+
41+/**
42+ * Reads a repository's docs again into one space: lists the default
43+ * branch, reads the files whose blob changed, drops the ones gone. Asks
44+ * repos without a viewer (`listFiles`, `rawBlobs`), so callers check the
45+ * repository can be read first, as adding one does.
46+ */
47+export async function indexRepoSpace(env: { DB: D1Database; REPOS: ServiceBinding }, space: RepoSpaceRow, now = new Date()): Promise<{ files: number; read: number }> {
48+ const db = env.DB;
49+ const repos = reposClient(env.REPOS);
50+ const listing = await repos.listFiles(space.repo_id, null, 10_000);
51+ const wanted = pickRepoDocs(listing.files.filter((f): f is { path: string; hash: string } => !!f.hash));
52+ const kept = new Map(
53+ (await db.prepare("SELECT path, hash FROM repo_files WHERE space_id = ?").bind(space.id).all<{ path: string; hash: string }>()).results.map((r) => [r.path, r.hash]),
54+ );
55+ const changed = wanted.filter((f) => kept.get(f.path) !== f.hash);
56+ const blobs = new Map<string, string | null>();
57+ for (let i = 0; i < changed.length; i += 100) {
58+ const batch = changed.slice(i, i + 100);
59+ for (const blob of await repos.rawBlobs(space.repo_id, [...new Set(batch.map((f) => f.hash))], MAX_FILE_BYTES)) blobs.set(blob.hash, blob.data);
60+ }
61+ const statements: D1PreparedStatement[] = [];
62+ const gone = [...kept.keys()].filter((path) => !wanted.some((f) => f.path === path));
63+ for (const path of gone) {
64+ statements.push(db.prepare("DELETE FROM repo_files WHERE space_id = ? AND path = ?").bind(space.id, path));
65+ statements.push(db.prepare("DELETE FROM repo_files_fts WHERE space_id = ? AND path = ?").bind(space.id, path));
66+ }
67+ for (const file of changed) {
68+ const data = blobs.get(file.hash);
69+ const markdown = data ? decodeBase64(data) : `_This file is too large to show here._ Open it in Code.\n`;
70+ const title = repoDocTitle(file.path, markdown);
71+ statements.push(
72+ db
73+ .prepare("INSERT INTO repo_files (space_id, path, hash, title, markdown) VALUES (?, ?, ?, ?, ?) ON CONFLICT (space_id, path) DO UPDATE SET hash = excluded.hash, title = excluded.title, markdown = excluded.markdown")
74+ .bind(space.id, file.path, file.hash, title, markdown),
75+ );
76+ statements.push(db.prepare("DELETE FROM repo_files_fts WHERE space_id = ? AND path = ?").bind(space.id, file.path));
77+ statements.push(db.prepare("INSERT INTO repo_files_fts (space_id, path, title, body) VALUES (?, ?, ?, ?)").bind(space.id, file.path, title, searchText(markdown)));
78+ }
79+ statements.push(db.prepare("UPDATE repo_spaces SET commit_sha = ?, indexed_at = ? WHERE id = ?").bind(listing.commit, now.toISOString(), space.id));
80+ for (let i = 0; i < statements.length; i += 50) await db.batch(statements.slice(i, i + 50));
81+ return { files: wanted.length, read: changed.length };
82+}
83+
84+/** Every space showing a repository's docs, read again after a push. Never throws for one space's sake. */
85+export async function reindexRepo(env: { DB: D1Database; REPOS: ServiceBinding }, repoId: string): Promise<void> {
86+ const spaces = (await env.DB.prepare("SELECT * FROM repo_spaces WHERE repo_id = ?").bind(repoId).all<RepoSpaceRow>()).results;
87+ for (const space of spaces) {
88+ try {
89+ await indexRepoSpace(env, space);
90+ } catch (error) {
91+ console.error("docs could not read a project's docs", space.repo, String(error));
92+ }
93+ }
94+}
+6−3
3030 import { atLeast } from "./access.ts";
3131 import { seed } from "./blocks.ts";
3232 import { anchorThread, applyEdit, findTarget, rangeIds, rangeMarkdown, restoreFrom, unanchorThread } from "./edits.ts";
33−import { documentMarkdown, mentionedIds, outline, type Outline } from "./markdown.ts";
33+import { bodyCitations } from "./citations.ts";
34+import { citationNodes, documentMarkdown, mentionedIds, outline, type Outline } from "./markdown.ts";
3435 import { save } from "./persist.ts";
3536 import { applyThreadAction, listThreads, setQuote, type ThreadResult } from "./threads.ts";
3637
5152
5253 type Attachment = RoomMember & { clients: number[] };
5354
54−type Env = { DB: D1Database; NOTIFY?: ServiceBinding };
55+type Env = { DB: D1Database; NOTIFY?: ServiceBinding; EVENTS?: ServiceBinding };
5556
5657 const FRAGMENT = "document-store";
5758 const MESSAGE_SYNC = 0;
172173 const fragment = this.fragment();
173174 const editors = this.meta<string[]>("editors", []);
174175 const pending = this.meta<string[]>("pending_authors", []);
176+ const markdown = documentMarkdown(fragment);
175177 const result = await save(this.env, {
176178 page_id: pageId,
177− markdown: documentMarkdown(fragment),
179+ markdown,
180+ citations: bodyCitations(citationNodes(fragment), markdown),
178181 editors,
179182 mentioned: mentionedIds(fragment).users,
180183 editor_names: this.meta<string[]>("editor_names", []),
+63−0
1+import assert from "node:assert/strict";
2+import { test } from "node:test";
3+
4+import { amzDate, sha256Hex, sign, uriEncode } from "./sigv4.ts";
5+
6+const EMPTY = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
7+
8+// AWS's SigV4 test suite, `get-vanilla`: the plainest request there is.
9+test("signs AWS's get-vanilla test vector", async () => {
10+ const signed = await sign({
11+ method: "GET",
12+ url: "https://example.amazonaws.com/",
13+ payload_hash: EMPTY,
14+ region: "us-east-1",
15+ service: "service",
16+ credentials: { access_key_id: "AKIDEXAMPLE", secret_access_key: "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY" },
17+ date: new Date("2015-08-30T12:36:00Z"),
18+ });
19+ assert.equal(signed.canonical_request, `GET\n/\n\nhost:example.amazonaws.com\nx-amz-date:20150830T123600Z\n\nhost;x-amz-date\n${EMPTY}`);
20+ assert.equal(signed.string_to_sign, "AWS4-HMAC-SHA256\n20150830T123600Z\n20150830/us-east-1/service/aws4_request\nbb579772317eb040ac9ed261061d46c1f17a8133879d6129b6e1c25292927e63");
21+ assert.equal(signed.signature, "5fa00fa31553b73ebf1942676e86291e8372ff2a2260956d9b8aae1d763fbf31");
22+ assert.equal(
23+ signed.headers.authorization,
24+ "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/service/aws4_request, SignedHeaders=host;x-amz-date, Signature=5fa00fa31553b73ebf1942676e86291e8372ff2a2260956d9b8aae1d763fbf31",
25+ );
26+});
27+
28+// The S3 documentation's worked example: GET Object with a Range header.
29+test("signs the S3 GET Object example", async () => {
30+ const signed = await sign({
31+ method: "GET",
32+ url: "https://examplebucket.s3.amazonaws.com/test.txt",
33+ headers: { range: "bytes=0-9", "x-amz-content-sha256": EMPTY },
34+ payload_hash: EMPTY,
35+ region: "us-east-1",
36+ service: "s3",
37+ credentials: { access_key_id: "AKIAIOSFODNN7EXAMPLE", secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" },
38+ date: new Date("2013-05-24T00:00:00Z"),
39+ });
40+ assert.equal(signed.signature, "f0e8bdb87c964420e857bd35b5d6ed310bd44f0170aba48dd91039c6036bdb41");
41+ assert.match(signed.headers.authorization!, /SignedHeaders=host;range;x-amz-content-sha256;x-amz-date,/);
42+});
43+
44+test("hashes and encodes as SigV4 wants", async () => {
45+ assert.equal(await sha256Hex(""), EMPTY);
46+ assert.equal(uriEncode("a b/c*~"), "a%20b%2Fc%2A~");
47+ assert.equal(amzDate(new Date("2015-08-30T12:36:00.123Z")), "20150830T123600Z");
48+});
49+
50+test("puts the query in order", async () => {
51+ const signed = await sign({
52+ method: "GET",
53+ url: "https://example.amazonaws.com/?Param2=value2&Param1=value1",
54+ payload_hash: EMPTY,
55+ region: "us-east-1",
56+ service: "service",
57+ credentials: { access_key_id: "AKIDEXAMPLE", secret_access_key: "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY" },
58+ date: new Date("2015-08-30T12:36:00Z"),
59+ });
60+ assert.equal(signed.canonical_request.split("\n")[2], "Param1=value1&Param2=value2");
61+ // AWS's `get-vanilla-query-order-key-case`.
62+ assert.equal(signed.signature, "b97d918cfa904a5beff61c982a1b6f458b799221646efd99d3219ec94cdf2500");
63+});
+124−0
1+/**
2+ * AWS Signature Version 4, by hand with WebCrypto: how a self-hosted g1t
3+ * signs requests to an S3-compatible store (MinIO, Ceph, Garage, AWS S3,
4+ * R2's S3 API) without an SDK. Checked against AWS's published test
5+ * vectors (sigv4.test.ts). Reference:
6+ * https://docs.aws.amazon.com/IAM/latest/UserGuide/create-signed-request.html
7+ */
8+
9+export type Credentials = { access_key_id: string; secret_access_key: string };
10+
11+export type SignInput = {
12+ method: string;
13+ url: string;
14+ /** Headers to sign besides `host` and `x-amz-date` (which are added). */
15+ headers?: Record<string, string>;
16+ /** The payload's SHA-256 in hex, or `UNSIGNED-PAYLOAD`. */
17+ payload_hash: string;
18+ region: string;
19+ service: string;
20+ credentials: Credentials;
21+ /** When it is signed; now by default. */
22+ date?: Date;
23+ /**
24+ * Whether path segments are encoded a second time, as every AWS service
25+ * but S3 expects. S3 signs the path as sent.
26+ */
27+ double_encode_path?: boolean;
28+};
29+
30+export type Signed = {
31+ /** Every header to send, `authorization` included (lowercase names). */
32+ headers: Record<string, string>;
33+ canonical_request: string;
34+ string_to_sign: string;
35+ signature: string;
36+};
37+
38+const encoder = new TextEncoder();
39+
40+function hex(bytes: ArrayBuffer): string {
41+ return [...new Uint8Array(bytes)].map((b) => b.toString(16).padStart(2, "0")).join("");
42+}
43+
44+export async function sha256Hex(data: string | ArrayBuffer | Uint8Array): Promise<string> {
45+ const bytes = typeof data === "string" ? encoder.encode(data) : data;
46+ return hex(await crypto.subtle.digest("SHA-256", bytes as BufferSource));
47+}
48+
49+async function hmac(key: ArrayBuffer | Uint8Array, data: string): Promise<ArrayBuffer> {
50+ const k = await crypto.subtle.importKey("raw", key as BufferSource, { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
51+ return crypto.subtle.sign("HMAC", k, encoder.encode(data));
52+}
53+
54+/** RFC 3986 encoding, as SigV4 wants it: unreserved characters stay, everything else is %XX in upper case. */
55+export function uriEncode(value: string): string {
56+ return encodeURIComponent(value).replace(/[!'()*]/g, (c) => `%${c.charCodeAt(0).toString(16).toUpperCase()}`);
57+}
58+
59+/** `20150830T123600Z`. */
60+export function amzDate(date: Date): string {
61+ return date.toISOString().replace(/[-:]/g, "").replace(/\.\d{3}/, "");
62+}
63+
64+function canonicalPath(pathname: string, double: boolean): string {
65+ if (!pathname || pathname === "/") return "/";
66+ return pathname
67+ .split("/")
68+ .map((segment) => {
69+ let raw: string;
70+ try {
71+ raw = decodeURIComponent(segment);
72+ } catch {
73+ raw = segment;
74+ }
75+ const once = uriEncode(raw);
76+ return double ? uriEncode(once) : once;
77+ })
78+ .join("/");
79+}
80+
81+function canonicalQuery(search: URLSearchParams): string {
82+ return [...search.entries()]
83+ .map(([k, v]) => [uriEncode(k), uriEncode(v)] as const)
84+ .sort(([ak, av], [bk, bv]) => (ak < bk ? -1 : ak > bk ? 1 : av < bv ? -1 : av > bv ? 1 : 0))
85+ .map(([k, v]) => `${k}=${v}`)
86+ .join("&");
87+}
88+
89+/** Signs a request: the headers to send with it, and what was signed (for tests and debugging). */
90+export async function sign(input: SignInput): Promise<Signed> {
91+ const url = new URL(input.url);
92+ const when = amzDate(input.date ?? new Date());
93+ const day = when.slice(0, 8);
94+ const headers: Record<string, string> = {};
95+ for (const [k, v] of Object.entries(input.headers ?? {})) headers[k.toLowerCase()] = String(v).trim().replace(/\s+/g, " ");
96+ headers.host = url.host;
97+ headers["x-amz-date"] = when;
98+ const names = Object.keys(headers).sort();
99+ const signedHeaders = names.join(";");
100+ const canonicalRequest = [
101+ input.method.toUpperCase(),
102+ canonicalPath(url.pathname, input.double_encode_path ?? input.service !== "s3"),
103+ canonicalQuery(url.searchParams),
104+ names.map((n) => `${n}:${headers[n]}\n`).join(""),
105+ signedHeaders,
106+ input.payload_hash,
107+ ].join("\n");
108+ const scope = `${day}/${input.region}/${input.service}/aws4_request`;
109+ const stringToSign = ["AWS4-HMAC-SHA256", when, scope, await sha256Hex(canonicalRequest)].join("\n");
110+ const kDate = await hmac(encoder.encode(`AWS4${input.credentials.secret_access_key}`), day);
111+ const kRegion = await hmac(kDate, input.region);
112+ const kService = await hmac(kRegion, input.service);
113+ const kSigning = await hmac(kService, "aws4_request");
114+ const signature = hex(await hmac(kSigning, stringToSign));
115+ return {
116+ headers: {
117+ ...headers,
118+ authorization: `AWS4-HMAC-SHA256 Credential=${input.credentials.access_key_id}/${scope}, SignedHeaders=${signedHeaders}, Signature=${signature}`,
119+ },
120+ canonical_request: canonicalRequest,
121+ string_to_sign: stringToSign,
122+ signature,
123+ };
124+}
+1−1
3535 }
3636
3737 /** Space slugs the site's routes use under `-/docs/`. */
38−export const RESERVED_SPACE_SLUGS = new Set(["new", "search", "trash", "templates", "live", "api", "threads", "upload", "export", "settings", "recent", "favorites"]);
38+export const RESERVED_SPACE_SLUGS = new Set(["new", "search", "trash", "templates", "live", "api", "threads", "upload", "export", "settings", "recent", "favorites", "stale", "repo"]);
3939
4040 /** A space slug that is free: `base`, else `base-2`, `base-3`, ... */
4141 export function freeSlug(base: string, taken: ReadonlySet<string>): string {
+294−0
1+/**
2+ * Pages that cite code a change touched become possibly out of date
3+ * (docs/WORKSPACE.md, "Agents and docs"). The events service sends this
4+ * service `git.push` and `pull.merged` (`SUBSCRIBER_DOCS`,
5+ * crates/contracts subscribers.rs), and the lifecycle events every
6+ * subscriber hears.
7+ *
8+ * - **What changed** is asked as g1t itself (a system viewer in the
9+ * repository's workspace): a merged pull request's files from the work
10+ * service, or a push's from comparing it with where the branch was.
11+ * Only pushes to the default branch count.
12+ * - **Cheap when nothing cites the repository:** the citations table is
13+ * asked first, and nothing else is.
14+ * - **Once per page and commit:** a merge is told twice (the push and the
15+ * pull request); both land on one row, which the pull request names.
16+ * Owners hear of it once, and `doc.page.stale` is published once.
17+ * - **Privacy:** what is stored is the change itself; what a reader sees
18+ * of it (the page, its banner, the inbox) is filtered by whether they can
19+ * read the repository.
20+ */
21+import {
22+ currentMovedPath,
23+ identityClient,
24+ notifyClient,
25+ repoMove,
26+ reposClient,
27+ staleMovedPaths,
28+ workClient,
29+ type G1tEvent,
30+ type ServiceBinding,
31+ type User,
32+} from "@g1t/contracts";
33+
34+import { touchedPaths } from "./citations.ts";
35+import type { PageRoom } from "./room.ts";
36+import { publishDocEvent } from "./events.ts";
37+import { pageSlug } from "./slugs.ts";
38+
39+export type StaleEnv = {
40+ DB: D1Database;
41+ IDENTITY: ServiceBinding;
42+ REPOS?: ServiceBinding;
43+ WORK?: ServiceBinding;
44+ NOTIFY?: ServiceBinding;
45+ EVENTS?: ServiceBinding;
46+ PAGES?: DurableObjectNamespace<PageRoom>;
47+};
48+
49+/** What a change touched, as this module records it. */
50+export type Change = {
51+ repo: string;
52+ repo_id: string;
53+ commit: string;
54+ pull: { number: number; title: string | null } | null;
55+ /** Every path the change touched. */
56+ changed: string[];
57+ /** Who made it (an account id), for the events it causes. */
58+ actor: string | null;
59+};
60+
61+/** g1t itself, reading in a repository's workspace: how staleness asks what changed. */
62+export function systemReader(namespace: string): User {
63+ return { id: "g1t", username: "g1t", kind: "system", verified: true, workspaces: [{ slug: namespace.toLowerCase(), role: "owner" }] };
64+}
65+
66+const ZERO = /^0+$/;
67+
68+async function pathById(repos: ServiceBinding, id: string): Promise<{ namespace: string; name: string } | null> {
69+ const response = await repos.fetch("https://service/rpc/path_by_id", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ id }) });
70+ if (!response.ok) return null;
71+ return (await response.json()) as { namespace: string; name: string } | null;
72+}
73+
74+/** Whether any live page cites the repository, or any workspace shows its docs. */
75+async function interested(db: D1Database, repo: string, repoId: string): Promise<{ cited: boolean; spaces: boolean }> {
76+ const [cited, spaces] = await db.batch<{ yes: number }>([
77+ db.prepare("SELECT 1 AS yes FROM citations c JOIN pages p ON p.id = c.page_id WHERE c.repo = ? AND p.archived_at IS NULL LIMIT 1").bind(repo),
78+ db.prepare("SELECT 1 AS yes FROM repo_spaces WHERE repo_id = ? LIMIT 1").bind(repoId),
79+ ]);
80+ return { cited: !!cited?.results.length, spaces: !!spaces?.results.length };
81+}
82+
83+/** What a push to the default branch changed: the files between where it was and where it is. */
84+async function pushChange(env: StaleEnv, path: { namespace: string; name: string }, repoId: string, before: string | undefined, after: string): Promise<string[] | null> {
85+ if (!env.REPOS || !before || ZERO.test(before)) return null;
86+ const compared = await reposClient(env.REPOS).compare(repoId, systemReader(path.namespace), before, after);
87+ if (!compared.ok) return null;
88+ return compared.value.files.map((f) => f.path);
89+}
90+
91+/** What a merged pull request changed, and its title; null when it didn't merge into the default branch. */
92+async function pullChange(env: StaleEnv, path: { namespace: string; name: string }, repoId: string, number: number): Promise<{ title: string; changed: string[] } | null> {
93+ if (!env.WORK || !env.REPOS) return null;
94+ const reader = systemReader(path.namespace);
95+ const [found, repo] = await Promise.all([workClient(env.WORK).getPull(path, number, reader), reposClient(env.REPOS).getById(repoId, reader)]);
96+ if (!found.ok || !repo.ok) return null;
97+ const pull = found.value.pull;
98+ if (pull.base && pull.base !== repo.value.defaultBranch) return null;
99+ let changed = (pull.files ?? []).map((f) => f.path);
100+ if (!changed.length && pull.mergeBase && pull.headCommit) {
101+ const compared = await reposClient(env.REPOS).compare(repoId, reader, pull.mergeBase, pull.headCommit);
102+ if (compared.ok) changed = compared.value.files.map((f) => f.path);
103+ }
104+ return { title: pull.title, changed };
105+}
106+
107+type CitedRow = { page_id: string; path: string };
108+type PageRow = { id: string; workspace_id: string; space_id: string; title: string; space_slug: string };
109+
110+/**
111+ * Records a change against every page whose citations it touches.
112+ * Returns the pages newly made stale by it.
113+ */
114+export async function record(env: StaleEnv, change: Change, now = new Date()): Promise<string[]> {
115+ const db = env.DB;
116+ const cited = (
117+ await db
118+ .prepare("SELECT c.page_id, c.path FROM citations c JOIN pages p ON p.id = c.page_id WHERE c.repo = ? AND p.archived_at IS NULL")
119+ .bind(change.repo)
120+ .all<CitedRow>()
121+ ).results;
122+ const byPage = new Map<string, CitedRow[]>();
123+ for (const row of cited) byPage.set(row.page_id, [...(byPage.get(row.page_id) ?? []), row]);
124+ const fresh: string[] = [];
125+ const hits = new Map<string, string[]>();
126+ for (const [pageId, rows] of byPage) {
127+ const touched = touchedPaths(rows, change.changed);
128+ if (!touched.length) continue;
129+ hits.set(pageId, touched);
130+ const at = now.toISOString();
131+ const [inserted] = await db.batch<{ page_id: string }>([
132+ db
133+ .prepare(
134+ "INSERT INTO page_changes (page_id, repo, repo_id, commit_sha, pull_number, pull_title, paths, detected_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?) ON CONFLICT (page_id, repo, commit_sha) DO NOTHING RETURNING page_id",
135+ )
136+ .bind(pageId, change.repo, change.repo_id, change.commit, change.pull?.number ?? null, change.pull?.title ?? null, JSON.stringify(touched), at),
137+ ...(change.pull
138+ ? [
139+ db
140+ .prepare("UPDATE page_changes SET pull_number = ?, pull_title = ? WHERE page_id = ? AND repo = ? AND commit_sha = ?")
141+ .bind(change.pull.number, change.pull.title, pageId, change.repo, change.commit),
142+ ]
143+ : []),
144+ ]);
145+ if (inserted?.results.length) fresh.push(pageId);
146+ }
147+ if (fresh.length) await tellOf(env, change, fresh, hits);
148+ return fresh;
149+}
150+
151+/** Owners hear of pages newly stale, rooms ask their readers to look again, and `doc.page.stale` goes out. */
152+async function tellOf(env: StaleEnv, change: Change, pageIds: string[], hits: Map<string, string[]>): Promise<void> {
153+ const db = env.DB;
154+ const marks = pageIds.map(() => "?").join(",");
155+ const [pages, owners] = await Promise.all([
156+ db
157+ .prepare(`SELECT p.id, p.workspace_id, p.space_id, p.title, s.slug AS space_slug FROM pages p JOIN spaces s ON s.id = p.space_id WHERE p.id IN (${marks})`)
158+ .bind(...pageIds)
159+ .all<PageRow>(),
160+ db.prepare(`SELECT page_id, principal FROM page_owners WHERE page_id IN (${marks})`).bind(...pageIds).all<{ page_id: string; principal: string }>(),
161+ ]);
162+ const workspaceIds = [...new Set(pages.results.map((p) => p.workspace_id))];
163+ const slugs = await identityClient(env.IDENTITY)
164+ .usernames(workspaceIds)
165+ .catch(() => ({}) as Record<string, string>);
166+ // Which owners can read the repository: they are told which change it was.
167+ const ownerIds = [...new Set(owners.results.filter((o) => o.principal.startsWith("user:")).map((o) => o.principal.slice(5)))];
168+ const people = ownerIds.length ? await identityClient(env.IDENTITY).usersForAudience(ownerIds).catch(() => [] as User[]) : [];
169+ const [namespace, name] = change.repo.split("/") as [string, string];
170+ const canRead = new Set<string>();
171+ if (env.REPOS) {
172+ await Promise.all(
173+ people.map(async (person) => {
174+ const found = await reposClient(env.REPOS!)
175+ .get({ namespace, name }, person)
176+ .catch(() => null);
177+ if (found?.ok) canRead.add(person.id);
178+ }),
179+ );
180+ }
181+ const what = change.pull ? `${change.repo}#${change.pull.number}` : `${change.repo}@${change.commit.slice(0, 7)}`;
182+ const notify = env.NOTIFY ? notifyClient(env.NOTIFY) : null;
183+ for (const page of pages.results) {
184+ const slug = slugs[page.workspace_id];
185+ if (!slug) continue;
186+ const href = `/${slug}/-/docs/${page.space_slug}/${pageSlug(page.title, page.id)}`;
187+ const paths = hits.get(page.id) ?? [];
188+ const keys = owners.results.filter((o) => o.page_id === page.id).map((o) => o.principal);
189+ const work: Promise<unknown>[] = [];
190+ if (notify) {
191+ for (const key of keys.filter((k) => k.startsWith("user:"))) {
192+ const id = key.slice(5);
193+ const known = canRead.has(id);
194+ work.push(
195+ notify
196+ .notify(
197+ { user_id: id },
198+ {
199+ id: `doc-stale:${page.id}:${change.commit}:${id}`,
200+ kind: "inbox",
201+ workspace: slug,
202+ title: `${page.title || "Untitled"} may be out of date`,
203+ body: known ? `${what} changed ${paths.slice(0, 3).join(", ")}${paths.length > 3 ? ` and ${paths.length - 3} more` : ""}` : "A change to code this page cites was merged.",
204+ href,
205+ actor: { kind: "system", id: "g1t", name: "g1t", avatar: null, avatar_seed: null },
206+ created_at: new Date().toISOString(),
207+ },
208+ )
209+ .catch(() => undefined),
210+ );
211+ }
212+ }
213+ if (env.PAGES) {
214+ work.push(
215+ env.PAGES.get(env.PAGES.idFromName(page.id))
216+ .notice({ type: "page.staleness" })
217+ .catch(() => undefined),
218+ );
219+ }
220+ work.push(
221+ publishDocEvent(
222+ env.EVENTS,
223+ "doc.page.stale",
224+ {
225+ workspace: slug,
226+ workspaceId: page.workspace_id,
227+ pageId: page.id,
228+ spaceId: page.space_id,
229+ title: page.title,
230+ path: href,
231+ repoId: change.repo_id,
232+ repo: change.repo,
233+ commit: change.commit,
234+ pull: change.pull?.number ?? null,
235+ paths,
236+ owners: keys,
237+ },
238+ change.actor ? `user:${change.actor}` : null,
239+ ),
240+ );
241+ await Promise.all(work);
242+ }
243+}
244+
245+/** A repository moved: rows kept under its old path follow it. */
246+async function followMove(env: StaleEnv, event: G1tEvent): Promise<void> {
247+ const move = repoMove(event);
248+ if (!move || !env.REPOS) return;
249+ const current = (await currentMovedPath(env.REPOS, move)).toLowerCase();
250+ const stale = staleMovedPaths(move, current).map((p) => p.toLowerCase());
251+ if (!stale.length) return;
252+ const db = env.DB;
253+ const statements: D1PreparedStatement[] = [];
254+ for (const old of stale) {
255+ statements.push(
256+ db.prepare("UPDATE OR IGNORE citations SET repo = ? WHERE repo = ?").bind(current, old),
257+ db.prepare("UPDATE OR IGNORE page_changes SET repo = ? WHERE repo = ?").bind(current, old),
258+ db.prepare("UPDATE OR IGNORE page_projects SET repo = ? WHERE repo = ?").bind(current, old),
259+ db.prepare("UPDATE OR IGNORE space_projects SET repo = ? WHERE repo = ?").bind(current, old),
260+ db.prepare("UPDATE repo_spaces SET repo = ? WHERE repo_id = ?").bind(current, move.repoId),
261+ );
262+ }
263+ await db.batch(statements);
264+}
265+
266+/** One event, as this service acts on it. `reindex` reads a project's docs again (src/repo-spaces.ts). */
267+export async function onEvent(env: StaleEnv, event: G1tEvent, reindex: (repoId: string, commit: string) => Promise<void>): Promise<void> {
268+ if (event.type === "repo.renamed" || event.type === "repo.transferred") return followMove(env, event);
269+ if (event.type === "repo.purged") {
270+ await env.DB.prepare("DELETE FROM repo_spaces WHERE repo_id = ?").bind(event.data.repoId).run();
271+ return;
272+ }
273+ if (event.type !== "git.push" && event.type !== "pull.merged") return;
274+ if (!env.REPOS) return;
275+ const repoId = event.repoId ?? (event.data as { repoId?: string }).repoId ?? null;
276+ if (!repoId) return;
277+ if (event.type === "git.push" && !event.data.defaultBranch) return;
278+ const path = await pathById(env.REPOS, repoId);
279+ if (!path) return;
280+ const repo = `${path.namespace}/${path.name}`.toLowerCase();
281+ const wants = await interested(env.DB, repo, repoId);
282+ if (event.type === "git.push") {
283+ if (wants.spaces) await reindex(repoId, event.data.after);
284+ if (!wants.cited) return;
285+ const changed = await pushChange(env, path, repoId, event.data.before, event.data.after);
286+ if (!changed?.length) return;
287+ await record(env, { repo, repo_id: repoId, commit: event.data.after, pull: null, changed, actor: event.actor });
288+ return;
289+ }
290+ if (!wants.cited) return;
291+ const pulled = await pullChange(env, path, repoId, event.data.number);
292+ if (!pulled?.changed.length) return;
293+ await record(env, { repo, repo_id: repoId, commit: event.data.commit, pull: { number: event.data.number, title: pulled.title }, changed: pulled.changed, actor: event.actor });
294+}
+17−1
2323 // Files people put in pages (images, attachments), served from the
2424 // usercontent origin at /docs-files/<key> (apps/web/workers/usercontent.ts).
2525 // Create it once: npx wrangler r2 bucket create g1t-docs-files
26+ // Self-hosted, any S3-compatible store instead: DOCS_FILES=s3 and the
27+ // DOCS_S3_* settings (src/files.ts, docs/SELF_HOSTING.md).
2628 "r2_buckets": [{ "binding": "FILES", "bucket_name": "g1t-docs-files" }],
2729 "services": [
2830 // Workspaces by slug, people's names and avatars, and teams.
3032 // The workspace's agents: who they are, for agents' calls and mentions.
3133 { "binding": "AGENTS", "service": "g1t-agents" },
3234 // Mentions and suggestions reach people's inboxes (services/notify).
33− { "binding": "NOTIFY", "service": "g1t-notify" }
35+ { "binding": "NOTIFY", "service": "g1t-notify" },
36+ // Who may read a repository, what a change touched, and a project's
37+ // docs folder (src/staleness.ts, src/repo-spaces.ts).
38+ { "binding": "REPOS", "service": "g1t-repos" },
39+ // What a merged pull request changed.
40+ { "binding": "WORK", "service": "g1t-work" },
41+ // doc.page.* events on the bus (src/events.ts).
42+ { "binding": "EVENTS", "service": "g1t-events" }
3443 ],
44+ // Merges and pushes to default branches: pages whose cited code
45+ // changed, and projects' docs read again (src/staleness.ts). The events
46+ // service sends it what g1t_contracts::subscribers routes to
47+ // SUBSCRIBER_DOCS. Create it once: npx wrangler queues create g1t-events-docs
48+ "queues": {
49+ "consumers": [{ "queue": "g1t-events-docs", "max_batch_size": 20, "max_batch_timeout": 2, "max_retries": 3, "dead_letter_queue": "g1t-events-dlq" }]
50+ },
3551 // One room per page: it holds the page's Yjs document and open sockets,
3652 // hibernating while nobody types (src/room.ts).
3753 "durable_objects": {
+4−0
3636 { "binding": "SUBSCRIBER_CONTEXT", "queue": "g1t-events-context" },
3737 { "binding": "SUBSCRIBER_SEARCH", "queue": "g1t-events-search" },
3838 { "binding": "SUBSCRIBER_PACKAGES", "queue": "g1t-events-packages" },
39+ // Pages that cite code a merge changed, and projects' docs folders
40+ // (services/docs/src/staleness.ts). Create it once:
41+ // npx wrangler queues create g1t-events-docs
42+ { "binding": "SUBSCRIBER_DOCS", "queue": "g1t-events-docs" },
3943 // Agents' routines that run on events (services/agents/src/triggers.ts).
4044 { "binding": "SUBSCRIBER_AGENTS", "queue": "g1t-events-agents" }
4145 ],