Personal-data erasure has an intake, a staff case with its trail, and cleanup that identity commits with the purge and retries until each service confirms it; one-time-code fields work with password managers and take recovery codes; the Ask panel closes when you follow a link from it. Settings → Account records an erasure request (identity request_erasure / my_erasure, migration 0049), even when a workspace you own alone or billing stands in the way of closing the account; it deletes nothing, keeps the first deadline when asked again, and the deadline is the same day next calendar month, clamped to its last day. sudo → Customers → Privacy lists the cases (the sidebar counts open ones), records a request received elsewhere with its original receipt time (an intake for an open case joins its trail and can only move the deadline earlier), shows each case's trail of intake, reviews, refusal or completion, and refuses completion while the account or any of its cleanup jobs remain or any of the seven evidence areas is empty. Purging an account or a workspace now writes five purge jobs (avatar, Chat, Notify, Artifacts, the lifecycle event) in the same D1 batch as the deletion, guarded like every statement in it, so a restore that won leaves neither; the 15-minute tick runs them, keeps a failure pending with backoff to 16 hours, never stores a response body, and clears a done job's payload. Publishing user.deleted or workspace.deleted is delivery to the bus, not proof its consumers finished. A workspace's jobs carry its members at purge time and its former slugs no live workspace holds. Chat (migration 0006): the person's direct message with g1t goes with its answers and pending job; in shared conversations only the person is shut out and their messages show as ghost, where before the room's clear told every member's tab the conversation was emptied and dropped others' pending answers; a workspace's channels are deleted 500 messages at a time within a budget, answering pending until done; the channels to visit are listed once with the tombstone; tombstones, looked up by key with new indexes, refuse late messages, members, reactions, channels, settings, emoji and meter rows. Notify: an erased feed drops later deliveries quietly instead of throwing; a purged workspace's slug can be made again by the person whose username it is, so its marks are times, not bans: notifications from before the purge are dropped, counts, seeds and visits refused for a day, and its presence room refuses only words said before the purge; the purge reaches identity's members and the room's people under every slug, and clears complete:, forgot: and the workspace's notification level. Artifacts (migration 0010): unshared private folios and their files, versions, search rows and vectors go; authorship (now also cleared_by and repo_spaces.added_by) shows as ghost and mention lists drop the username; a workspace is swept a page at a time and resumes after a failure; full-text rows of a document are deleted by document; an index run that races an erasure writes no full-text row without a chunk row and takes back the vectors it wrote; a room's destroy waits for its index run; ordinary page deletion now destroys the page's live document. Not covered automatically, and listed in docs/operations/privacy-erasure.md as manual review: comment threads inside shared documents' live state and version snapshots, private-space pages, rooms of pages deleted before this, folio trash past 200 rooms, earlier folios' version and upload blobs, former members' and outsiders' notifications, backups. One-time codes: the sign-in's second step, two-factor settings and email confirmation share otpInputProps (lib/otp.ts): a stable id and label, autocomplete one-time-code, no password-manager ignore attributes; the sign-in field takes either kind of code at recovery-code length whichever is chosen, and Use a recovery code switches the keyboard. The Ask panel closes on every completed navigation (location.key), including its direct-message link and citations to the page you are on and back or forward; links opened in another tab and revalidation leave it open. The Privacy Policy distinguishes closure from erasure; the authentication guide says how to ask and how recovery codes are entered; the services page lists identity's new CHAT, NOTIFY and ARTIFACTS bindings. Deploy migrations first, then chat, notify and artifacts before identity, then web and sudo. Checked: cargo test -p g1t-identity --lib (175) and its wasm32 check; node --test conformance/erasure.test.mjs (11, new: a shared channel's room shuts out only the erased member, a large channel deleted in chunks with late writes refused, a reclaimed slug keeps new notifications and presence, vectors taken back after a racing erasure, an upload sweep resuming after a failure, late deliveries dropped quietly); artifacts' 126 tests and typecheck, notify's 49 and typecheck, chat's typecheck; the site's tsc -b --force, 926 tests (new: otp) and a production build; sudo's typecheck, 156 tests and build; contracts' 48; the deploy tool's tests (the one failure is services/runner/base.json, from separate uncommitted runner work); the docs build.
46 files+1802−810/46 viewed
| 21 | 21 | | `apps/status` | TypeScript | edge | `status.g1t.sh`: probes every minute and keeps incidents | D1 `g1t-status` | `billing`, `repos` | | |
| 22 | 22 | | `apps/sudo` | TypeScript | front | g1t.sh's staff console, behind Cloudflare Access | Nothing | `actions`, `billing`, `events`, `identity`, `projects`, `status`, `telemetry` | | |
| 23 | 23 | | `apps/docs` | Astro | front | `docs.g1t.sh`, these pages | Nothing | none | | |
| 24 | − | | `services/identity` | Rust | core | Accounts, sessions, workspaces, members, teams, repository roles, tokens, OAuth, device sign-in, 2FA | D1 `g1t`, the avatars KV | `billing`, `events`, `projects`, `repos` | | |
| 24 | + | | `services/identity` | Rust | core | Accounts, sessions, workspaces, members, teams, repository roles, tokens, OAuth, device sign-in, 2FA | D1 `g1t`, the avatars KV | `artifacts`, `billing`, `chat`, `events`, `notify`, `projects`, `repos` | | |
| 25 | 25 | | `services/repos` | Rust | core | Repositories and git: smart HTTP, pre-receive checks, caches, forks, landing merges, releases, backups | D1 `g1t-repos`, the git store, the git cache KV, the packs and backups buckets | `billing`, `events`, `identity`, `security`, `work` | | |
| 26 | 26 | | `services/work` | Rust | core | Issues, pull requests, comments, reviews, labels, milestones, rulesets, checks, statuses, CODEOWNERS, the merge queue | D1 `g1t-work` | `actions`, `events`, `identity`, `repos` | | |
| 27 | 27 | | `services/events` | Rust | core | The event bus and log, the audit log, the inbox | D1 `g1t-events`, the `g1t-events` queue and its subscriber queues | `billing`, `identity`, `notify`, `repos`, `work` | |
| 95 | 95 | too, for a phone clock a little off. Each code works once. After five wrong | |
| 96 | 96 | codes, or ten minutes, start the sign-in again. | |
| 97 | 97 | ||
| 98 | − | Lost your phone? Enter a recovery code instead of the app's code. Each | |
| 99 | − | works once, and your security log records its use. | |
| 98 | + | Lost your phone? Choose **Use a recovery code**, which gives the field a | |
| 99 | + | keyboard with letters, and enter one of your recovery codes; the field | |
| 100 | + | takes either kind of code whichever you choose, so a pasted recovery code | |
| 101 | + | works too. Each works once, and your security log records its use. | |
| 100 | 102 | ||
| 103 | + | The code field is marked as a one-time code, so a password manager that | |
| 104 | + | keeps your authenticator (1Password, iCloud Keychain, Bitwarden) can | |
| 105 | + | offer to fill it. | |
| 106 | + | ||
| 101 | 107 | Git over HTTPS never takes your password while two-factor authentication | |
| 102 | 108 | is on: use a [personal access token](#access-tokens) as the password, or | |
| 103 | 109 | [SSH](/guides/git/). Access tokens, SSH keys and OAuth applications are | |
| ⋯ | |||
| 1292 | 1298 | | Statements, invoices and audit logs | Kept with your username, for the workspaces they belong to. | | |
| 1293 | 1299 | | Your username | Never given to another account or workspace, so links, mentions and remotes that use it keep meaning what they meant. `ghost` is reserved for this, and nobody can register it. | | |
| 1294 | 1300 | ||
| 1301 | + | ## Asking for your personal data to be erased | |
| 1302 | + | ||
| 1303 | + | Deleting your account and asking for your personal data to be erased are | |
| 1304 | + | separate. You can ask for erasure even when something stands in the way | |
| 1305 | + | of deleting your account, such as a workspace you own alone or a bill. | |
| 1306 | + | ||
| 1307 | + | 1. Open [Settings → Account](https://g1t.sh/settings/account) and go to | |
| 1308 | + | **Erase my personal data**. | |
| 1309 | + | 2. Choose **Request erasure**. g1t records the request and shows its | |
| 1310 | + | number and the date we respond by: one calendar month from when we | |
| 1311 | + | received it. Asking again while it is open keeps the first date. | |
| 1312 | + | ||
| 1313 | + | Recording the request deletes nothing by itself. g1t's staff review it: | |
| 1314 | + | what can be erased, shared content such as messages and documents you | |
| 1315 | + | wrote in other people's workspaces, and anything we must keep (for | |
| 1316 | + | example invoices). Then we erase what can be erased, without the usual | |
| 1317 | + | 30-day wait, and write to you with the outcome: what was erased, what | |
| 1318 | + | remains and why. The page shows the request's status until then. | |
| 1319 | + | ||
| 1320 | + | For specific content, a request without signing in, or a question about | |
| 1321 | + | a request, write to support@g1t.sh. The | |
| 1322 | + | [Privacy Policy](https://g1t.sh/privacy) says what we keep and for how long. | |
| 1323 | + | ||
| 1295 | 1324 | ## What g1t stores | |
| 1296 | 1325 | ||
| 1297 | 1326 | Passwords are stored as salted PBKDF2-SHA256 hashes. Sessions and tokens are | |
| 40 | 40 | ||
| 41 | 41 | test("the built pages are not marked soon", () => { | |
| 42 | 42 | const built = navItems().filter((item) => !item.soon).map((item) => item.to); | |
| 43 | − | assert.deepEqual(built, ["/", "/reach-out", "/workspaces", "/enterprises", "/invites", "/aliases", "/requests", "/overages", "/velocity", "/invoices", "/credits", "/stripe", "/costs", "/abuse", "/blocked-domains", "/incidents", "/audit"]); | |
| 43 | + | assert.deepEqual(built, ["/", "/reach-out", "/workspaces", "/privacy", "/enterprises", "/invites", "/aliases", "/requests", "/overages", "/velocity", "/invoices", "/credits", "/stripe", "/costs", "/abuse", "/blocked-domains", "/incidents", "/audit"]); | |
| 44 | 44 | }); | |
| 45 | 45 | ||
| 46 | 46 | test("every soon page says what it will do, why, and what it will have", () => { |
| 66 | 66 | }; | |
| 67 | 67 | ||
| 68 | 68 | /** The numbers the sidebar can show, read by root.tsx on every page. */ | |
| 69 | − | export type NavCount = "incidents"; | |
| 69 | + | export type NavCount = "incidents" | "privacy"; | |
| 70 | 70 | ||
| 71 | 71 | export type NavCounts = Partial<Record<NavCount, number>>; | |
| 72 | 72 | ||
| ⋯ | |||
| 94 | 94 | icon: "customers", | |
| 95 | 95 | items: [ | |
| 96 | 96 | { label: "Workspaces", to: "/workspaces", icon: "workspaces", about: "Every workspace, who owns it, and how it pays." }, | |
| 97 | + | { label: "Privacy", to: "/privacy", icon: "people", count: "privacy", about: "Erasure requests, response deadlines, and cleanup retries." }, | |
| 97 | 98 | { label: "Enterprises", to: "/enterprises", icon: "enterprises", about: "Customers paying for several workspaces with one bill." }, | |
| 98 | 99 | { | |
| 99 | 100 | label: "Invites", | |
| 2 | 2 | ||
| 3 | 3 | import { | |
| 4 | 4 | type StatusAdminApi, | |
| 5 | + | privacyClient, | |
| 5 | 6 | accountsAdminClient, | |
| 6 | 7 | billingAdminClient, | |
| 7 | 8 | billingClient, | |
| ⋯ | |||
| 48 | 49 | ||
| 49 | 50 | /** STAFF_EMAILS, for suggesting staff in the incident roles. */ | |
| 50 | 51 | export const staffEmails = (): string => env.STAFF_EMAILS ?? ""; | |
| 52 | + | ||
| 53 | + | export const privacy = privacyClient(env.IDENTITY); | |
| 1 | + | import { privacy } from "./lib/services.server"; | |
| 1 | 2 | import { isRouteErrorResponse, Links, Meta, Outlet, useRouteLoaderData } from "react-router"; | |
| 2 | 3 | ||
| 3 | 4 | import type { Route } from "./+types/root"; | |
| ⋯ | |||
| 27 | 28 | export async function loader({ context }: Route.LoaderArgs) { | |
| 28 | 29 | const { email } = requireStaff(context); | |
| 29 | 30 | // The sidebar's counts: a service that does not answer shows none. | |
| 30 | − | const [incidents, alerts, caps, guard] = await Promise.all([ | |
| 31 | + | const [incidents, alerts, caps, guard, erasures] = await Promise.all([ | |
| 31 | 32 | settle(statusAdmin.openCount()), | |
| 32 | 33 | settle(admin.costAlerts()), | |
| 33 | 34 | settle(admin.spendCaps()), | |
| 34 | 35 | settle(admin.platformGuard()), | |
| 36 | + | settle(privacy.requests()), | |
| 35 | 37 | ]); | |
| 36 | − | const counts: NavCounts = { incidents: incidents.ok ? incidents.value : 0 }; | |
| 38 | + | const counts: NavCounts = { incidents: incidents.ok ? incidents.value : 0, privacy: erasures.ok ? erasures.value.filter(r => !r.closed_at).length : 0 }; | |
| 37 | 39 | // Every page says times in this zone (components/ui.tsx `When`). | |
| 38 | 40 | const { zone, chosen } = context.get(zoneContext); | |
| 39 | 41 | // Margin alerts (billing's margin guard): a red bar on every page until they clear. | |
| 8 | 8 | route("workspaces/deleted", "routes/deleted-workspaces.tsx"), | |
| 9 | 9 | route("workspaces/:slug", "routes/workspace.tsx"), | |
| 10 | 10 | route("workspaces/:slug/abuse", "routes/workspace-abuse.tsx"), | |
| 11 | + | route("privacy", "routes/privacy.tsx"), | |
| 11 | 12 | route("users/deleted", "routes/deleted-accounts.tsx"), | |
| 12 | 13 | route("users/:username", "routes/user.tsx"), | |
| 13 | 14 | route("enterprises", "routes/enterprises.tsx"), |
| 1 | + | import { Form, Link, data } from "react-router"; | |
| 2 | + | import type { Route } from "./+types/privacy"; | |
| 3 | + | import { Button, Input, Notice, PageHeader, Textarea, When } from "~/components/ui"; | |
| 4 | + | import { privacy } from "~/lib/services.server"; | |
| 5 | + | import { requireStaff } from "~/lib/staff"; | |
| 6 | + | ||
| 7 | + | const evidenceFields = ["identity", "downstream", "shared_content", "processors", "backups", "response", "retention"] as const; | |
| 8 | + | ||
| 9 | + | export async function loader({ context }: Route.LoaderArgs) { | |
| 10 | + | requireStaff(context); | |
| 11 | + | const [requests, pending] = await Promise.all([privacy.requests(), privacy.pending()]); | |
| 12 | + | return { requests, pending, now: new Date().toISOString() }; | |
| 13 | + | } | |
| 14 | + | ||
| 15 | + | export async function action({ request, context }: Route.ActionArgs) { | |
| 16 | + | // The worker refuses a post from anywhere but sudo's own pages (lib/access.ts isSameOrigin). | |
| 17 | + | const staff = requireStaff(context); | |
| 18 | + | const form = await request.formData(); | |
| 19 | + | const status = String(form.get("status") ?? ""); | |
| 20 | + | if (status === "intake") { | |
| 21 | + | const result = await privacy.intake(String(form.get("username") ?? ""), String(form.get("received_at") ?? ""), staff.email, String(form.get("notes") ?? "")); | |
| 22 | + | return result.ok ? { saved: true } : data({ error: result.error.message }, { status: 422 }); | |
| 23 | + | } | |
| 24 | + | const evidence = status === "completed" | |
| 25 | + | ? JSON.stringify(Object.fromEntries(evidenceFields.map(key => [key, String(form.get(key) ?? "")]))) | |
| 26 | + | : String(form.get("notes") ?? ""); | |
| 27 | + | const result = await privacy.review(String(form.get("id") ?? ""), staff.email, status, evidence); | |
| 28 | + | return result.ok ? { saved: true } : data({ error: result.error.message }, { status: 422 }); | |
| 29 | + | } | |
| 30 | + | ||
| 31 | + | export default function Privacy({ loaderData, actionData }: Route.ComponentProps) { | |
| 32 | + | return <main className="mx-auto max-w-5xl space-y-6 px-4 py-8"> | |
| 33 | + | <PageHeader title="Privacy requests" description="Respond without undue delay, by the recorded deadline. Automatic cleanup is only part of fulfilling an erasure request." /> | |
| 34 | + | {actionData && "error" in actionData && <Notice tone="error">{actionData.error}</Notice>} | |
| 35 | + | {actionData && "saved" in actionData && <Notice tone="ok">Review recorded.</Notice>} | |
| 36 | + | <p className="text-sm text-muted">Verify the request and assess shared content and lawful retention before using <Link to="/users/deleted" className="underline">account purge</Link>. A completed event publication still needs downstream verification. Follow <code>docs/operations/privacy-erasure.md</code>.</p> | |
| 37 | + | <details><summary className="cursor-pointer">Record a request received elsewhere</summary><Form method="post" className="my-3 space-y-3"> | |
| 38 | + | <label className="block text-sm">Username<Input name="username" required /></label> | |
| 39 | + | <label className="block text-sm">Original receipt time (UTC)<Input name="received_at" required placeholder="2026-10-14T15:30:00.000Z" /></label> | |
| 40 | + | <label className="block text-sm">Source and verification notes<Textarea name="notes" required rows={3} maxLength={16000} /></label> | |
| 41 | + | <Button name="status" value="intake">Record request</Button> | |
| 42 | + | </Form></details> | |
| 43 | + | {loaderData.requests.length === 0 && <p>No erasure requests recorded.</p>} | |
| 44 | + | {loaderData.requests.map(request => <section key={request.id} className="space-y-3 rounded-lg border border-line p-4"> | |
| 45 | + | <h2 className="font-medium"><Link to={`/users/${request.username}`} className="underline">{request.username}</Link> — {request.status}</h2> | |
| 46 | + | <p className="text-sm">{request.id}. Received <When at={request.requested_at} />. Response due <When at={request.due_at} />{!request.closed_at && request.due_at < loaderData.now && <strong className="text-danger"> — overdue</strong>}.</p> | |
| 47 | + | <p className="text-sm text-muted">{request.scope}</p> | |
| 48 | + | {request.reviews.length > 0 && <details><summary className="cursor-pointer text-sm">Case trail ({request.reviews.length})</summary> | |
| 49 | + | <ol className="my-2 space-y-2">{request.reviews.map((review, i) => <li key={i} className="text-sm"> | |
| 50 | + | <When at={review.at} time /> · {review.status} · {review.staff} | |
| 51 | + | <pre className="whitespace-pre-wrap break-words text-xs">{review.evidence}</pre> | |
| 52 | + | </li>)}</ol> | |
| 53 | + | </details>} | |
| 54 | + | {!request.closed_at && <Form method="post" className="space-y-3"> | |
| 55 | + | <input type="hidden" name="id" value={request.id} /> | |
| 56 | + | <label className="block text-sm">Review notes or refusal grounds, response and appeal information<Textarea name="notes" rows={3} maxLength={16000} /></label> | |
| 57 | + | <div className="flex gap-2"><Button name="status" value="reviewing">Save review</Button><Button name="status" value="refused" variant="quiet">Record refusal</Button></div> | |
| 58 | + | <details><summary className="cursor-pointer text-sm">Completion evidence</summary> | |
| 59 | + | <p className="my-3 text-sm text-muted">Record the actual outcome for each area, including evidence references, retained records and their reason and review date. Never paste credentials or unnecessary personal content. The response must already have been sent.</p> | |
| 60 | + | {evidenceFields.map(key => <label key={key} className="my-3 block text-sm">{key.replaceAll("_", " ")}<Textarea name={key} rows={2} maxLength={2000} /></label>)} | |
| 61 | + | <Button name="status" value="completed">Record completion</Button> | |
| 62 | + | </details> | |
| 63 | + | </Form>} | |
| 64 | + | </section>)} | |
| 65 | + | <section className="space-y-3"><h2 className="font-medium">Pending cleanup</h2> | |
| 66 | + | <p className="text-sm text-muted">Retries run every 15 minutes with backoff, up to 16 hours. Failed work remains here until it succeeds. This list does not prove that downstream event consumers have finished.</p> | |
| 67 | + | {loaderData.pending.length === 0 ? <p className="text-sm">No pending jobs.</p> : <ul className="space-y-2">{loaderData.pending.map(job => <li key={`${job.subject_id}:${job.target}`} className="text-sm"> | |
| 68 | + | <code>{job.subject_id}</code> · {job.target} · {job.attempts} attempts · retry <When at={job.retry_at} time />{job.last_error && ` · ${job.last_error}`} | |
| 69 | + | </li>)}</ul>} | |
| 70 | + | </section> | |
| 71 | + | </main>; | |
| 72 | + | } |
| 6 | 6 | * carries it until then. Kept mounted once loaded, so the conversation | |
| 7 | 7 | * stays while the panel is closed. | |
| 8 | 8 | */ | |
| 9 | − | import { Suspense, lazy, useState } from "react"; | |
| 9 | + | import { Suspense, lazy, useEffect, useState } from "react"; | |
| 10 | + | import { useLocation } from "react-router"; | |
| 10 | 11 | ||
| 11 | 12 | import { MarkGlyph } from "./logo"; | |
| 12 | 13 | import { Button } from "./ui/button"; | |
| ⋯ | |||
| 20 | 21 | export function AskButton({ slug }: { slug: string }) { | |
| 21 | 22 | const [open, setOpen] = useState(false); | |
| 22 | 23 | const [wanted, setWanted] = useState(false); | |
| 24 | + | const location = useLocation(); | |
| 25 | + | // Any completed navigation, including answer citations and the current DM, | |
| 26 | + | // reveals its destination. Revalidation and links opened in another tab do not. | |
| 27 | + | useEffect(() => setOpen(false), [location.key]); | |
| 23 | 28 | const change = (next: boolean) => { | |
| 24 | 29 | if (next) setWanted(true); | |
| 25 | 30 | setOpen(next); | |
| 12 | 12 | export function Field({ | |
| 13 | 13 | label, | |
| 14 | 14 | hint, | |
| 15 | + | htmlFor, | |
| 15 | 16 | children, | |
| 16 | 17 | }: { | |
| 17 | 18 | label: string; | |
| 18 | 19 | hint?: string; | |
| 20 | + | htmlFor?: string; | |
| 19 | 21 | children: ReactNode; | |
| 20 | 22 | }) { | |
| 21 | 23 | return ( | |
| 22 | − | <label className="block"> | |
| 24 | + | <label className="block" htmlFor={htmlFor}> | |
| 23 | 25 | <span className="mb-1.5 block text-sm font-medium text-muted"> | |
| 24 | 26 | {label} | |
| 25 | 27 | </span> |
| 115 | 115 | ||
| 116 | 116 | | What | How long | | |
| 117 | 117 | | --- | --- | | |
| 118 | − | | Your account, profile and content | While your account or workspace exists. After you delete it, kept 30 days so it can be restored, then purged. | | |
| 118 | + | | Your account and private content | Ordinary account closure has a 30-day recovery period before permanent cleanup begins. Shared content and records retained for legal reasons are assessed separately, as described below. An approved erasure request can be handled sooner. | | |
| 119 | 119 | | Agent sessions and logs | As long as their project, unless you delete them. | | |
| 120 | 120 | | Observability data | The project's retention, 1 to 30 days. | | |
| 121 | 121 | | The audit log | 7 days on a free workspace, 90 on the g1t plan, longer where agreed. | | |
| ⋯ | |||
| 124 | 124 | | Request logs | 7 days. | | |
| 125 | 125 | | Database history | g1t's databases keep 30 days of point-in-time history (Cloudflare D1 Time Travel), so purged data leaves it within 30 days of being purged. | | |
| 126 | 126 | ||
| 127 | − | Some content can stay after you leave because it is part of someone else's work: forks others made of your public repositories, and comments and reviews you left in other people's projects. Ask us, and we'll remove your name from what stays where we can. | |
| 127 | + | Some content can stay after you leave because it is part of someone else's work, including shared conversations and documents, forks, comments and reviews. Where appropriate, we replace account attribution with `ghost`. This does not necessarily remove personal information within the content itself. If you request erasure, we assess that content and any lawful grounds for retaining it, and explain the outcome. We keep a limited record of privacy requests and deletion identifiers where needed to demonstrate our response, prevent restoration of erased data, and prevent reassignment of deleted usernames. | |
| 128 | 128 | ||
| 129 | 129 | ## Your choices and rights | |
| 130 | 130 | ||
| ⋯ | |||
| 132 | 132 | ||
| 133 | 133 | - **See and correct** your information: most of it is in your settings, your workspaces and your projects. | |
| 134 | 134 | - **Take it with you**: clone your repositories with git, use the API for everything else, and export the audit log and statements as CSV or JSON. Ask us and we'll send you a copy of your account's personal information. | |
| 135 | − | - **Delete it**: delete content in the app, or write to us to delete your account or workspace. | |
| 135 | + | - **Delete it**: delete content in the app, request personal-data erasure from **Settings > Account**, or contact us. You can request erasure even when workspace ownership or billing prevents ordinary account closure. | |
| 136 | 136 | - **Object or restrict**: ask us to stop a particular use of your information. | |
| 137 | 137 | ||
| 138 | 138 | If you're in the EEA, the UK or another place with similar laws (such as under the GDPR), you also have the right to object to processing based on legitimate interests, and to complain to your data protection authority. If you're a California resident, you have the right to know what we collect, to delete it, to correct it, and not to be treated differently for using these rights. We don't sell or share personal information as California law defines those words. | |
| 102 | 102 | { | |
| 103 | 103 | date: "2026-10-14", | |
| 104 | 104 | change: | |
| 105 | − | "Removed the draft banner. Standardized contact addresses on support@g1t.sh and named Delaware in the governing-law clause. Clarified automatic renewal, consumer cancellation and refund rights, and consent for changes where required. Privacy Policy: clarified analytics consent and the limits of credential redaction; removed unverified assurances about provider contracts and transfer arrangements. Privacy Policy and Subprocessors: Cloudflare Turnstile protects email sign-up.", | |
| 105 | + | "Removed the draft banner. Standardized contact addresses on support@g1t.sh and named Delaware in the governing-law clause. Clarified automatic renewal, consumer cancellation and refund rights, and consent for changes where required. Privacy Policy: clarified analytics consent and the limits of credential redaction; removed unverified assurances about provider contracts and transfer arrangements. Privacy Policy and Subprocessors: Cloudflare Turnstile protects email sign-up. Privacy Policy: distinguished ordinary closure from personal-data erasure requests, shared content review, and minimal deletion records.", | |
| 106 | 106 | }, | |
| 107 | 107 | { | |
| 108 | 108 | date: "2026-10-13", |
| 1 | + | import assert from "node:assert/strict"; | |
| 2 | + | import { test } from "node:test"; | |
| 3 | + | ||
| 4 | + | import { otpInputProps } from "./otp.ts"; | |
| 5 | + | ||
| 6 | + | test("one-time-code fields are findable by password managers and carry no ignore attributes", () => { | |
| 7 | + | for (const recovery of [false, true]) { | |
| 8 | + | const props: Record<string, unknown> = otpInputProps({ id: "totp-code", recovery }); | |
| 9 | + | assert.equal(props.autoComplete, "one-time-code"); | |
| 10 | + | assert.equal(props.id, "totp-code"); | |
| 11 | + | assert.equal(props.name, "code"); | |
| 12 | + | assert.equal(props.type, "text"); | |
| 13 | + | for (const key of Object.keys(props)) assert.doesNotMatch(key, /1p|lpignore|bwignore|form-type/i); | |
| 14 | + | } | |
| 15 | + | }); | |
| 16 | + | ||
| 17 | + | test("an authenticator code field takes six digits, with or without the space", () => { | |
| 18 | + | const props = otpInputProps({ id: "totp-code" }); | |
| 19 | + | assert.equal(props.inputMode, "numeric"); | |
| 20 | + | const pattern = new RegExp(`^(?:${props.pattern})$`); | |
| 21 | + | for (const code of ["123456", "123 456"]) { | |
| 22 | + | assert.match(code, pattern); | |
| 23 | + | assert.ok(code.length <= props.maxLength); | |
| 24 | + | } | |
| 25 | + | }); | |
| 26 | + | ||
| 27 | + | test("a field that takes recovery codes neither cuts nor refuses one", () => { | |
| 28 | + | const props = otpInputProps({ id: "totp-code", recovery: true }); | |
| 29 | + | assert.equal(props.inputMode, "text"); | |
| 30 | + | assert.equal(props.pattern, undefined); | |
| 31 | + | for (const code of ["k7m2q-9xw4d", "K7M2Q 9XW4D", "123456"]) assert.ok(code.length <= props.maxLength); | |
| 32 | + | }); |
| 1 | + | /** | |
| 2 | + | * The attributes every one-time-code field shares: the sign-in's second | |
| 3 | + | * step, two-factor settings and email confirmation. Password managers | |
| 4 | + | * (1Password, iCloud Keychain, Bitwarden) and phones' SMS/email code | |
| 5 | + | * suggestions find the field by `autocomplete="one-time-code"` and a | |
| 6 | + | * stable id, so it carries no ignore attributes. A field that also takes | |
| 7 | + | * a recovery code (`k7m2q-9xw4d`) has a text keyboard and room for one: | |
| 8 | + | * a numeric `pattern` or a six-character `maxLength` would refuse or cut | |
| 9 | + | * a pasted recovery code before identity could check it. | |
| 10 | + | */ | |
| 11 | + | export function otpInputProps({ id, recovery = false }: { id: string; recovery?: boolean }) { | |
| 12 | + | return { | |
| 13 | + | id, | |
| 14 | + | name: "code", | |
| 15 | + | type: "text", | |
| 16 | + | autoComplete: "one-time-code", | |
| 17 | + | autoCapitalize: "none", | |
| 18 | + | autoCorrect: "off", | |
| 19 | + | spellCheck: false, | |
| 20 | + | inputMode: recovery ? "text" : "numeric", | |
| 21 | + | maxLength: recovery ? 20 : 7, | |
| 22 | + | pattern: recovery ? undefined : "[0-9 ]{6,7}", | |
| 23 | + | placeholder: recovery ? "k7m2q-9xw4d" : "123 456", | |
| 24 | + | } as const; | |
| 25 | + | } |
| 1 | 1 | import { env } from "cloudflare:workers"; | |
| 2 | 2 | ||
| 3 | 3 | import { | |
| 4 | + | privacyClient, | |
| 4 | 5 | accountsClient, | |
| 5 | 6 | actionsClient, | |
| 6 | 7 | advisoriesClient, | |
| ⋯ | |||
| 91 | 92 | * metrics and errors. Null where it is not deployed; the pages say so. | |
| 92 | 93 | */ | |
| 93 | 94 | export const telemetry = TELEMETRY ? telemetryClient(TELEMETRY) : null; | |
| 95 | + | ||
| 96 | + | export const privacy = privacyClient(env.IDENTITY); | |
| 128 | 128 | {address && ( | |
| 129 | 129 | <Form method="post" className="mt-6 space-y-4"> | |
| 130 | 130 | <input type="hidden" name="intent" value="confirm" /> | |
| 131 | − | <Field label="Confirmation code" hint={`The code and the link work for ${MINUTES} minutes.`}> | |
| 131 | + | <Field label="Confirmation code" htmlFor="email-verification-code" hint={`The code and the link work for ${MINUTES} minutes.`}> | |
| 132 | 132 | <Input | |
| 133 | + | id="email-verification-code" | |
| 133 | 134 | name="code" | |
| 135 | + | type="text" | |
| 136 | + | autoCapitalize="none" | |
| 137 | + | autoCorrect="off" | |
| 138 | + | spellCheck={false} | |
| 134 | 139 | autoComplete="one-time-code" | |
| 135 | 140 | inputMode="numeric" | |
| 136 | 141 | pattern="[0-9 -]*" |
| 1 | + | import { useState } from "react"; | |
| 1 | 2 | import { Form, Link, redirect } from "react-router"; | |
| 2 | 3 | ||
| 3 | 4 | import type { Route } from "./+types/login-two-factor"; | |
| 4 | 5 | import { AuthCard } from "../components/auth-card"; | |
| 5 | 6 | import { ErrorText, Field, Input, SubmitButton } from "../components/ui"; | |
| 7 | + | import { Button } from "../components/ui/button"; | |
| 6 | 8 | import { PENDING_COOKIE, TWO_FACTOR_COOKIE, cookie, readCookie } from "../lib/github"; | |
| 7 | 9 | import { githubSignIn } from "../lib/github.server"; | |
| 8 | 10 | import { page } from "../lib/meta"; | |
| 11 | + | import { otpInputProps } from "../lib/otp"; | |
| 9 | 12 | import { identity } from "../lib/services.server"; | |
| 10 | 13 | import { assertSameOrigin, clientOf, getViewer, nextPath, startSession } from "../lib/session.server"; | |
| 11 | 14 | ||
| ⋯ | |||
| 46 | 49 | } | |
| 47 | 50 | ||
| 48 | 51 | export default function LoginTwoFactor({ actionData }: Route.ComponentProps) { | |
| 52 | + | const [recovery, setRecovery] = useState(false); | |
| 49 | 53 | return ( | |
| 50 | 54 | <AuthCard | |
| 51 | 55 | title="Two-factor authentication" | |
| ⋯ | |||
| 56 | 60 | </Link> | |
| 57 | 61 | } | |
| 58 | 62 | > | |
| 59 | − | <Form method="post" className="space-y-4"> | |
| 60 | − | <Field label="Code" hint="Lost your phone? Enter one of your recovery codes instead."> | |
| 61 | − | <Input name="code" required autoFocus autoComplete="one-time-code" inputMode="numeric" maxLength={20} placeholder="123 456" /> | |
| 63 | + | <Form id="two-factor-sign-in" method="post" className="space-y-4"> | |
| 64 | + | <Field label={recovery ? "Recovery code" : "Authentication code"} htmlFor="totp-code"> | |
| 65 | + | {/* Either kind is accepted whichever is chosen, so a pasted recovery code is never cut; the choice sets the keyboard. */} | |
| 66 | + | <Input {...otpInputProps({ id: "totp-code", recovery: true })} inputMode={recovery ? "text" : "numeric"} placeholder={recovery ? "k7m2q-9xw4d" : "123 456"} required autoFocus /> | |
| 62 | 67 | </Field> | |
| 68 | + | {/* A phone shows digits for an app's code; a recovery code needs letters. */} | |
| 69 | + | <Button type="button" variant="link" onClick={() => setRecovery(!recovery)}> | |
| 70 | + | {recovery ? "Use an authenticator code" : "Use a recovery code"} | |
| 71 | + | </Button> | |
| 63 | 72 | <ErrorText>{actionData?.error}</ErrorText> | |
| 64 | 73 | <div className="pt-2 *:w-full"> | |
| 65 | 74 | <SubmitButton pending="Checking…">Verify</SubmitButton> | |
| 1 | − | import { Link, redirect } from "react-router"; | |
| 1 | + | import { Form, Link, redirect } from "react-router"; | |
| 2 | 2 | ||
| 3 | 3 | import type { Route } from "./+types/account"; | |
| 4 | 4 | import { DeleteAccountAction } from "../../components/delete-account"; | |
| 5 | 5 | import { DangerZone } from "../../components/danger-zone"; | |
| 6 | 6 | import { DeveloperModeSwitch } from "../../components/developer-mode-switch"; | |
| 7 | 7 | import { ThemeSwitch } from "../../components/theme-switch"; | |
| 8 | + | import { ErrorText, SubmitButton } from "../../components/ui"; | |
| 8 | 9 | import { githubSignIn } from "../../lib/github.server"; | |
| 9 | 10 | import { page } from "../../lib/meta"; | |
| 10 | − | import { accounts } from "../../lib/services.server"; | |
| 11 | + | import { accounts, privacy } from "../../lib/services.server"; | |
| 11 | 12 | import { assertSameOrigin, clientOf, endSession, requireUser, sessionTokenOf } from "../../lib/session.server"; | |
| 12 | 13 | ||
| 13 | 14 | export function meta(args: Route.MetaArgs) { | |
| ⋯ | |||
| 18 | 19 | const user = requireUser(context, request); | |
| 19 | 20 | // What deleting it would take, and anything in the way, shown before | |
| 20 | 21 | // anyone types: workspaces you own alone, each with what billing needs. | |
| 21 | − | const [deletion, github] = await Promise.all([ | |
| 22 | + | const [deletion, github, erasure] = await Promise.all([ | |
| 22 | 23 | accounts.checkAccountDeletion(user).catch(() => null), | |
| 23 | 24 | githubSignIn.account(user).catch(() => null), | |
| 25 | + | privacy.mine(user).catch(() => undefined), | |
| 24 | 26 | ]); | |
| 25 | 27 | return { | |
| 28 | + | erasure, | |
| 26 | 29 | username: user.username, | |
| 27 | 30 | deletion: deletion?.ok ? deletion.value : null, | |
| 28 | 31 | hasPassword: github?.hasPassword ?? true, | |
| ⋯ | |||
| 38 | 41 | assertSameOrigin(request); | |
| 39 | 42 | const user = requireUser(context, request); | |
| 40 | 43 | const form = await request.formData(); | |
| 44 | + | if (form.get("intent") === "request-erasure") { | |
| 45 | + | // Identity unreachable is not a refusal: say so, and where else to ask. | |
| 46 | + | const result = await privacy.request(user).catch(() => null); | |
| 47 | + | if (!result) return { erasureError: "We could not record your request just now. Try again, or email support@g1t.sh." }; | |
| 48 | + | return result.ok ? { erasureNotice: `Request ${result.value.id} received. We will respond by ${result.value.due_at.slice(0, 10)}.` } : { erasureError: result.error.message }; | |
| 49 | + | } | |
| 41 | 50 | if (form.get("intent") !== "delete-account") return null; | |
| 42 | 51 | const password = String(form.get("password") ?? ""); | |
| 43 | 52 | const result = await accounts.deleteAccount(user, String(form.get("confirm") ?? ""), { | |
| ⋯ | |||
| 83 | 92 | <DeveloperModeSwitch /> | |
| 84 | 93 | </div> | |
| 85 | 94 | </section> | |
| 95 | + | <section aria-labelledby="erasure-heading" className="space-y-3"> | |
| 96 | + | <h2 id="erasure-heading" className="font-medium">Erase my personal data</h2> | |
| 97 | + | <p className="text-sm text-muted">Request erasure even if workspace ownership or billing prevents account closure. We review shared content and any records we must retain, then explain the outcome. This request does not immediately delete your account.</p> | |
| 98 | + | {loaderData.erasure ? <p className="text-sm">Request {loaderData.erasure.id}: {loaderData.erasure.status}. Response due {loaderData.erasure.due_at.slice(0, 10)}.</p> : loaderData.erasure === undefined ? <p className="text-sm">Request status is unavailable. Try again or email <a href="mailto:support@g1t.sh" className="underline">support@g1t.sh</a>.</p> : null} | |
| 99 | + | {actionData?.erasureNotice && <p role="status" className="text-sm">{actionData.erasureNotice}</p>} | |
| 100 | + | <ErrorText>{actionData?.erasureError}</ErrorText> | |
| 101 | + | {(!loaderData.erasure || ["completed", "refused"].includes(loaderData.erasure.status)) && <Form method="post"> | |
| 102 | + | <SubmitButton name="intent" value="request-erasure" pending="Recording request...">Request erasure</SubmitButton> | |
| 103 | + | </Form>} | |
| 104 | + | <p className="text-sm text-muted">For specific content, a request without signing in, or a question about an existing request, email <a href="mailto:support@g1t.sh" className="underline">support@g1t.sh</a>.</p> | |
| 105 | + | </section> | |
| 86 | 106 | <DangerZone> | |
| 87 | 107 | <DeleteAccountAction | |
| 88 | 108 | username={username} | |
| 11 | 11 | import { Card } from "../../components/ui/card"; | |
| 12 | 12 | import { githubSignIn } from "../../lib/github.server"; | |
| 13 | 13 | import { page } from "../../lib/meta"; | |
| 14 | + | import { otpInputProps } from "../../lib/otp"; | |
| 14 | 15 | import { accounts } from "../../lib/services.server"; | |
| 15 | 16 | import { assertSameOrigin, requireUser } from "../../lib/session.server"; | |
| 16 | 17 | import { TWO_FACTOR_INTENTS, twoFactorAction } from "../../lib/two-factor.server"; | |
| ⋯ | |||
| 49 | 50 | /** The six digits an app shows, typed. */ | |
| 50 | 51 | function CodeInput({ label = "Code from your app", recovery = false }: { label?: string; recovery?: boolean }) { | |
| 51 | 52 | return ( | |
| 52 | − | <Field label={label} hint={recovery ? "Or a recovery code, such as k7m2q-9xw4d." : undefined}> | |
| 53 | − | <Input | |
| 54 | − | name="code" | |
| 55 | − | required | |
| 56 | − | autoComplete="one-time-code" | |
| 57 | − | inputMode={recovery ? "text" : "numeric"} | |
| 58 | − | pattern={recovery ? undefined : "[0-9 ]{6,7}"} | |
| 59 | − | maxLength={recovery ? 20 : 7} | |
| 60 | − | placeholder={recovery ? "123456" : "123 456"} | |
| 61 | − | /> | |
| 53 | + | <Field label={label} htmlFor="totp-code" hint={recovery ? "Or a recovery code, such as k7m2q-9xw4d." : undefined}> | |
| 54 | + | <Input {...otpInputProps({ id: "totp-code", recovery })} required /> | |
| 62 | 55 | </Field> | |
| 63 | 56 | ); | |
| 64 | 57 | } | |
| 1 | + | import assert from "node:assert/strict"; | |
| 2 | + | import { readFileSync, readdirSync } from "node:fs"; | |
| 3 | + | import { DatabaseSync } from "node:sqlite"; | |
| 4 | + | import { test } from "node:test"; | |
| 5 | + | import { build } from "esbuild"; | |
| 6 | + | ||
| 7 | + | // Execute the real service code against SQLite, with only the Cloudflare host replaced. | |
| 8 | + | async function service(path) { | |
| 9 | + | const bundle = await build({ entryPoints: [path], bundle: true, write: false, format: "esm", platform: "node", plugins: [{ name: "worker-host", setup(build) { | |
| 10 | + | build.onResolve({ filter: /^yjs$/ }, () => ({ path: import.meta.resolve("yjs"), external: true })); | |
| 11 | + | build.onResolve({ filter: /^cloudflare:workers$/ }, () => ({ path: "host", namespace: "host" })); | |
| 12 | + | build.onLoad({ filter: /.*/, namespace: "host" }, () => ({ contents: "export class DurableObject { constructor(ctx, env) { this.ctx = ctx; this.env = env; } }" })); | |
| 13 | + | }}] }); | |
| 14 | + | return import(`data:text/javascript;base64,${Buffer.from(bundle.outputFiles[0].text).toString("base64")}`); | |
| 15 | + | } | |
| 16 | + | ||
| 17 | + | function database(service) { | |
| 18 | + | const raw = new DatabaseSync(":memory:"); | |
| 19 | + | const dir = new URL(`../services/${service}/migrations/`, import.meta.url); | |
| 20 | + | for (const name of readdirSync(dir).sort()) raw.exec(readFileSync(new URL(name, dir), "utf8")); | |
| 21 | + | raw.exec("PRAGMA foreign_keys = ON"); | |
| 22 | + | const run = (s) => ({ meta: { changes: Number(raw.prepare(s.sql).run(...s.params).changes) } }); | |
| 23 | + | const prepare = (sql, params = []) => ({ sql, params, bind: (...p) => prepare(sql, p), first: async () => raw.prepare(sql).get(...params) ?? null, all: async () => ({ results: raw.prepare(sql).all(...params) }), run: async () => run({ sql, params }) }); | |
| 24 | + | return { raw, prepare, batch: async (statements) => { raw.exec("BEGIN"); try { const result = statements.map(run); raw.exec("COMMIT"); return result; } catch (e) { raw.exec("ROLLBACK"); throw e; } } }; | |
| 25 | + | } | |
| 26 | + | const namespace = (get) => ({ idFromName: id => id, get }); | |
| 27 | + | const AT = "2026-10-14T00:00:00.000Z"; | |
| 28 | + | ||
| 29 | + | test("identity purge work is atomic, idempotent and a prior restore prevents scheduling", () => { | |
| 30 | + | const db = database("identity").raw; | |
| 31 | + | const enqueue = db.prepare(readFileSync(new URL("../services/identity/src/purge_job.sql", import.meta.url), "utf8")); | |
| 32 | + | db.prepare("INSERT INTO users (id, username, password_hash, created_at, deleted_at, purge_after) VALUES ('usr_fixture', 'fixture', 'fixture', ?, ?, ?)").run(AT, AT, AT); | |
| 33 | + | const args = { "?1": "usr_fixture", "?2": "user", "?3": "CHAT", "?4": "{}", "?5": AT }; | |
| 34 | + | db.exec("BEGIN"); | |
| 35 | + | enqueue.run(args); | |
| 36 | + | db.exec("DELETE FROM users WHERE id = 'usr_fixture'"); | |
| 37 | + | db.exec("ROLLBACK"); | |
| 38 | + | assert.equal(db.prepare("SELECT count(*) n FROM purge_jobs").get().n, 0); | |
| 39 | + | assert.ok(db.prepare("SELECT 1 FROM users WHERE id = 'usr_fixture'").get()); | |
| 40 | + | db.exec("UPDATE users SET deleted_at = NULL, purge_after = NULL WHERE id = 'usr_fixture'"); | |
| 41 | + | enqueue.run(args); | |
| 42 | + | assert.equal(db.prepare("SELECT count(*) n FROM purge_jobs").get().n, 0); | |
| 43 | + | db.prepare("UPDATE users SET deleted_at = ?, purge_after = ? WHERE id = 'usr_fixture'").run(AT, AT); | |
| 44 | + | db.exec("BEGIN"); | |
| 45 | + | enqueue.run(args); | |
| 46 | + | enqueue.run(args); | |
| 47 | + | db.exec("DELETE FROM users WHERE id = 'usr_fixture'; COMMIT"); | |
| 48 | + | assert.equal(db.prepare("SELECT count(*) n FROM purge_jobs").get().n, 1); | |
| 49 | + | assert.equal(db.prepare("SELECT completed_at FROM purge_jobs").get().completed_at, null); | |
| 50 | + | }); | |
| 51 | + | ||
| 52 | + | test("chat purges the private AI conversation, preserves other tenants, retries failed rooms and rejects late writes", async () => { | |
| 53 | + | const { purgePersonalData } = await service("services/chat/src/erasure.ts"); | |
| 54 | + | const db = database("chat"); | |
| 55 | + | const { G1T_KEY } = await service("packages/contracts/src/chat.ts"); | |
| 56 | + | for (const [id, person] of [["mine", "usr_a"], ["other", "usr_b"]]) { | |
| 57 | + | db.raw.prepare("INSERT INTO channels (id, workspace_id, kind, created_by, created_at) VALUES (?, ?, 'dm', ?, ?)").run(id, id, `user:${person}`, AT); | |
| 58 | + | for (const key of [`user:${person}`, G1T_KEY]) db.raw.prepare("INSERT INTO channel_members (channel_id, principal, joined_at) VALUES (?, ?, ?)").run(id, key, AT); | |
| 59 | + | db.raw.prepare("INSERT INTO messages (id, channel_id, author, body, created_at) VALUES (?, ?, ?, 'private prompt', ?)").run(id, id, `user:${person}`, AT); | |
| 60 | + | } | |
| 61 | + | let fail = true; | |
| 62 | + | const env = { DB: db, ROOMS: namespace(() => ({ erase: async () => { if (fail) throw Error("unavailable"); } })), AVATARS: { delete: async () => {} } }; | |
| 63 | + | await assert.rejects(purgePersonalData(env, { kind: "user", id: "usr_a" })); | |
| 64 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM messages").get().n, 2); | |
| 65 | + | fail = false; | |
| 66 | + | assert.deepEqual(await purgePersonalData(env, { kind: "user", id: "usr_a" }), { ok: true }); | |
| 67 | + | assert.deepEqual(await purgePersonalData(env, { kind: "user", id: "usr_a" }), { ok: true }); | |
| 68 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM messages").get().n, 1); | |
| 69 | + | assert.equal(db.raw.prepare("SELECT body FROM messages WHERE id = 'other'").get().body, "private prompt"); | |
| 70 | + | db.raw.prepare("INSERT INTO messages (id, channel_id, author, body, created_at) VALUES ('late', 'mine', 'user:usr_a', 'late', ?)").run(AT); | |
| 71 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM messages").get().n, 1); | |
| 72 | + | }); | |
| 73 | + | ||
| 74 | + | test("artifacts keep retry references when file deletion fails, then remove private state with FK enforcement", async () => { | |
| 75 | + | const { purgePersonalData } = await service("services/artifacts/src/erasure.ts"); | |
| 76 | + | const db = database("artifacts"); | |
| 77 | + | for (const [id, owner] of [["mine", "usr_a"], ["other", "usr_b"]]) { | |
| 78 | + | db.raw.prepare("INSERT INTO folios (id, workspace_id, kind, owner, position, acl_root, path, created_by, created_at, updated_at, edited_at) VALUES (?, ?, 'doc', ?, 1, ?, ?, ?, ?, ?, ?)").run(id, id, `user:${owner}`, id, `/${id}/`, `user:${owner}`, AT, AT, AT); | |
| 79 | + | db.raw.prepare("INSERT INTO folio_files (id, workspace_id, folio_id, key, name, content_type, bytes, created_by, created_at) VALUES (?, ?, ?, ?, 'private.txt', 'text/plain', 10, ?, ?)").run(id, id, id, id, `user:${owner}`, AT); | |
| 80 | + | } | |
| 81 | + | let fail = true; const deleted = [], rooms = []; | |
| 82 | + | const env = { DB: db, FOLIOS: namespace(id => ({ forgetPerson: async () => {}, destroy: async () => { rooms.push(id); } })), FILES: { delete: async key => { if (fail) throw Error("storage down"); deleted.push(key); } } }; | |
| 83 | + | await assert.rejects(purgePersonalData(env, { kind: "user", id: "usr_a" })); | |
| 84 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM folio_files").get().n, 2); | |
| 85 | + | fail = false; | |
| 86 | + | assert.deepEqual(await purgePersonalData(env, { kind: "user", id: "usr_a" }), { ok: true }); | |
| 87 | + | assert.deepEqual(await purgePersonalData(env, { kind: "user", id: "usr_a" }), { ok: true }); | |
| 88 | + | assert.deepEqual(deleted, ["docs/mine"]); | |
| 89 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM folios").get().n, 1); | |
| 90 | + | assert.equal(db.raw.prepare("SELECT id FROM folio_files").get().id, "other"); | |
| 91 | + | assert.ok(rooms.every(id => id === "mine")); | |
| 92 | + | assert.deepEqual(await purgePersonalData(env, { kind: "workspace", id: "other" }), { ok: true }); | |
| 93 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM folios").get().n, 0); | |
| 94 | + | }); | |
| 95 | + | ||
| 96 | + | function objectState() { | |
| 97 | + | const raw = new DatabaseSync(":memory:"); | |
| 98 | + | const kv = new Map(); | |
| 99 | + | const sql = { exec(query, ...params) { | |
| 100 | + | const stmt = raw.prepare(query); | |
| 101 | + | if (stmt.columns().length) { const rows = stmt.all(...params); return { toArray: () => rows, one: () => rows[0], [Symbol.iterator]: () => rows[Symbol.iterator]() }; } | |
| 102 | + | const result = stmt.run(...params); return { toArray: () => [], rowsWritten: Number(result.changes) }; | |
| 103 | + | } }; | |
| 104 | + | return { raw, blockConcurrencyWhile: fn => fn(), storage: { sql, get: async key => kv.get(key), put: async (key, value) => { kv.set(key, value); }, delete: async key => kv.delete(key), deleteAlarm: async () => {}, transactionSync(fn) { raw.exec("BEGIN"); try { fn(); raw.exec("COMMIT"); } catch (e) { raw.exec("ROLLBACK"); throw e; } } }, getWebSockets: () => [], setWebSocketAutoResponse: () => {}, waitUntil: () => {} }; | |
| 105 | + | } | |
| 106 | + | ||
| 107 | + | test("notification erasure retries presence cleanup, clears credentials and permanently blocks late delivery", async () => { | |
| 108 | + | globalThis.WebSocketRequestResponsePair = class {}; | |
| 109 | + | const { Feed } = await service("services/notify/src/feed.ts"); | |
| 110 | + | const state = objectState(); let fail = true, calls = 0; | |
| 111 | + | const feed = new Feed(state, { ROOMS: namespace(() => ({ forget: async () => { calls++; if (fail) throw Error("offline"); } })) }); | |
| 112 | + | state.raw.exec(`INSERT INTO meta VALUES ('workspaces', '["acme"]'); INSERT INTO devices VALUES ('token', 'ios', 'phone', 1); INSERT INTO notifications VALUES ('n', 1, '{}');`); | |
| 113 | + | await assert.rejects(feed.erase("usr_a")); | |
| 114 | + | // A late delivery is dropped quietly, never refused: senders would log and retry it. | |
| 115 | + | assert.deepEqual(await feed.deliver([{ workspace: "acme", notification: { id: "late", created_at: AT } }]), { ok: true }); | |
| 116 | + | assert.deepEqual(await feed.notify({ id: "late", kind: "mention", workspace: "acme", created_at: AT }), { ok: true }); | |
| 117 | + | fail = false; | |
| 118 | + | assert.deepEqual(await feed.erase("usr_a"), { ok: true }); | |
| 119 | + | assert.equal(calls, 2); | |
| 120 | + | for (const table of ["devices", "notifications", "counts", "subscriptions", "visits", "watching"]) assert.equal(state.raw.prepare(`SELECT count(*) n FROM ${table}`).get().n, 0); | |
| 121 | + | assert.deepEqual(state.raw.prepare("SELECT key FROM meta").all().map(r => r.key), ["erased"]); | |
| 122 | + | await assert.rejects(feed.registerDevice({ token: "late", platform: "ios" }), /Account erased/); | |
| 123 | + | assert.equal((await feed.fetch(new Request("https://test/live"))).status, 410); | |
| 124 | + | assert.deepEqual(await feed.erase("usr_a"), { ok: true }); | |
| 125 | + | }); | |
| 126 | + | ||
| 127 | + | test("destroyed document rooms reject late ensure calls and keep only a tombstone", async () => { | |
| 128 | + | globalThis.WebSocketRequestResponsePair = class {}; | |
| 129 | + | for (const [file, name, init] of [ | |
| 130 | + | ["services/artifacts/src/room.ts", "PageRoom", { page_id: "p", workspace_slug: "acme", markdown: "private" }], | |
| 131 | + | ["services/artifacts/src/folios/room.ts", "FolioRoom", { folio_id: "f", kind: "doc", workspace_slug: "acme", text: "private" }], | |
| 132 | + | ]) { | |
| 133 | + | const module = await service(file), state = objectState(); | |
| 134 | + | const room = new module[name](state, {}); | |
| 135 | + | state.raw.exec("INSERT INTO meta VALUES ('workspace_slug', '\"acme\"'); INSERT INTO snapshot VALUES (1, x'01');"); | |
| 136 | + | await room.destroy(); | |
| 137 | + | await room.destroy(); | |
| 138 | + | assert.equal(state.raw.prepare("SELECT count(*) n FROM snapshot").get().n, 0); | |
| 139 | + | assert.deepEqual(state.raw.prepare("SELECT key FROM meta").all().map(r => r.key), ["erased"]); | |
| 140 | + | await assert.rejects(room.ensure(init), /Document erased/); | |
| 141 | + | assert.equal((await room.fetch(new Request("https://test/live"))).status, 410); | |
| 142 | + | assert.deepEqual(state.raw.prepare("SELECT key FROM meta").all().map(r => r.key), ["erased"]); | |
| 143 | + | } | |
| 144 | + | }); | |
| 145 | + | ||
| 146 | + | test("erasing one member of a shared channel leaves everyone else's conversation alone", async () => { | |
| 147 | + | const { purgePersonalData } = await service("services/chat/src/erasure.ts"); | |
| 148 | + | const db = database("chat"); | |
| 149 | + | db.raw.prepare("INSERT INTO channels (id, workspace_id, kind, name, created_by, created_at) VALUES ('general', 'ws', 'channel', 'general', 'user:usr_a', ?)").run(AT); | |
| 150 | + | for (const person of ["usr_a", "usr_b"]) db.raw.prepare("INSERT INTO channel_members (channel_id, principal, joined_at) VALUES ('general', ?, ?)").run(`user:${person}`, AT); | |
| 151 | + | db.raw.prepare("INSERT INTO messages (id, channel_id, author, body, created_at) VALUES ('m1', 'general', 'user:usr_a', 'from a', ?), ('m2', 'general', 'user:usr_b', 'from b', ?)").run(AT, AT); | |
| 152 | + | db.raw.prepare("INSERT INTO reactions (message_id, principal, emoji, created_at) VALUES ('m2', 'user:usr_a', 'ok', ?), ('m2', 'user:usr_b', 'ok', ?)").run(AT, AT); | |
| 153 | + | const erased = []; | |
| 154 | + | const env = { DB: db, ROOMS: namespace(() => ({ erase: async (id, principal) => { erased.push([id, principal]); } })), AVATARS: { delete: async () => {} } }; | |
| 155 | + | assert.deepEqual(await purgePersonalData(env, { kind: "user", id: "usr_a" }), { ok: true }); | |
| 156 | + | // The room shuts out only the erased person; it is never cleared for everyone. | |
| 157 | + | assert.deepEqual(erased, [["general", "user:usr_a"]]); | |
| 158 | + | assert.deepEqual(db.raw.prepare("SELECT id, author, body FROM messages ORDER BY id").all().map(r => ({ ...r })), [ | |
| 159 | + | { id: "m1", author: "user:usr_ghost", body: "from a" }, | |
| 160 | + | { id: "m2", author: "user:usr_b", body: "from b" }, | |
| 161 | + | ]); | |
| 162 | + | assert.deepEqual(db.raw.prepare("SELECT principal FROM channel_members").all().map(r => r.principal), ["user:usr_b"]); | |
| 163 | + | assert.deepEqual(db.raw.prepare("SELECT principal FROM reactions").all().map(r => r.principal), ["user:usr_b"]); | |
| 164 | + | assert.equal(db.raw.prepare("SELECT created_by FROM channels").get().created_by, "user:usr_ghost"); | |
| 165 | + | }); | |
| 166 | + | ||
| 167 | + | test("a shared channel's room shuts out only the erased member and tells nobody it was cleared", async () => { | |
| 168 | + | globalThis.WebSocketRequestResponsePair = class {}; | |
| 169 | + | const { ChannelRoom } = await service("services/chat/src/room.ts"); | |
| 170 | + | const state = objectState(); | |
| 171 | + | const socket = (who) => ({ who, sent: [], closed: null, send(text) { this.sent.push(text); }, close(code) { this.closed = code; } }); | |
| 172 | + | const sockets = [socket("user:usr_a"), socket("user:usr_b")]; | |
| 173 | + | state.getWebSockets = (tag) => sockets.filter(s => !tag || s.who === tag); | |
| 174 | + | await state.storage.put("job", { message: { id: "answer_for_b" } }); | |
| 175 | + | const deletes = []; | |
| 176 | + | const room = new ChannelRoom(state, { DB: { prepare: () => ({ bind: (id) => ({ run: async () => { deletes.push(id); } }) }) } }); | |
| 177 | + | await room.erase("general", "user:usr_a"); | |
| 178 | + | assert.equal(sockets[0].closed, 4403); | |
| 179 | + | assert.equal(sockets[1].closed, null); | |
| 180 | + | assert.deepEqual(sockets[1].sent, []); | |
| 181 | + | // Someone else's pending answer carries on. | |
| 182 | + | assert.deepEqual(await state.storage.get("job"), { message: { id: "answer_for_b" } }); | |
| 183 | + | assert.deepEqual(deletes, []); | |
| 184 | + | assert.equal((await room.fetch(new Request("https://test/live"))).status, 426); | |
| 185 | + | }); | |
| 186 | + | ||
| 187 | + | test("a workspace's large channel is deleted in chunks across calls, and nothing of it comes back", async () => { | |
| 188 | + | const { purgePersonalData } = await service("services/chat/src/erasure.ts"); | |
| 189 | + | const db = database("chat"); | |
| 190 | + | db.raw.prepare("INSERT INTO channels (id, workspace_id, kind, name, created_by, created_at) VALUES ('big', 'ws', 'channel', 'big', 'user:usr_a', ?), ('kept', 'other', 'channel', 'kept', 'user:usr_b', ?)").run(AT, AT); | |
| 191 | + | const insert = db.raw.prepare("INSERT INTO messages (id, channel_id, author, body, created_at) VALUES (?, ?, 'user:usr_a', 'x', ?)"); | |
| 192 | + | for (let i = 0; i < 5600; i++) insert.run(`m${String(i).padStart(5, "0")}`, "big", AT); | |
| 193 | + | insert.run("other", "kept", AT); | |
| 194 | + | const env = { DB: db, ROOMS: namespace(() => ({ erase: async () => {} })), AVATARS: { delete: async () => {} } }; | |
| 195 | + | assert.deepEqual(await purgePersonalData(env, { kind: "workspace", id: "ws" }), { ok: false, pending: true }); | |
| 196 | + | assert.ok(db.raw.prepare("SELECT count(*) n FROM messages WHERE channel_id = 'big'").get().n > 0); | |
| 197 | + | assert.deepEqual(await purgePersonalData(env, { kind: "workspace", id: "ws" }), { ok: true }); | |
| 198 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM messages WHERE channel_id = 'big'").get().n, 0); | |
| 199 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM channels WHERE workspace_id = 'ws'").get().n, 0); | |
| 200 | + | assert.equal(db.raw.prepare("SELECT body FROM messages WHERE id = 'other'").get().body, "x"); | |
| 201 | + | db.raw.prepare("INSERT INTO channels (id, workspace_id, kind, name, created_by, created_at) VALUES ('late', 'ws', 'channel', 'late', 'user:usr_b', ?)").run(AT); | |
| 202 | + | db.raw.prepare("INSERT INTO chat_settings (workspace_id) VALUES ('ws')").run(); | |
| 203 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM channels WHERE workspace_id = 'ws'").get().n, 0); | |
| 204 | + | assert.equal(db.raw.prepare("SELECT count(*) n FROM chat_settings").get().n, 0); | |
| 205 | + | }); | |
| 206 | + | ||
| 207 | + | test("a purged workspace's slug can be made again: only what was said before the purge is dropped", async () => { | |
| 208 | + | globalThis.WebSocketRequestResponsePair = class {}; | |
| 209 | + | const { Feed } = await service("services/notify/src/feed.ts"); | |
| 210 | + | const { Room } = await service("services/notify/src/room.ts"); | |
| 211 | + | const state = objectState(); | |
| 212 | + | const feed = new Feed(state, {}); | |
| 213 | + | const before = new Date(Date.now() - 60_000).toISOString(); | |
| 214 | + | state.raw.prepare("INSERT INTO notifications VALUES ('old', 1, ?)").run(JSON.stringify({ id: "old", workspace: "alice", created_at: before })); | |
| 215 | + | state.raw.prepare("INSERT INTO meta VALUES ('complete:alice', '1'), ('forgot:alice:ch_1', ?), ('forgot:alicex:ch_1', ?)").run(before, before); | |
| 216 | + | assert.deepEqual(await feed.eraseWorkspace("alice"), { ok: true }); | |
| 217 | + | assert.equal(state.raw.prepare("SELECT count(*) n FROM notifications").get().n, 0); | |
| 218 | + | assert.deepEqual(state.raw.prepare("SELECT key FROM meta WHERE key LIKE 'forgot:%' OR key LIKE 'complete:%'").all().map(r => r.key), ["forgot:alicex:ch_1"]); | |
| 219 | + | // A late notification from before the purge is dropped; one from the workspace made again is kept. | |
| 220 | + | await feed.notify({ id: "late", kind: "mention", workspace: "alice", title: "t", url: "/alice", created_at: before }); | |
| 221 | + | await feed.notify({ id: "new", kind: "mention", workspace: "alice", title: "t", url: "/alice", created_at: new Date(Date.now() + 1000).toISOString() }); | |
| 222 | + | assert.deepEqual(state.raw.prepare("SELECT id FROM notifications").all().map(r => r.id), ["new"]); | |
| 223 | + | ||
| 224 | + | const room = new Room(objectState(), {}); | |
| 225 | + | await room.report("alice", { user_id: "usr_b", at: 1, presence: "online" }); | |
| 226 | + | assert.deepEqual(await room.eraseMembers(), ["usr_b"]); | |
| 227 | + | await room.dropPerson("usr_b"); | |
| 228 | + | assert.equal(await room.report("alice", { user_id: "usr_c", at: 2, presence: "online" }), null); | |
| 229 | + | await room.report("alice", { user_id: "usr_b", at: Date.now() + 1000, presence: "online" }); | |
| 230 | + | assert.deepEqual((await room.people()).map(p => p.user_id), ["usr_b"]); | |
| 231 | + | }); | |
| 232 | + | ||
| 233 | + | test("vectors written while a document was being erased are taken back", async () => { | |
| 234 | + | const { dropIfErased, erasedSource } = await service("services/artifacts/src/indexer.ts"); | |
| 235 | + | const db = database("artifacts"); | |
| 236 | + | const deleted = []; | |
| 237 | + | const store = { delete: async ids => { deleted.push(...ids); } }; | |
| 238 | + | await dropIfErased(db, store, "folio", "f1", "ws", ["f1:0"]); | |
| 239 | + | assert.deepEqual(deleted, []); | |
| 240 | + | db.raw.prepare("INSERT INTO erased_documents (id, kind) VALUES ('f1', 'folio')").run(); | |
| 241 | + | assert.equal(await erasedSource(db, "folio", "f1", "ws"), true); | |
| 242 | + | assert.equal(await erasedSource(db, "page", "f1", "ws"), false); | |
| 243 | + | await dropIfErased(db, store, "folio", "f1", "ws", ["f1:0", "f1:1"]); | |
| 244 | + | assert.deepEqual(deleted, ["f1:0", "f1:1"]); | |
| 245 | + | db.raw.prepare("INSERT INTO erased_subjects VALUES ('ws2', 'workspace')").run(); | |
| 246 | + | assert.equal(await erasedSource(db, null, "repo_file", "ws2"), true); | |
| 247 | + | }); | |
| 248 | + | ||
| 249 | + | test("a workspace's uploads are swept a page at a time, resuming after a failure", async () => { | |
| 250 | + | const { purgePersonalData } = await service("services/artifacts/src/erasure.ts"); | |
| 251 | + | const db = database("artifacts"); | |
| 252 | + | const insert = db.raw.prepare("INSERT INTO files (id, workspace_id, page_id, key, name, content_type, bytes, created_by, created_at) VALUES (?, ?, NULL, ?, 'f', 'text/plain', 1, 'user:usr_a', ?)"); | |
| 253 | + | for (let i = 0; i < 2100; i++) insert.run(`f${i}`, "ws", `k${i}`, AT); | |
| 254 | + | insert.run("kept", "other", "kept", AT); | |
| 255 | + | let fail = 3; const deleted = new Set(); | |
| 256 | + | const env = { DB: db, FILES: { delete: async key => { if (key === "docs/k5" && fail-- > 0) throw Error("storage down"); deleted.add(key); } } }; | |
| 257 | + | await assert.rejects(purgePersonalData(env, { kind: "workspace", id: "ws" })); | |
| 258 | + | let result; | |
| 259 | + | for (let i = 0; i < 6 && !result?.ok; i++) result = await purgePersonalData(env, { kind: "workspace", id: "ws" }).catch(() => null); | |
| 260 | + | assert.deepEqual(result, { ok: true }); | |
| 261 | + | assert.equal(deleted.size, 2100); | |
| 262 | + | assert.ok(!deleted.has("docs/kept")); | |
| 263 | + | assert.equal(db.raw.prepare("SELECT id FROM files").get().id, "kept"); | |
| 264 | + | }); |
| 1 | + | # Personal-data erasure | |
| 2 | + | ||
| 3 | + | Owner: Flagon support (`support@g1t.sh`). Check **sudo → Customers → Privacy** and the support inbox each business day. The sidebar counts open requests. Check failed cleanup jobs even when there are no open requests. An empty queue is not proof that an erasure request has been fulfilled. | |
| 4 | + | ||
| 5 | + | ## Receive and assess | |
| 6 | + | ||
| 7 | + | Settings → Account accepts a request independently of account closure, sole workspace ownership and outstanding billing. Intake does not delete anything. Repeated submissions keep the original deadline. Staff can record requests received elsewhere with their original receipt time; recording one for a person who already has an open case adds the notes to that case's trail and can only move its deadline earlier. Each case shows its full trail (intake, reviews, refusal or completion) in sudo. A request about a person without an account belongs in the restricted support case system; it must not be rejected because no username exists. | |
| 8 | + | ||
| 9 | + | Respond without undue delay, ordinarily within one calendar month of receipt. The application clamps the corresponding day to the last day of the next month. It does not automatically grant extensions or reset the deadline during verification. If an extension is legally justified, explain it to the requester within the original period and record the reason and revised date in the review. Ask only for proportionate identity evidence; use the existing authenticated session or verified address where sufficient. Never request passwords, recovery codes, or unnecessary identity documents. | |
| 10 | + | ||
| 11 | + | Record the requested scope, identity verification, affected account/workspace/repository IDs, and whether Flagon is controller or processing workspace content for a customer. Contact the responsible workspace controller where needed. Ownership and unpaid bills are operational matters to resolve, not blanket grounds to reject a personal-data request. Transfer an organization's ownership appropriately; assess which records are necessary for tax/accounting or legal claims, with a reason, restricted access and review/deletion date for each retained category. Do not delete another person's workspace or shared work simply because the requester authored it. | |
| 12 | + | ||
| 13 | + | ## Execute and verify | |
| 14 | + | ||
| 15 | + | For approved account erasure, use the existing staff account-deletion controls after the workspace disposition has been resolved, then **Deleted accounts → Purge** promptly. Do not impose the ordinary 30-day recovery delay on an approved erasure request. Workspace purge is separate. If closure is blocked by billing, resolve the retained billing records and workspace disposition with the operator; do not close the privacy case or insist that payment is a condition of considering erasure. | |
| 16 | + | ||
| 17 | + | Identity commits five jobs atomically with a purge: avatar cleanup, Chat, Notify, Artifacts, and event publication. A restore that won the transaction prevents both deletion and scheduling. Jobs run every 15 minutes, in batches, and remain pending on error. Error retries back off to at most 16 hours; large successful batches continue at the next sweep. Repair the failing service/binding and verify the next retry. Do not manually mark failed jobs completed. Successful jobs remove their payload and retain a minimal receipt. | |
| 18 | + | ||
| 19 | + | Automatic cleanup covers: | |
| 20 | + | ||
| 21 | + | - Identity's account record, emails, credentials and personal settings; the avatar is removed when no account/workspace still references it. | |
| 22 | + | - Chat memberships, reactions and join state; the person's private conversation with g1t, including AI answers and pending jobs, is removed. Authorship in shared conversations becomes `ghost`; the shared conversation itself, and other members' pending answers, are untouched. A workspace purge removes its channels, messages (in chunks, over several retries for a large workspace), settings and unreferenced emoji files, and tombstones reject late channels, members, messages, reactions, settings, emoji and meter rows. | |
| 23 | + | - The person's notification feed, browser push subscription credentials, Expo device tokens, visit state and known presence/topic registrations. A permanent tombstone makes later deliveries to the erased feed no-ops (dropped, not refused). Presence rooms recorded before this implementation may include historical memberships no longer discoverable from a feed: check those rooms from the case's membership inventory as well. | |
| 24 | + | - For a workspace purge, Notify clears the workspace from the feeds of the members identity held at purge time and of everyone its presence rooms know, under its current slug and any former slug no live workspace holds. The person whose username it is may make the slug again, so these marks are times, not bans: notifications from before the purge are dropped, and counts, seeds and visits (which carry no time) are refused for 24 hours. **Not reached:** former members and non-members (watchers, people mentioned on public repositories) who were notified under the workspace's name. Review these where the case requires it. | |
| 25 | + | - Unshared private folios (in any workspace), their live room, files, versions, proposals, text search and vector records. Personal visits, favorites, access requests, memberships, mention lists and authorship fields (including cleared-by and repository-space added-by) are removed or replaced in shared artifacts. Workspace purge also removes pages, spaces, repository-doc copies, uploads and embedding state, a page at a time. Failures preserve the SQL references needed for retry. An index run that races an erasure writes no search rows and takes back the vectors it wrote. | |
| 26 | + | - **Not covered automatically in Artifacts:** comment threads and their authors inside a shared document's live state and its saved version snapshots; pages in a private space (kept and ghosted, unlike private folios); live documents of pages deleted before this change (ordinary page deletion now destroys them); rooms beyond the first 200 of an emptied folio trash; large folio version blobs (`docs/versions/<folio>/…`) and upload blobs of folios deleted earlier; vectors whose deletion failed during an earlier ordinary deletion. Treat these as manual review items. | |
| 27 | + | - Publication of `user.deleted` / `workspace.deleted` to existing lifecycle consumers. **Publication is not a consumer acknowledgement.** Check the bus and subscriber dead-letter queues and verify the resulting records in Work, Repos, Projects, Integrations, Webhooks, Actions, Deployments, Billing, Security, Search, Packages and the inbox. Retry failed lifecycle work and verify its effect before recording completion. | |
| 28 | + | ||
| 29 | + | The automatic path does **not** discover every occurrence of a person's information inside free text, git commits, collaborative document state/history, attachments, AI output or another user's notifications. Review these explicitly. Replacing a username with `ghost` is not proof of anonymisation. Use a scoped, reviewed data correction/removal for content that must be erased, covering current content, revision/Yjs state, exported blobs, FTS, vectors and caches. For git history, assess authorship, legal basis and downstream clones before a coordinated history rewrite. Record any permitted retention and explain it to the requester. | |
| 30 | + | ||
| 31 | + | Review copies held by Cloudflare logging/storage, HeyCatch, analytics, Google Workspace mail, Expo and any workspace-configured AI/integration providers as applicable. Removing a push token prevents future deliveries; it does not recall messages already delivered. Contact recipients where required, obtain deletion confirmations or document their bounded retention/exception, and record reference IDs rather than sensitive response bodies. No provider request is sent automatically by this code. | |
| 32 | + | ||
| 33 | + | ## Backups, restores and closure | |
| 34 | + | ||
| 35 | + | Before completion, export a minimal restricted deletion ledger outside the databases that might be restored: subject IDs, affected resources, deletion date, required re-erasure actions and evidence references. Include deletions that predate the new job table. Do not put credentials or copies of erased content in that ledger. | |
| 36 | + | ||
| 37 | + | For each backup/versioned store, record its actual configured retention and expiry, who can access it, and how it is kept beyond ordinary use. Inspect D1 Time Travel, Durable Object backups, object-store versions, exports, developer copies and provider logs; do not assume they all have D1's schedule. Explain residual backup retention in the response. | |
| 38 | + | ||
| 39 | + | Restore procedure: keep the restored service isolated from public traffic and outbound notifications/jobs; replay the deletion ledger against restored identity data, all service databases, live-object state, files, indexes and cached copies; verify the subjects cannot authenticate and deleted content is inaccessible; then enable traffic. Restoring an older identity database also restores older outbox state, so that outbox cannot be the sole source of the replay ledger. Test this procedure on disposable fixtures before using it for recovery. | |
| 40 | + | ||
| 41 | + | Send the requester a clear outcome: what was erased, what remains and why, the retention/review dates, backup treatment, and relevant complaint/remedy information if any part is refused. Record the response reference. The staff completion form requires evidence for identity, downstream services, shared content, processors, backups, response and retention; it also refuses completion while the account or pending cleanup jobs remain. These records support an operator's assessment; the application does not certify legal compliance. Keep case notes minimal and review their necessity with the retention schedule. | |
| 42 | + | ||
| 43 | + | ## Deployment and validation | |
| 44 | + | ||
| 45 | + | Apply the identity `0049_privacy`, Chat `0006_erasure`, and Artifacts `0010_erasure` migrations before deploying code that uses them. Deploy Chat, Notify and Artifacts before Identity, then the web and sudo apps. Identity needs the new CHAT, NOTIFY and ARTIFACTS service bindings. A transient mismatch leaves jobs pending and retryable. These changes have no production effect until deployed. | |
| 46 | + | ||
| 47 | + | Validate on disposable staging accounts: one with shared work, a private g1t DM, a private folio with an upload, browser/mobile subscriptions and presence; keep another account's comparable data as a control. Submit a request, verify its original deadline survives a duplicate, exercise a storage/service failure, purge, and inspect all receipts and stores. Exercise a delayed message/job after erasure and a restore followed by ledger replay. Check legacy deletion tombstones separately: these changes do not automatically reconstruct cleanup references lost by previous purges. | |
| 48 | + | ||
| 49 | + | Local fixture tests: `cargo test -p g1t-identity --lib` and `node --test --test-isolation=none conformance/erasure.test.mjs`. These verify code behavior, not production provider configuration or completion of a real person's request. | |
| 50 | + | ||
| 51 | + | Legal references: [GDPR Articles 12, 17 and 19](https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng) and [ICO erasure guidance](https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/individual-rights/individual-rights/right-to-erasure/). Check applicable law and current guidance when assessing a case. |
| 57 | 57 | export * from "./people"; | |
| 58 | 58 | export * from "./webhooks"; | |
| 59 | 59 | export * from "./work"; | |
| 60 | + | ||
| 61 | + | export * from "./privacy"; |
| 1 | + | import type { ServiceBinding } from "./clients"; | |
| 2 | + | import type { User } from "./identity"; | |
| 3 | + | import type { Result } from "./result"; | |
| 4 | + | ||
| 5 | + | export type ErasureStatus = { id: string; requested_at: string; due_at: string; status: "received" | "reviewing" | "completed" | "refused" }; | |
| 6 | + | export type ErasureRequest = ErasureStatus & { user_id: string; username: string; scope: string; evidence: string; updated_at: string; updated_by: string | null; closed_at: string | null; reviews: ErasureReview[] }; | |
| 7 | + | /** One entry in a case's trail: intake notes, a review, a refusal or the completion evidence. Staff only. */ | |
| 8 | + | export type ErasureReview = { at: string; staff: string; status: string; evidence: string }; | |
| 9 | + | export type PurgeJob = { subject_id: string; kind: string; target: string; attempts: number; retry_at: string; last_error: string | null; created_at: string }; | |
| 10 | + | ||
| 11 | + | export function privacyClient(service: ServiceBinding) { | |
| 12 | + | async function rpc<T>(method: string, args: object = {}): Promise<T> { | |
| 13 | + | const response = await service.fetch(`https://service/rpc/${method}`, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(args) }); | |
| 14 | + | if (!response.ok) throw new Error("Privacy service unavailable"); | |
| 15 | + | return response.json() as Promise<T>; | |
| 16 | + | } | |
| 17 | + | return { | |
| 18 | + | request: (user: User) => rpc<Result<ErasureStatus>>("request_erasure", { user }), | |
| 19 | + | mine: (user: User) => rpc<ErasureStatus | null>("my_erasure", { user }), | |
| 20 | + | intake: (username: string, received_at: string, staff: string, evidence: string) => rpc<Result<boolean>>("admin_intake_erasure", { username, received_at, staff, evidence }), | |
| 21 | + | requests: () => rpc<ErasureRequest[]>("admin_erasure_requests"), | |
| 22 | + | pending: () => rpc<PurgeJob[]>("admin_pending_purges"), | |
| 23 | + | review: (id: string, staff: string, status: string, evidence: string) => rpc<Result<boolean>>("admin_review_erasure", { id, staff, status, evidence }), | |
| 24 | + | }; | |
| 25 | + | } |
| 1 | + | CREATE TABLE erased_subjects (id TEXT PRIMARY KEY, kind TEXT NOT NULL); | |
| 2 | + | CREATE TABLE purge_documents (subject_id TEXT NOT NULL, kind TEXT NOT NULL, document_id TEXT NOT NULL, PRIMARY KEY(subject_id, kind, document_id)); | |
| 3 | + | ||
| 4 | + | CREATE TABLE erased_documents (id TEXT PRIMARY KEY, kind TEXT NOT NULL); | |
| 5 | + | CREATE TRIGGER erased_spaces BEFORE INSERT ON spaces WHEN EXISTS ( | |
| 6 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 7 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 8 | + | CREATE TRIGGER erased_pages BEFORE INSERT ON pages WHEN EXISTS ( | |
| 9 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 10 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 11 | + | CREATE TRIGGER erased_folios BEFORE INSERT ON folios WHEN EXISTS ( | |
| 12 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 13 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 14 | + | CREATE TRIGGER erased_files BEFORE INSERT ON files WHEN EXISTS ( | |
| 15 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 16 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 17 | + | CREATE TRIGGER erased_folio_files BEFORE INSERT ON folio_files WHEN EXISTS ( | |
| 18 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 19 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 20 | + | CREATE TRIGGER erased_templates BEFORE INSERT ON templates WHEN EXISTS ( | |
| 21 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 22 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 23 | + | CREATE TRIGGER erased_folio_templates BEFORE INSERT ON folio_templates WHEN EXISTS ( | |
| 24 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 25 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 26 | + | CREATE TRIGGER erased_doc_chunks BEFORE INSERT ON doc_chunks WHEN EXISTS ( | |
| 27 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 28 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 29 | + | CREATE TRIGGER erased_folio_chunks BEFORE INSERT ON folio_chunks WHEN EXISTS ( | |
| 30 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 31 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 32 | + | CREATE TRIGGER erased_repo_spaces BEFORE INSERT ON repo_spaces WHEN EXISTS ( | |
| 33 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 34 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 35 | + | CREATE TRIGGER erased_doc_embed_usage BEFORE INSERT ON doc_embed_usage WHEN EXISTS ( | |
| 36 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 37 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 38 | + | CREATE TRIGGER erased_doc_index_runs BEFORE INSERT ON doc_index_runs WHEN EXISTS ( | |
| 39 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 40 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 41 | + | CREATE TRIGGER erased_workspace_billing BEFORE INSERT ON workspace_billing WHEN EXISTS ( | |
| 42 | + | SELECT 1 FROM erased_subjects WHERE kind = 'workspace' AND id = NEW.workspace_id | |
| 43 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 44 | + | CREATE TRIGGER erased_document_files BEFORE INSERT ON files WHEN EXISTS ( | |
| 45 | + | SELECT 1 FROM erased_documents WHERE id = NEW.page_id AND kind = 'page' | |
| 46 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 47 | + | CREATE TRIGGER erased_document_folio_files BEFORE INSERT ON folio_files WHEN EXISTS ( | |
| 48 | + | SELECT 1 FROM erased_documents WHERE id = NEW.folio_id AND kind = 'folio' | |
| 49 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 50 | + | CREATE TRIGGER erased_document_doc_chunks BEFORE INSERT ON doc_chunks WHEN EXISTS ( | |
| 51 | + | SELECT 1 FROM erased_documents WHERE id = NEW.page_id AND kind = 'page' | |
| 52 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 53 | + | CREATE TRIGGER erased_document_folio_chunks BEFORE INSERT ON folio_chunks WHEN EXISTS ( | |
| 54 | + | SELECT 1 FROM erased_documents WHERE id = NEW.folio_id AND kind = 'folio' | |
| 55 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 56 | + | ||
| 57 | + | CREATE TABLE purge_access (subject_id TEXT NOT NULL, kind TEXT NOT NULL, document_id TEXT NOT NULL, done INTEGER NOT NULL DEFAULT 0, PRIMARY KEY(subject_id, kind, document_id)); | |
| 58 | + | CREATE TRIGGER erased_author_spaces BEFORE INSERT ON spaces WHEN EXISTS ( | |
| 59 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 60 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 61 | + | CREATE TRIGGER erased_author_pages BEFORE INSERT ON pages WHEN EXISTS ( | |
| 62 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 63 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 64 | + | CREATE TRIGGER erased_author_folios BEFORE INSERT ON folios WHEN EXISTS ( | |
| 65 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 66 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 67 | + | CREATE TRIGGER erased_author_files BEFORE INSERT ON files WHEN EXISTS ( | |
| 68 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 69 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 70 | + | CREATE TRIGGER erased_author_folio_files BEFORE INSERT ON folio_files WHEN EXISTS ( | |
| 71 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 72 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 73 | + | CREATE TRIGGER erased_author_templates BEFORE INSERT ON templates WHEN EXISTS ( | |
| 74 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 75 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 76 | + | CREATE TRIGGER erased_author_folio_templates BEFORE INSERT ON folio_templates WHEN EXISTS ( | |
| 77 | + | SELECT 1 FROM erased_subjects WHERE kind = 'user' AND 'user:' || id = NEW.created_by | |
| 78 | + | ) BEGIN SELECT RAISE(IGNORE); END; |
| 1 | + | import { GHOST_ID } from "@g1t/contracts"; | |
| 2 | + | ||
| 3 | + | import { cloudflareVectors } from "./vectors.ts"; | |
| 4 | + | import { fileStore, type FileStoreEnv } from "./files.ts"; | |
| 5 | + | import type { IndexEnv } from "./indexer.ts"; | |
| 6 | + | import type { PageRoom } from "./room.ts"; | |
| 7 | + | import type { FolioRoom } from "./folios/room.ts"; | |
| 8 | + | ||
| 9 | + | type Env = IndexEnv & FileStoreEnv & { PAGES?: DurableObjectNamespace<PageRoom>; FOLIOS?: DurableObjectNamespace<FolioRoom> }; | |
| 10 | + | type Subject = { kind: "user" | "workspace"; id: string; username?: string }; | |
| 11 | + | const GHOST = `user:${GHOST_ID}`; | |
| 12 | + | /** Rows (chunks or files) a workspace sweep takes per pass; each pass deletes what it read, so a retry resumes. */ | |
| 13 | + | const PAGE = 200; | |
| 14 | + | /** Documents destroyed per call; the rest wait for identity's next retry. */ | |
| 15 | + | const DOCUMENTS = 25; | |
| 16 | + | /** Sweep passes per call, so a large workspace stays within one invocation's subrequests. */ | |
| 17 | + | const PASSES = 10; | |
| 18 | + | ||
| 19 | + | /** | |
| 20 | + | * Called only by identity's durable purge jobs. Failures leave the SQL | |
| 21 | + | * references in place, every removal is idempotent, and work left over | |
| 22 | + | * answers `pending`, so identity retries rather than recording success. | |
| 23 | + | */ | |
| 24 | + | export async function purgePersonalData(env: Env, args: Subject): Promise<{ ok: boolean; pending?: boolean }> { | |
| 25 | + | if (!args?.id || !["user", "workspace"].includes(args.kind)) throw new Error("Invalid purge subject"); | |
| 26 | + | const db = env.DB, key = `user:${args.id}`; | |
| 27 | + | await db.prepare("INSERT OR IGNORE INTO erased_subjects VALUES (?, ?)").bind(args.id, args.kind).run(); | |
| 28 | + | if (args.kind === "workspace") { | |
| 29 | + | await db.batch([ | |
| 30 | + | db.prepare("INSERT OR IGNORE INTO purge_documents SELECT ?, 'page', id FROM pages WHERE workspace_id = ?").bind(args.id, args.id), | |
| 31 | + | db.prepare("INSERT OR IGNORE INTO purge_documents SELECT ?, 'folio', id FROM folios WHERE workspace_id = ?").bind(args.id, args.id), | |
| 32 | + | ]); | |
| 33 | + | } else { | |
| 34 | + | await db.batch([ | |
| 35 | + | db.prepare("INSERT OR IGNORE INTO purge_access (subject_id, kind, document_id) SELECT ?1, 'page', id FROM pages WHERE created_by = ?2 OR updated_by = ?2 OR id IN (SELECT page_id FROM page_views WHERE user_id = ?1)").bind(args.id, key), | |
| 36 | + | db.prepare("INSERT OR IGNORE INTO purge_access (subject_id, kind, document_id) SELECT ?1, 'folio', id FROM folios WHERE owner = ?2 OR created_by = ?2 OR updated_by = ?2 OR id IN (SELECT folio_id FROM folio_visits WHERE user_id = ?1)").bind(args.id, key), | |
| 37 | + | ]); | |
| 38 | + | const access = (await db.prepare("SELECT kind, document_id FROM purge_access WHERE subject_id = ? AND done = 0 LIMIT 50").bind(args.id).all<{ kind: string; document_id: string }>()).results; | |
| 39 | + | for (const doc of access) { | |
| 40 | + | const rooms = doc.kind === "folio" ? env.FOLIOS : env.PAGES; | |
| 41 | + | if (!rooms) throw new Error("Document access cleanup unavailable"); | |
| 42 | + | await rooms.get(rooms.idFromName(doc.document_id)).forgetPerson(key, args.username ?? ""); | |
| 43 | + | await db.prepare("UPDATE purge_access SET done = 1 WHERE subject_id = ? AND kind = ? AND document_id = ?").bind(args.id, doc.kind, doc.document_id).run(); | |
| 44 | + | } | |
| 45 | + | if (await db.prepare("SELECT 1 FROM purge_access WHERE subject_id = ? AND done = 0 LIMIT 1").bind(args.id).first()) return { ok: false, pending: true }; | |
| 46 | + | // Only private folios nobody else can open, in any workspace: the folio | |
| 47 | + | // and everything under it the person's own, with no other principal's | |
| 48 | + | // access or grant. Shared material needs a case-specific assessment. | |
| 49 | + | await db.prepare(`INSERT OR IGNORE INTO purge_documents SELECT ?1, 'folio', f.id FROM folios f | |
| 50 | + | WHERE f.owner = ?2 AND f.space_id IS NULL AND f.general_access = 'none' | |
| 51 | + | AND NOT EXISTS (SELECT 1 FROM folios child WHERE substr(child.path, 1, length(f.path)) = f.path AND ( | |
| 52 | + | child.owner != ?2 OR child.general_access != 'none' | |
| 53 | + | OR EXISTS (SELECT 1 FROM folio_access a WHERE a.folio_id = child.id AND a.principal != ?2) | |
| 54 | + | OR EXISTS (SELECT 1 FROM folio_grants g WHERE g.folio_id = child.id AND g.principal != ?2)))`).bind(args.id, key).run(); | |
| 55 | + | } | |
| 56 | + | const docs = (await db.prepare("SELECT kind, document_id FROM purge_documents WHERE subject_id = ? LIMIT ?").bind(args.id, DOCUMENTS).all<{ kind: string; document_id: string }>()).results; | |
| 57 | + | for (const doc of docs) { | |
| 58 | + | const folio = doc.kind === "folio", id = doc.document_id; | |
| 59 | + | // First, so an upload, an index run or a save racing this cannot write rows for it again (migration 0010, indexer.ts). | |
| 60 | + | await db.prepare("INSERT OR IGNORE INTO erased_documents (id, kind) VALUES (?, ?)").bind(id, doc.kind).run(); | |
| 61 | + | const rooms = folio ? env.FOLIOS : env.PAGES; | |
| 62 | + | if (!rooms) throw new Error("Document room cleanup unavailable"); | |
| 63 | + | await rooms.get(rooms.idFromName(id)).destroy(); | |
| 64 | + | const chunks = (await db.prepare(`SELECT id FROM ${folio ? "folio_chunks" : "doc_chunks"} WHERE ${folio ? "folio_id" : "page_id"} = ?`).bind(id).all<{ id: string }>()).results.map(r => r.id); | |
| 65 | + | const vectors = folio ? env.FOLIO_VECTORS : env.VECTORS; | |
| 66 | + | if (vectors && chunks.length) await cloudflareVectors(vectors).delete(chunks); | |
| 67 | + | const files = (await db.prepare(`SELECT key FROM ${folio ? "folio_files" : "files"} WHERE ${folio ? "folio_id" : "page_id"} = ?`).bind(id).all<{ key: string }>()).results; | |
| 68 | + | for (const { key: file } of files) await fileStore(env).delete(`docs/${file}`); | |
| 69 | + | if (folio) { | |
| 70 | + | const states = (await db.prepare("SELECT state_key AS key FROM folio_versions WHERE folio_id = ?1 AND state_key IS NOT NULL UNION SELECT update_key AS key FROM folio_proposals WHERE folio_id = ?1 AND update_key IS NOT NULL").bind(id).all<{ key: string }>()).results; | |
| 71 | + | for (const { key: file } of states) await fileStore(env).delete(file); | |
| 72 | + | } | |
| 73 | + | const table = folio ? "folios" : "pages", column = folio ? "folio_id" : "page_id"; | |
| 74 | + | await db.batch([ | |
| 75 | + | // By document, so passages a late index run wrote without a chunk row go too. | |
| 76 | + | db.prepare(`DELETE FROM ${folio ? "folio_chunks_fts WHERE folio_id" : "doc_chunks_fts WHERE doc_id"} = ?`).bind(id), | |
| 77 | + | db.prepare(`DELETE FROM ${folio ? "folio_chunks" : "doc_chunks"} WHERE ${column} = ?`).bind(id), | |
| 78 | + | db.prepare(`DELETE FROM ${folio ? "folios_fts" : "pages_fts"} WHERE ${column} = ?`).bind(id), | |
| 79 | + | db.prepare(`DELETE FROM ${folio ? "folio_files" : "files"} WHERE ${column} = ?`).bind(id), | |
| 80 | + | db.prepare(`UPDATE ${table} SET parent_id = NULL WHERE parent_id = ?`).bind(id), | |
| 81 | + | db.prepare(`DELETE FROM ${table} WHERE id = ?`).bind(id), | |
| 82 | + | db.prepare("DELETE FROM purge_documents WHERE subject_id = ? AND kind = ? AND document_id = ?").bind(args.id, doc.kind, id), | |
| 83 | + | ]); | |
| 84 | + | } | |
| 85 | + | if (await db.prepare("SELECT 1 FROM purge_documents WHERE subject_id = ? LIMIT 1").bind(args.id).first()) return { ok: false, pending: true }; | |
| 86 | + | if (args.kind === "workspace") { | |
| 87 | + | // What no document holds any more: repository docs' passages, passages | |
| 88 | + | // and uploads of documents deleted earlier. A page at a time, each | |
| 89 | + | // deleted once its vectors or blobs are, until none is left. | |
| 90 | + | let passes = PASSES; | |
| 91 | + | for (const [table, store] of [["doc_chunks", env.VECTORS], ["folio_chunks", env.FOLIO_VECTORS]] as const) { | |
| 92 | + | const fts = table === "doc_chunks" ? "doc_chunks_fts" : "folio_chunks_fts"; | |
| 93 | + | for (;;) { | |
| 94 | + | if (passes-- <= 0) return { ok: false, pending: true }; | |
| 95 | + | const ids = (await db.prepare(`SELECT id FROM ${table} WHERE workspace_id = ? LIMIT ${PAGE}`).bind(args.id).all<{ id: string }>()).results.map(r => r.id); | |
| 96 | + | if (!ids.length) break; | |
| 97 | + | if (store) await cloudflareVectors(store).delete(ids); | |
| 98 | + | await db.batch([ | |
| 99 | + | db.prepare(`DELETE FROM ${fts} WHERE chunk_id IN (SELECT value FROM json_each(?))`).bind(JSON.stringify(ids)), | |
| 100 | + | db.prepare(`DELETE FROM ${table} WHERE id IN (SELECT value FROM json_each(?))`).bind(JSON.stringify(ids)), | |
| 101 | + | ]); | |
| 102 | + | } | |
| 103 | + | } | |
| 104 | + | for (const table of ["files", "folio_files"]) { | |
| 105 | + | for (;;) { | |
| 106 | + | if (passes-- <= 0) return { ok: false, pending: true }; | |
| 107 | + | const rows = (await db.prepare(`SELECT key FROM ${table} WHERE workspace_id = ? LIMIT ${PAGE}`).bind(args.id).all<{ key: string }>()).results; | |
| 108 | + | if (!rows.length) break; | |
| 109 | + | for (const row of rows) await fileStore(env).delete(`docs/${row.key}`); | |
| 110 | + | await db.prepare(`DELETE FROM ${table} WHERE key IN (SELECT value FROM json_each(?))`).bind(JSON.stringify(rows.map(r => r.key))).run(); | |
| 111 | + | } | |
| 112 | + | } | |
| 113 | + | await db.batch([ | |
| 114 | + | db.prepare("DELETE FROM repo_files_fts WHERE space_id IN (SELECT id FROM repo_spaces WHERE workspace_id = ?)").bind(args.id), | |
| 115 | + | ...["repo_spaces", "templates", "folio_templates", "spaces", "doc_index_runs", "doc_embed_usage", "workspace_billing"].map(table => db.prepare(`DELETE FROM ${table} WHERE workspace_id = ?`).bind(args.id)), | |
| 116 | + | ]); | |
| 117 | + | } else { | |
| 118 | + | const statements: D1PreparedStatement[] = []; | |
| 119 | + | for (const table of ["page_views", "favorites", "folio_visits", "folio_favorites", "folio_access_requests", "space_joins"]) statements.push(db.prepare(`DELETE FROM ${table} WHERE user_id = ?`).bind(args.id)); | |
| 120 | + | for (const table of ["space_members", "page_owners", "folio_grants", "folio_access"]) statements.push(db.prepare(`DELETE FROM ${table} WHERE principal = ?`).bind(key)); | |
| 121 | + | const authors: Record<string, string[]> = { | |
| 122 | + | spaces: ["created_by", "deleted_by"], pages: ["created_by", "updated_by", "archived_by"], | |
| 123 | + | folios: ["owner", "created_by", "updated_by", "edited_by", "trashed_by"], | |
| 124 | + | templates: ["created_by"], folio_templates: ["created_by"], files: ["created_by"], folio_files: ["created_by"], | |
| 125 | + | suggestions: ["author", "asked_by", "decided_by"], folio_suggestions: ["author", "asked_by", "decided_by"], folio_proposals: ["author", "asked_by", "decided_by"], | |
| 126 | + | space_members: ["added_by"], folio_grants: ["granted_by"], repo_spaces: ["added_by"], | |
| 127 | + | page_changes: ["cleared_by"], folio_changes: ["cleared_by"], | |
| 128 | + | }; | |
| 129 | + | for (const [table, columns] of Object.entries(authors)) for (const column of columns) statements.push(db.prepare(`UPDATE ${table} SET ${column} = ? WHERE ${column} = ?`).bind(GHOST, key)); | |
| 130 | + | // The cheap LIKE first, so json_each runs only on the versions that name them. | |
| 131 | + | for (const table of ["page_versions", "folio_versions"]) statements.push(db.prepare(`UPDATE ${table} SET authors = (SELECT json_group_array(CASE WHEN value = ?1 THEN ?2 ELSE value END) FROM json_each(${table}.authors)) WHERE authors LIKE '%' || ?1 || '%' AND EXISTS (SELECT 1 FROM json_each(${table}.authors) WHERE value = ?1)`).bind(key, GHOST)); | |
| 132 | + | // Who was already told of a mention, by username. | |
| 133 | + | const username = (args.username ?? "").toLowerCase(); | |
| 134 | + | if (username) for (const table of ["pages", "folios"]) statements.push(db.prepare(`UPDATE ${table} SET mentioned = (SELECT json_group_array(value) FROM json_each(${table}.mentioned) WHERE value != ?1) WHERE mentioned LIKE '%' || ?1 || '%' AND EXISTS (SELECT 1 FROM json_each(${table}.mentioned) WHERE value = ?1)`).bind(username)); | |
| 135 | + | for (let i = 0; i < statements.length; i += 80) await db.batch(statements.slice(i, i + 80)); | |
| 136 | + | } | |
| 137 | + | await db.prepare("DELETE FROM purge_access WHERE subject_id = ?").bind(args.id).run(); | |
| 138 | + | return { ok: true }; | |
| 139 | + | } |
| 77 | 77 | ||
| 78 | 78 | export class FolioRoom extends DurableObject<FolioRoomEnv> { | |
| 79 | 79 | private doc: Y.Doc | null = null; | |
| 80 | + | /** Saves and index runs in flight: destroy() waits for them, so none writes after it. */ | |
| 81 | + | private saves = new Set<Promise<unknown>>(); | |
| 80 | 82 | /** The last awareness update each client sent. Lost on hibernation; clients resend every 15 s. */ | |
| 81 | 83 | private awareness = new Map<number, Uint8Array>(); | |
| 82 | 84 | private clocks = new Map<number, number>(); | |
| ⋯ | |||
| 104 | 106 | } | |
| 105 | 107 | ||
| 106 | 108 | private setMeta(key: string, value: unknown): void { | |
| 109 | + | if (key !== "erased" && this.meta("erased", false)) return; | |
| 107 | 110 | this.ctx.storage.sql.exec("INSERT INTO meta (key, value) VALUES (?, ?) ON CONFLICT (key) DO UPDATE SET value = excluded.value", key, JSON.stringify(value)); | |
| 108 | 111 | } | |
| 109 | 112 | ||
| ⋯ | |||
| 114 | 117 | return model; | |
| 115 | 118 | } | |
| 116 | 119 | ||
| 120 | + | async forgetPerson(key: string, username: string): Promise<void> { | |
| 121 | + | if (this.meta("erased", false)) return; | |
| 122 | + | this.setMeta(`erased:${key}`, true); | |
| 123 | + | await this.setRole(key, null); | |
| 124 | + | await Promise.allSettled([...this.saves]); | |
| 125 | + | for (const field of ["editors", "pending_authors"]) this.setMeta(field, this.meta<string[]>(field, []).filter(value => value !== key)); | |
| 126 | + | this.setMeta("editor_names", this.meta<string[]>("editor_names", []).filter(value => value.toLowerCase() !== username.toLowerCase())); | |
| 127 | + | } | |
| 128 | + | ||
| 129 | + | async destroy(): Promise<void> { | |
| 130 | + | // Synchronous tombstone comes first; no reentrant save can recreate state. | |
| 131 | + | this.setMeta("erased", true); | |
| 132 | + | // Open tabs hear the folio is gone, as before tombstones (4410, folio.trashed). | |
| 133 | + | await this.closeAll("Deleted"); | |
| 134 | + | await Promise.allSettled([...this.saves]); | |
| 135 | + | this.doc?.destroy(); | |
| 136 | + | this.doc = null; | |
| 137 | + | this.awareness.clear(); | |
| 138 | + | this.clocks.clear(); | |
| 139 | + | this.ctx.storage.transactionSync(() => { | |
| 140 | + | this.ctx.storage.sql.exec("DELETE FROM updates"); | |
| 141 | + | this.ctx.storage.sql.exec("DELETE FROM snapshot"); | |
| 142 | + | this.ctx.storage.sql.exec("DELETE FROM meta WHERE key != 'erased'"); | |
| 143 | + | }); | |
| 144 | + | await this.ctx.storage.deleteAlarm(); | |
| 145 | + | } | |
| 146 | + | ||
| 117 | 147 | private load(): Y.Doc { | |
| 148 | + | if (this.meta("erased", false)) throw new Error("Document erased"); | |
| 118 | 149 | if (this.doc) return this.doc; | |
| 119 | 150 | const doc = new Y.Doc({ gc: true }); | |
| 120 | 151 | const snapshot = this.ctx.storage.sql.exec<{ data: ArrayBuffer }>("SELECT data FROM snapshot WHERE id = 1").toArray()[0]; | |
| ⋯ | |||
| 128 | 159 | } | |
| 129 | 160 | ||
| 130 | 161 | private onUpdate(update: Uint8Array, origin: unknown): void { | |
| 162 | + | if (this.meta("erased", false)) return; | |
| 131 | 163 | this.ctx.storage.sql.exec("INSERT INTO updates (data) VALUES (?)", update); | |
| 132 | 164 | const count = this.ctx.storage.sql.exec<{ n: number }>("SELECT COUNT(*) AS n FROM updates").one().n; | |
| 133 | 165 | if (count >= COMPACT_AT) this.compact(); | |
| ⋯ | |||
| 162 | 194 | ||
| 163 | 195 | private alarmBy(when: number): void { | |
| 164 | 196 | void this.ctx.storage.getAlarm().then((at) => { | |
| 197 | + | if (this.meta("erased", false)) return; | |
| 165 | 198 | if (at == null || at > when) return this.ctx.storage.setAlarm(when); | |
| 166 | 199 | }); | |
| 167 | 200 | } | |
| ⋯ | |||
| 179 | 212 | ||
| 180 | 213 | /** Saves to D1 now: the rendition, search, links, citations, and a version if one is due or asked for. */ | |
| 181 | 214 | private async persist(version: FolioOrigin | null = null): Promise<string | null> { | |
| 215 | + | if (this.meta("erased", false)) return null; | |
| 216 | + | const saving = this.save(version); | |
| 217 | + | this.saves.add(saving); | |
| 218 | + | try { return await saving; } finally { this.saves.delete(saving); } | |
| 219 | + | } | |
| 220 | + | ||
| 221 | + | private async save(version: FolioOrigin | null = null): Promise<string | null> { | |
| 182 | 222 | const folioId = this.meta<string | null>("folio_id", null); | |
| 183 | 223 | if (!folioId || !kindModel(this.meta<string | null>("kind", null))) return null; | |
| 184 | 224 | const doc = this.load(); | |
| ⋯ | |||
| 216 | 256 | } | |
| 217 | 257 | ||
| 218 | 258 | override async alarm(): Promise<void> { | |
| 259 | + | if (this.meta("erased", false)) return; | |
| 219 | 260 | await this.persist(); | |
| 220 | 261 | const due = this.meta<number | null>("index_at", null); | |
| 221 | 262 | if (due == null) return; | |
| 222 | 263 | if (Date.now() < due) return this.alarmBy(due); | |
| 223 | 264 | this.setMeta("index_at", null); | |
| 224 | 265 | const folioId = this.meta<string | null>("folio_id", null); | |
| 225 | − | if (folioId) await indexFolio(this.env, folioId); | |
| 266 | + | if (folioId) { | |
| 267 | + | const indexing = indexFolio(this.env, folioId); | |
| 268 | + | this.saves.add(indexing); | |
| 269 | + | try { await indexing; } finally { this.saves.delete(indexing); } | |
| 270 | + | } | |
| 226 | 271 | } | |
| 227 | 272 | ||
| 228 | 273 | // ── Calls from the Worker ────────────────────────────────────────────── | |
| ⋯ | |||
| 376 | 421 | } | |
| 377 | 422 | ||
| 378 | 423 | /** Forgets everything: the folio was deleted for good. */ | |
| 379 | − | async destroy(): Promise<void> { | |
| 380 | − | await this.closeAll("Deleted"); | |
| 381 | − | this.doc = null; | |
| 382 | − | await this.ctx.storage.deleteAlarm(); | |
| 383 | − | await this.ctx.storage.deleteAll(); | |
| 384 | − | } | |
| 385 | 424 | ||
| 425 | + | ||
| 386 | 426 | // ── Sockets ──────────────────────────────────────────────────────────── | |
| 387 | 427 | ||
| 388 | 428 | override async fetch(request: Request): Promise<Response> { | |
| 429 | + | if (this.meta("erased", false)) return new Response("Document erased", { status: 410 }); | |
| 389 | 430 | if (request.headers.get("upgrade")?.toLowerCase() !== "websocket") return new Response("Expected a WebSocket upgrade\n", { status: 426 }); | |
| 390 | 431 | let who: FolioRoomMember; | |
| 391 | 432 | try { | |
| ⋯ | |||
| 394 | 435 | return new Response("Missing member\n", { status: 400 }); | |
| 395 | 436 | } | |
| 396 | 437 | if (!kindModel(this.meta<string | null>("kind", null))) return new Response("Not ready\n", { status: 409 }); | |
| 438 | + | if (this.meta(`erased:${who.key}`, false)) return new Response("Account erased", { status: 410 }); | |
| 397 | 439 | if (who.workspace_slug) this.setMeta("workspace_slug", who.workspace_slug); | |
| 398 | 440 | const pair = new WebSocketPair(); | |
| 399 | 441 | const [client, server] = Object.values(pair) as [WebSocket, WebSocket]; | |
| ⋯ | |||
| 416 | 458 | override async webSocketMessage(socket: WebSocket, message: string | ArrayBuffer): Promise<void> { | |
| 417 | 459 | if (typeof message === "string") return; | |
| 418 | 460 | const who = socket.deserializeAttachment() as Attachment | null; | |
| 419 | − | if (!who) return; | |
| 461 | + | if (!who || this.meta("erased", false) || this.meta(`erased:${who.key}`, false)) return; | |
| 420 | 462 | const data = new Uint8Array(message); | |
| 421 | 463 | const decoder = decoding.createDecoder(data); | |
| 422 | 464 | const type = decoding.readVarUint(decoder); | |
| 2142 | 2142 | const key = [...crypto.getRandomValues(new Uint8Array(32))].map((b) => b.toString(16).padStart(2, "0")).join(""); | |
| 2143 | 2143 | const id = newId("fil"); | |
| 2144 | 2144 | await fileStore(this.env).put(`docs/${key}`, request.body ?? new Uint8Array(), contentType); | |
| 2145 | − | await this.db | |
| 2145 | + | try { | |
| 2146 | + | const inserted = await this.db | |
| 2146 | 2147 | .prepare("INSERT INTO folio_files (id, workspace_id, folio_id, key, name, content_type, bytes, created_by, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)") | |
| 2147 | 2148 | .bind(id, ctx.workspace.id, opened.value.row.id, key, name, contentType, bytes, ctx.key, now()) | |
| 2148 | 2149 | .run(); | |
| 2150 | + | if (Number(inserted.meta.changes) === 0) { | |
| 2151 | + | await fileStore(this.env).delete(`docs/${key}`); | |
| 2152 | + | return new Response("Upload destination erased", { status: 410 }); | |
| 2153 | + | } | |
| 2154 | + | } catch (error) { | |
| 2155 | + | await fileStore(this.env).delete(`docs/${key}`); | |
| 2156 | + | throw error; | |
| 2157 | + | } | |
| 2149 | 2158 | return Response.json(ok({ id, url: `/docs-files/${key}`, name, content_type: contentType, bytes })); | |
| 2150 | 2159 | } | |
| 2151 | 2160 |
| 1 | + | import { purgePersonalData } from "./erasure.ts"; | |
| 1 | 2 | /** | |
| 2 | 3 | * The artifacts service: a workspace's spaces and pages, their live documents, | |
| 3 | 4 | * history, comments, suggestions, templates and search | |
| ⋯ | |||
| 1390 | 1391 | const ids = descendants(all, page.id); | |
| 1391 | 1392 | await this.db.batch(ids.flatMap((id) => [this.db.prepare("DELETE FROM pages_fts WHERE page_id = ?").bind(id), this.db.prepare("DELETE FROM pages WHERE id = ?").bind(id)])); | |
| 1392 | 1393 | this.defer(forgetDocs(this.env, { page_ids: ids })); | |
| 1394 | + | // Deleted for good: its live document (text and comment threads) goes too, not only the rows. | |
| 1395 | + | // Its uploads stay: a copy of the page may still show them by URL. | |
| 1396 | + | for (const id of ids) this.defer(this.room(id).destroy().catch((error) => console.error("docs could not destroy a deleted page's room", id, String(error)))); | |
| 1393 | 1397 | return ok(true); | |
| 1394 | 1398 | } | |
| 1395 | 1399 | ||
| ⋯ | |||
| 2122 | 2126 | const key = [...random].map((b) => b.toString(16).padStart(2, "0")).join(""); | |
| 2123 | 2127 | const id = newId("fil"); | |
| 2124 | 2128 | await fileStore(this.env).put(`docs/${key}`, request.body ?? new Uint8Array(), contentType); | |
| 2125 | − | await this.db | |
| 2129 | + | try { | |
| 2130 | + | const inserted = await this.db | |
| 2126 | 2131 | .prepare("INSERT INTO files (id, workspace_id, page_id, key, name, content_type, bytes, created_by, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)") | |
| 2127 | 2132 | .bind(id, found.value.workspace.id, found.value.page.id, key, name, contentType, bytes, this.userKey(viewer!), now()) | |
| 2128 | 2133 | .run(); | |
| 2134 | + | if (Number(inserted.meta.changes) === 0) { | |
| 2135 | + | await fileStore(this.env).delete(`docs/${key}`); | |
| 2136 | + | return new Response("Upload destination erased", { status: 410 }); | |
| 2137 | + | } | |
| 2138 | + | } catch (error) { | |
| 2139 | + | await fileStore(this.env).delete(`docs/${key}`); | |
| 2140 | + | throw error; | |
| 2141 | + | } | |
| 2129 | 2142 | const file: DocFile = { id, url: `/docs-files/${key}`, name, content_type: contentType, bytes }; | |
| 2130 | 2143 | return Response.json(ok(file)); | |
| 2131 | 2144 | } | |
| ⋯ | |||
| 2249 | 2262 | const opened = openD1(env.DB, request); | |
| 2250 | 2263 | const scoped = Object.create(env, { DB: { value: opened.db } }) as Env; | |
| 2251 | 2264 | const args = (await request.json().catch(() => ({}))) as any; | |
| 2265 | + | if (match[1] === "purge_personal_data") return Response.json(await purgePersonalData(env, args)); | |
| 2252 | 2266 | try { | |
| 2253 | 2267 | // Folios first (src/folios/rpc.ts), then Docs' pages. | |
| 2254 | 2268 | const folio = folioHandler(match[1]!); | |
| 143 | 143 | at, | |
| 144 | 144 | ), | |
| 145 | 145 | db.prepare("DELETE FROM doc_chunks_fts WHERE chunk_id = ?").bind(c.id), | |
| 146 | − | db.prepare("INSERT INTO doc_chunks_fts (chunk_id, space_id, doc_id, heading, text) VALUES (?, ?, ?, ?, ?)").bind(c.id, doc.space_id, doc.doc_id, c.heading ?? "", c.text), | |
| 146 | + | // Only beside a chunk row the database kept: an erased document's insert was ignored (migration 0010). | |
| 147 | + | db.prepare("INSERT INTO doc_chunks_fts (chunk_id, space_id, doc_id, heading, text) SELECT ?1, ?2, ?3, ?4, ?5 WHERE EXISTS (SELECT 1 FROM doc_chunks WHERE id = ?1)").bind(c.id, doc.space_id, doc.doc_id, c.heading ?? "", c.text), | |
| 147 | 148 | ); | |
| 148 | 149 | } | |
| 149 | 150 | const keep = new Set(chunks.map((c) => c.id)); | |
| ⋯ | |||
| 154 | 155 | for (let i = 0; i < statements.length; i += 60) await db.batch(statements.slice(i, i + 60)); | |
| 155 | 156 | ||
| 156 | 157 | if (!store || !embedder) return { chunks: chunks.length, embedded: 0, capped: false }; | |
| 158 | + | if (await erasedSource(db, doc.kind === "page" ? "page" : null, doc.doc_id, doc.workspace_id)) return { chunks: 0, embedded: 0, capped: false }; | |
| 157 | 159 | const result = await embedChanged(db, embedder, store, doc, chunks, old, byId, now); | |
| 160 | + | await dropIfErased(db, store, doc.kind === "page" ? "page" : null, doc.doc_id, doc.workspace_id, chunks.map((c) => c.id)); | |
| 158 | 161 | // Last, so a passage that moved from a place now gone could take its vector first. | |
| 159 | 162 | try { | |
| 160 | 163 | if (gone.some((r) => r.vector_hash)) await store.delete(gone.filter((r) => r.vector_hash).map((r) => r.id)); | |
| ⋯ | |||
| 488 | 491 | ) | |
| 489 | 492 | .bind(c.id, row.workspace_id, row.id, row.kind, scope, c.seq, c.heading, c.text, c.hash, at), | |
| 490 | 493 | db.prepare("DELETE FROM folio_chunks_fts WHERE chunk_id = ?").bind(c.id), | |
| 491 | − | db.prepare("INSERT INTO folio_chunks_fts (chunk_id, scope, folio_id, heading, text) VALUES (?, ?, ?, ?, ?)").bind(c.id, scope, row.id, c.heading ?? "", c.text), | |
| 494 | + | db.prepare("INSERT INTO folio_chunks_fts (chunk_id, scope, folio_id, heading, text) SELECT ?1, ?2, ?3, ?4, ?5 WHERE EXISTS (SELECT 1 FROM folio_chunks WHERE id = ?1)").bind(c.id, scope, row.id, c.heading ?? "", c.text), | |
| 492 | 495 | ); | |
| 493 | 496 | } | |
| 494 | 497 | const keep = new Set(chunks.map((c) => c.id)); | |
| ⋯ | |||
| 498 | 501 | ||
| 499 | 502 | const { embedder, store } = folioAdapters(env); | |
| 500 | 503 | if (!embedder || !store) return { capped: false }; | |
| 504 | + | if (await erasedSource(db, "folio", row.id, row.workspace_id)) return { capped: false }; | |
| 501 | 505 | // What needs a vector written: text the index doesn't hold under this id, or a new scope. | |
| 502 | 506 | const stale = chunks.filter((c) => { | |
| 503 | 507 | const was = byId.get(c.id); | |
| ⋯ | |||
| 536 | 540 | ); | |
| 537 | 541 | } | |
| 538 | 542 | if (vectors.length) await store.upsert(vectors); | |
| 543 | + | if (vectors.length) await dropIfErased(db, store, "folio", row.id, row.workspace_id, vectors.map((v) => v.id)); | |
| 539 | 544 | const hashOf = new Map(chunks.map((c) => [c.id, c.hash])); | |
| 540 | 545 | for (const v of vectors) done.push({ id: v.id, hash: hashOf.get(v.id)! }); | |
| 541 | 546 | } catch (error) { | |
| ⋯ | |||
| 587 | 592 | console.error("folios could not forget passages", String(error)); | |
| 588 | 593 | } | |
| 589 | 594 | } | |
| 595 | + | ||
| 596 | + | /** Whether a document, or its workspace, was erased (src/erasure.ts): nothing of it is indexed again. */ | |
| 597 | + | export async function erasedSource(db: D1Database, kind: "page" | "folio" | null, id: string, workspaceId: string): Promise<boolean> { | |
| 598 | + | const row = await db | |
| 599 | + | .prepare("SELECT 1 FROM erased_subjects WHERE id = ?1 AND kind = 'workspace' UNION ALL SELECT 1 FROM erased_documents WHERE id = ?2 AND kind = ?3 LIMIT 1") | |
| 600 | + | .bind(workspaceId, id, kind ?? "") | |
| 601 | + | .first(); | |
| 602 | + | return row != null; | |
| 603 | + | } | |
| 604 | + | ||
| 605 | + | /** | |
| 606 | + | * Checked again after vectors were written: an erasure that ran while they | |
| 607 | + | * were being embedded found no chunk rows for them, so they are taken back | |
| 608 | + | * here. Throws when they cannot be, so the caller does not record them. | |
| 609 | + | */ | |
| 610 | + | export async function dropIfErased(db: D1Database, store: VectorStore, kind: "page" | "folio" | null, id: string, workspaceId: string, ids: string[]): Promise<void> { | |
| 611 | + | if (!ids.length || !(await erasedSource(db, kind, id, workspaceId))) return; | |
| 612 | + | await store.delete(ids); | |
| 613 | + | await db.batch([ | |
| 614 | + | db.prepare(`DELETE FROM ${kind === "folio" ? "folio_chunks_fts WHERE folio_id" : "doc_chunks_fts WHERE doc_id"} = ?`).bind(id), | |
| 615 | + | ]); | |
| 616 | + | } | |
| 71 | 71 | ||
| 72 | 72 | export class PageRoom extends DurableObject<Env> { | |
| 73 | 73 | private doc: Y.Doc | null = null; | |
| 74 | + | /** Saves and index runs in flight: destroy() waits for them, so none writes after it. */ | |
| 75 | + | private saves = new Set<Promise<unknown>>(); | |
| 74 | 76 | /** The last awareness update each client sent, so newcomers see everyone at once. Lost on hibernation; clients resend every 15 s. */ | |
| 75 | 77 | private awareness = new Map<number, Uint8Array>(); | |
| 76 | 78 | /** Each client's last awareness clock, to mark it gone with the next one. */ | |
| ⋯ | |||
| 99 | 101 | } | |
| 100 | 102 | ||
| 101 | 103 | private setMeta(key: string, value: unknown): void { | |
| 104 | + | if (key !== "erased" && this.meta("erased", false)) return; | |
| 102 | 105 | this.ctx.storage.sql.exec("INSERT INTO meta (key, value) VALUES (?, ?) ON CONFLICT (key) DO UPDATE SET value = excluded.value", key, JSON.stringify(value)); | |
| 103 | 106 | } | |
| 104 | 107 | ||
| 105 | 108 | /** The document, read from storage the first time it is needed after waking. */ | |
| 109 | + | async forgetPerson(key: string, username: string): Promise<void> { | |
| 110 | + | if (this.meta("erased", false)) return; | |
| 111 | + | this.setMeta(`erased:${key}`, true); | |
| 112 | + | await this.setRole(key, null); | |
| 113 | + | await Promise.allSettled([...this.saves]); | |
| 114 | + | for (const field of ["editors", "pending_authors"]) this.setMeta(field, this.meta<string[]>(field, []).filter(value => value !== key)); | |
| 115 | + | this.setMeta("editor_names", this.meta<string[]>("editor_names", []).filter(value => value.toLowerCase() !== username.toLowerCase())); | |
| 116 | + | } | |
| 117 | + | ||
| 118 | + | async destroy(): Promise<void> { | |
| 119 | + | // Synchronous tombstone comes first; no reentrant save can recreate state. | |
| 120 | + | this.setMeta("erased", true); | |
| 121 | + | for (const socket of this.ctx.getWebSockets()) { try { socket.close(4403, "Document erased"); } catch {} } | |
| 122 | + | await Promise.allSettled([...this.saves]); | |
| 123 | + | this.doc?.destroy(); | |
| 124 | + | this.doc = null; | |
| 125 | + | this.awareness.clear(); | |
| 126 | + | this.clocks.clear(); | |
| 127 | + | this.ctx.storage.transactionSync(() => { | |
| 128 | + | this.ctx.storage.sql.exec("DELETE FROM updates"); | |
| 129 | + | this.ctx.storage.sql.exec("DELETE FROM snapshot"); | |
| 130 | + | this.ctx.storage.sql.exec("DELETE FROM meta WHERE key != 'erased'"); | |
| 131 | + | }); | |
| 132 | + | await this.ctx.storage.deleteAlarm(); | |
| 133 | + | } | |
| 134 | + | ||
| 106 | 135 | private load(): Y.Doc { | |
| 136 | + | if (this.meta("erased", false)) throw new Error("Document erased"); | |
| 107 | 137 | if (this.doc) return this.doc; | |
| 108 | 138 | const doc = new Y.Doc({ gc: true }); | |
| 109 | 139 | const snapshot = this.ctx.storage.sql.exec<{ data: ArrayBuffer }>("SELECT data FROM snapshot WHERE id = 1").toArray()[0]; | |
| ⋯ | |||
| 121 | 151 | } | |
| 122 | 152 | ||
| 123 | 153 | private onUpdate(update: Uint8Array, origin: unknown): void { | |
| 154 | + | if (this.meta("erased", false)) return; | |
| 124 | 155 | this.ctx.storage.sql.exec("INSERT INTO updates (data) VALUES (?)", update); | |
| 125 | 156 | const count = this.ctx.storage.sql.exec<{ n: number }>("SELECT COUNT(*) AS n FROM updates").one().n; | |
| 126 | 157 | if (count >= COMPACT_AT) this.compact(); | |
| ⋯ | |||
| 162 | 193 | /** Makes sure the alarm rings by `when`: the save's, or the index's, whichever is first. */ | |
| 163 | 194 | private alarmBy(when: number): void { | |
| 164 | 195 | void this.ctx.storage.getAlarm().then((at) => { | |
| 196 | + | if (this.meta("erased", false)) return; | |
| 165 | 197 | if (at == null || at > when) return this.ctx.storage.setAlarm(when); | |
| 166 | 198 | }); | |
| 167 | 199 | } | |
| ⋯ | |||
| 179 | 211 | ||
| 180 | 212 | /** Saves to D1 now: the Markdown, search, links, mentions, and a version if one is due or asked for. */ | |
| 181 | 213 | private async persist(version: Origin | null = null): Promise<string | null> { | |
| 214 | + | if (this.meta("erased", false)) return null; | |
| 215 | + | const saving = this.save(version); | |
| 216 | + | this.saves.add(saving); | |
| 217 | + | try { return await saving; } finally { this.saves.delete(saving); } | |
| 218 | + | } | |
| 219 | + | ||
| 220 | + | private async save(version: Origin | null = null): Promise<string | null> { | |
| 182 | 221 | const pageId = this.meta<string | null>("page_id", null); | |
| 183 | 222 | if (!pageId) return null; | |
| 184 | 223 | const fragment = this.fragment(); | |
| ⋯ | |||
| 213 | 252 | } | |
| 214 | 253 | ||
| 215 | 254 | override async alarm(): Promise<void> { | |
| 255 | + | if (this.meta("erased", false)) return; | |
| 216 | 256 | await this.persist(); | |
| 217 | 257 | const due = this.meta<number | null>("index_at", null); | |
| 218 | 258 | if (due == null) return; | |
| ⋯ | |||
| 220 | 260 | this.setMeta("index_at", null); | |
| 221 | 261 | const pageId = this.meta<string | null>("page_id", null); | |
| 222 | 262 | // Never throws; a failure is retried on the next change. | |
| 223 | − | if (pageId) await indexPage(this.env, pageId); | |
| 263 | + | if (pageId) { | |
| 264 | + | const indexing = indexPage(this.env, pageId); | |
| 265 | + | this.saves.add(indexing); | |
| 266 | + | try { await indexing; } finally { this.saves.delete(indexing); } | |
| 267 | + | } | |
| 224 | 268 | } | |
| 225 | 269 | ||
| 226 | 270 | // ── Calls from the Worker ────────────────────────────────────────────── | |
| ⋯ | |||
| 356 | 400 | // ── Sockets ──────────────────────────────────────────────────────────── | |
| 357 | 401 | ||
| 358 | 402 | override async fetch(request: Request): Promise<Response> { | |
| 403 | + | if (this.meta("erased", false)) return new Response("Document erased", { status: 410 }); | |
| 359 | 404 | if (request.headers.get("upgrade")?.toLowerCase() !== "websocket") { | |
| 360 | 405 | return new Response("Expected a WebSocket upgrade\n", { status: 426 }); | |
| 361 | 406 | } | |
| ⋯ | |||
| 365 | 410 | } catch { | |
| 366 | 411 | return new Response("Missing member\n", { status: 400 }); | |
| 367 | 412 | } | |
| 413 | + | if (this.meta(`erased:${who.key}`, false)) return new Response("Account erased", { status: 410 }); | |
| 368 | 414 | if (who.workspace_slug) this.setMeta("workspace_slug", who.workspace_slug); | |
| 369 | 415 | if (who.page_id) this.setMeta("page_id", who.page_id); | |
| 370 | 416 | const pair = new WebSocketPair(); | |
| ⋯ | |||
| 390 | 436 | // Text frames are for the Worker's notices only; clients speak binary. | |
| 391 | 437 | if (typeof message === "string") return; | |
| 392 | 438 | const who = socket.deserializeAttachment() as Attachment | null; | |
| 393 | − | if (!who) return; | |
| 439 | + | if (!who || this.meta("erased", false) || this.meta(`erased:${who.key}`, false)) return; | |
| 394 | 440 | const data = new Uint8Array(message); | |
| 395 | 441 | const decoder = decoding.createDecoder(data); | |
| 396 | 442 | const type = decoding.readVarUint(decoder); | |
| 1 | + | -- Permanent tombstones for erased people and workspaces (src/erasure.ts), | |
| 2 | + | -- and the channels a purge still has to visit. | |
| 3 | + | CREATE TABLE erased_subjects (id TEXT PRIMARY KEY, kind TEXT NOT NULL); | |
| 4 | + | CREATE TABLE purge_channels (subject_id TEXT NOT NULL, channel_id TEXT NOT NULL, PRIMARY KEY(subject_id, channel_id)); | |
| 5 | + | ||
| 6 | + | -- A purge finds a person's messages, reactions and channels without scanning every row. | |
| 7 | + | CREATE INDEX messages_by_author ON messages (author); | |
| 8 | + | CREATE INDEX reactions_by_principal ON reactions (principal); | |
| 9 | + | CREATE INDEX channels_by_creator ON channels (created_by); | |
| 10 | + | ||
| 11 | + | -- Late, in-flight writes cannot recreate the erased person's membership, | |
| 12 | + | -- messages or reactions. Each looks the tombstone up by its key. | |
| 13 | + | CREATE TRIGGER erased_message BEFORE INSERT ON messages WHEN NEW.author LIKE 'user:%' AND EXISTS ( | |
| 14 | + | SELECT 1 FROM erased_subjects WHERE id = substr(NEW.author, 6) AND kind = 'user' | |
| 15 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 16 | + | CREATE TRIGGER erased_member BEFORE INSERT ON channel_members WHEN NEW.principal LIKE 'user:%' AND EXISTS ( | |
| 17 | + | SELECT 1 FROM erased_subjects WHERE id = substr(NEW.principal, 6) AND kind = 'user' | |
| 18 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 19 | + | CREATE TRIGGER erased_reaction BEFORE INSERT ON reactions WHEN (NEW.principal LIKE 'user:%' AND EXISTS ( | |
| 20 | + | SELECT 1 FROM erased_subjects WHERE id = substr(NEW.principal, 6) AND kind = 'user' | |
| 21 | + | )) OR NOT EXISTS (SELECT 1 FROM messages WHERE id = NEW.message_id) BEGIN SELECT RAISE(IGNORE); END; | |
| 22 | + | ||
| 23 | + | -- Nothing of an erased workspace comes back. | |
| 24 | + | CREATE TRIGGER erased_channel BEFORE INSERT ON channels WHEN EXISTS ( | |
| 25 | + | SELECT 1 FROM erased_subjects WHERE id = NEW.workspace_id AND kind = 'workspace' | |
| 26 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 27 | + | CREATE TRIGGER erased_general_joined BEFORE INSERT ON general_joined WHEN EXISTS ( | |
| 28 | + | SELECT 1 FROM erased_subjects WHERE id = NEW.workspace_id AND kind = 'workspace' | |
| 29 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 30 | + | CREATE TRIGGER erased_custom_emoji BEFORE INSERT ON custom_emoji WHEN EXISTS ( | |
| 31 | + | SELECT 1 FROM erased_subjects WHERE id = NEW.workspace_id AND kind = 'workspace' | |
| 32 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 33 | + | CREATE TRIGGER erased_chat_settings BEFORE INSERT ON chat_settings WHEN EXISTS ( | |
| 34 | + | SELECT 1 FROM erased_subjects WHERE id = NEW.workspace_id AND kind = 'workspace' | |
| 35 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 36 | + | CREATE TRIGGER erased_chat_meter BEFORE INSERT ON chat_meter WHEN EXISTS ( | |
| 37 | + | SELECT 1 FROM erased_subjects WHERE id = NEW.workspace_id AND kind = 'workspace' | |
| 38 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 39 | + | ||
| 40 | + | -- A message or member of a channel that is gone (an erased one) is not kept. | |
| 41 | + | CREATE TRIGGER orphan_message BEFORE INSERT ON messages WHEN NOT EXISTS ( | |
| 42 | + | SELECT 1 FROM channels WHERE id = NEW.channel_id | |
| 43 | + | ) BEGIN SELECT RAISE(IGNORE); END; | |
| 44 | + | CREATE TRIGGER orphan_member BEFORE INSERT ON channel_members WHEN NOT EXISTS ( | |
| 45 | + | SELECT 1 FROM channels WHERE id = NEW.channel_id | |
| 46 | + | ) BEGIN SELECT RAISE(IGNORE); END; |
| 1 | + | import { GHOST_ID } from "@g1t/contracts"; | |
| 2 | + | ||
| 3 | + | import { g1tDmPerson } from "./g1t.ts"; | |
| 4 | + | import type { ChannelRoom } from "./room.ts"; | |
| 5 | + | import { clearConversation } from "./clear.ts"; | |
| 6 | + | ||
| 7 | + | type Env = { DB: D1Database; ROOMS: DurableObjectNamespace<ChannelRoom>; AVATARS: KVNamespace }; | |
| 8 | + | export type Erasure = { kind: "user" | "workspace"; id: string; username?: string; slug?: string }; | |
| 9 | + | ||
| 10 | + | const GHOST = `user:${GHOST_ID}`; | |
| 11 | + | /** Messages deleted per statement, and per call, so one huge channel never holds the database in one write. */ | |
| 12 | + | const CHUNK = 500; | |
| 13 | + | const BUDGET = 5000; | |
| 14 | + | /** Channels handled per call; the rest wait for identity's next retry. */ | |
| 15 | + | const CHANNELS = 25; | |
| 16 | + | ||
| 17 | + | /** | |
| 18 | + | * Called only by identity's durable purge jobs, through a service binding. | |
| 19 | + | * Safe to repeat: every step is idempotent, and work left over answers | |
| 20 | + | * `pending` so identity calls again rather than recording success. | |
| 21 | + | */ | |
| 22 | + | export async function purgePersonalData(env: Env, args: Erasure): Promise<{ ok: boolean; pending?: boolean }> { | |
| 23 | + | if (!args?.id || !["user", "workspace"].includes(args.kind)) throw new Error("Invalid purge subject"); | |
| 24 | + | const db = env.DB, key = `user:${args.id}`; | |
| 25 | + | // The channels to visit are listed once, in the transaction that writes | |
| 26 | + | // the tombstone: a retry works through the list instead of scanning | |
| 27 | + | // every message again, and the list can never be skipped. | |
| 28 | + | const unseeded = "NOT EXISTS (SELECT 1 FROM erased_subjects WHERE id = ?1)"; | |
| 29 | + | await db.batch([ | |
| 30 | + | args.kind === "workspace" | |
| 31 | + | ? db.prepare(`INSERT OR IGNORE INTO purge_channels SELECT ?1, id FROM channels WHERE workspace_id = ?1 AND ${unseeded}`).bind(args.id) | |
| 32 | + | : db.prepare(`INSERT OR IGNORE INTO purge_channels SELECT ?1, id FROM channels WHERE (created_by = ?2 OR id IN (SELECT channel_id FROM channel_members WHERE principal = ?2 UNION SELECT channel_id FROM messages WHERE author = ?2)) AND ${unseeded}`).bind(args.id, key), | |
| 33 | + | db.prepare("INSERT OR IGNORE INTO erased_subjects (id, kind) VALUES (?, ?)").bind(args.id, args.kind), | |
| 34 | + | ]); | |
| 35 | + | let budget = BUDGET; | |
| 36 | + | const channels = (await db.prepare("SELECT channel_id FROM purge_channels WHERE subject_id = ? LIMIT ?").bind(args.id, CHANNELS).all<{ channel_id: string }>()).results; | |
| 37 | + | for (const { channel_id: id } of channels) { | |
| 38 | + | if (budget <= 0) break; | |
| 39 | + | const channel = await db.prepare("SELECT kind FROM channels WHERE id = ?").bind(id).first<{ kind: string }>(); | |
| 40 | + | const members = (await db.prepare("SELECT principal FROM channel_members WHERE channel_id = ?").bind(id).all<{ principal: string }>()).results.map(r => r.principal); | |
| 41 | + | const destroy = args.kind === "workspace" || (channel?.kind === "dm" && g1tDmPerson(members) === key); | |
| 42 | + | // References stay in purge_channels until both the room and database confirm cleanup. | |
| 43 | + | await env.ROOMS.get(env.ROOMS.idFromName(id)).erase(id, destroy ? null : key); | |
| 44 | + | if (destroy) { | |
| 45 | + | // In chunks, newest last, until the channel is empty or this call's budget is spent. | |
| 46 | + | let left = true; | |
| 47 | + | while (budget > 0) { | |
| 48 | + | const [, gone] = await db.batch([ | |
| 49 | + | db.prepare(`DELETE FROM reactions WHERE message_id IN (SELECT id FROM messages WHERE channel_id = ? ORDER BY id LIMIT ${CHUNK})`).bind(id), | |
| 50 | + | db.prepare(`DELETE FROM messages WHERE id IN (SELECT id FROM messages WHERE channel_id = ? ORDER BY id LIMIT ${CHUNK})`).bind(id), | |
| 51 | + | ]); | |
| 52 | + | const changes = Number(gone?.meta?.changes ?? 0); | |
| 53 | + | budget -= Math.max(changes, 1); | |
| 54 | + | if (changes < CHUNK) { left = false; break; } | |
| 55 | + | } | |
| 56 | + | if (left) break; | |
| 57 | + | await clearConversation(db, id); | |
| 58 | + | await db.batch([ | |
| 59 | + | db.prepare("DELETE FROM channel_members WHERE channel_id = ?").bind(id), | |
| 60 | + | db.prepare("DELETE FROM channels WHERE id = ?").bind(id), | |
| 61 | + | ]); | |
| 62 | + | } else { | |
| 63 | + | await db.batch([ | |
| 64 | + | db.prepare("UPDATE messages SET author = ? WHERE channel_id = ? AND author = ?").bind(GHOST, id, key), | |
| 65 | + | db.prepare("DELETE FROM channel_members WHERE channel_id = ? AND principal = ?").bind(id, key), | |
| 66 | + | db.prepare("UPDATE channels SET created_by = CASE WHEN created_by = ?2 THEN ?3 ELSE created_by END, dm_key = CASE WHEN kind = 'dm' THEN NULL ELSE dm_key END WHERE id = ?1").bind(id, key, GHOST), | |
| 67 | + | ]); | |
| 68 | + | budget -= 1; | |
| 69 | + | } | |
| 70 | + | await db.prepare("DELETE FROM purge_channels WHERE subject_id = ? AND channel_id = ?").bind(args.id, id).run(); | |
| 71 | + | } | |
| 72 | + | if (await db.prepare("SELECT 1 FROM purge_channels WHERE subject_id = ? LIMIT 1").bind(args.id).first()) return { ok: false, pending: true }; | |
| 73 | + | if (args.kind === "user") { | |
| 74 | + | await db.batch([ | |
| 75 | + | db.prepare("DELETE FROM reactions WHERE principal = ?").bind(key), | |
| 76 | + | db.prepare("DELETE FROM general_joined WHERE principal = ?").bind(key), | |
| 77 | + | db.prepare("UPDATE custom_emoji SET created_by = ? WHERE created_by = ?").bind(GHOST, key), | |
| 78 | + | ]); | |
| 79 | + | } else { | |
| 80 | + | const files = (await db.prepare("SELECT DISTINCT file FROM custom_emoji WHERE workspace_id = ? AND file IS NOT NULL").bind(args.id).all<{ file: string }>()).results; | |
| 81 | + | for (const { file } of files) { | |
| 82 | + | const used = await db.prepare("SELECT 1 FROM custom_emoji WHERE workspace_id != ? AND file = ? LIMIT 1").bind(args.id, file).first(); | |
| 83 | + | if (!used) await env.AVATARS.delete(`emoji/${file}`); | |
| 84 | + | } | |
| 85 | + | await db.batch(["general_joined", "custom_emoji", "chat_settings", "former_agents", "chat_meter"].map(table => db.prepare(`DELETE FROM ${table} WHERE workspace_id = ?`).bind(args.id))); | |
| 86 | + | } | |
| 87 | + | return { ok: true }; | |
| 88 | + | } |
| 1 | + | import { purgePersonalData } from "./erasure.ts"; | |
| 1 | 2 | /** | |
| 2 | 3 | * The chat service: a workspace's channels, direct messages, threads and | |
| 3 | 4 | * messages. Conversations are between people (docs.g1t.sh/guides/chat/); | |
| ⋯ | |||
| 1813 | 1814 | } | |
| 1814 | 1815 | const match = url.pathname.match(/^\/rpc\/([a-z0-9_]+)$/); | |
| 1815 | 1816 | if (request.method !== "POST" || !match) return new Response("Not found\n", { status: 404 }); | |
| 1817 | + | if (match[1] === "purge_personal_data") return Response.json(await purgePersonalData(env, await request.json())); | |
| 1816 | 1818 | // A replica near the caller when it asks for one (@g1t/contracts d1.ts). | |
| 1817 | 1819 | const opened = openD1(env.DB, request); | |
| 1818 | 1820 | const service = new Chat(Object.create(env, { DB: { value: opened.db } }) as Env, (work) => ctx.waitUntil(work)); | |
| 84 | 84 | } | |
| 85 | 85 | ||
| 86 | 86 | override async fetch(request: Request): Promise<Response> { | |
| 87 | + | if (await this.ctx.storage.get("erased")) return new Response("Conversation erased", { status: 410 }); | |
| 87 | 88 | if (request.headers.get("upgrade")?.toLowerCase() !== "websocket") { | |
| 88 | 89 | return new Response("Expected a WebSocket upgrade\n", { status: 426 }); | |
| 89 | 90 | } | |
| ⋯ | |||
| 93 | 94 | } catch { | |
| 94 | 95 | return new Response("Missing member\n", { status: 400 }); | |
| 95 | 96 | } | |
| 97 | + | if (await this.ctx.storage.get(`erased:${principalKey(who.member)}`)) return new Response("Account erased", { status: 410 }); | |
| 96 | 98 | const pair = new WebSocketPair(); | |
| 97 | 99 | const [client, server] = Object.values(pair) as [WebSocket, WebSocket]; | |
| 98 | 100 | // Tagged with the member's key, so their sockets can be found to drop. | |
| ⋯ | |||
| 230 | 232 | this.broadcast({ type: "channel.cleared", channel_id: channelId }); | |
| 231 | 233 | } | |
| 232 | 234 | ||
| 235 | + | /** | |
| 236 | + | * An erasure (src/erasure.ts). With a member, only that person is shut | |
| 237 | + | * out: everyone else's view and answers carry on, and nothing is cleared | |
| 238 | + | * for them. Without one, the whole conversation is going: its answer | |
| 239 | + | * job stops and every socket closes. | |
| 240 | + | */ | |
| 241 | + | async erase(channelId: string, principal: string | null): Promise<void> { | |
| 242 | + | if (principal) { | |
| 243 | + | await this.ctx.storage.put(`erased:${principal}`, true); | |
| 244 | + | this.drop(principal); | |
| 245 | + | return; | |
| 246 | + | } | |
| 247 | + | await this.ctx.storage.put("erased", true); | |
| 248 | + | await this.clear(channelId); | |
| 249 | + | await this.ctx.storage.deleteAlarm(); | |
| 250 | + | for (const socket of this.ctx.getWebSockets()) { try { socket.close(4403, "Conversation erased"); } catch {} } | |
| 251 | + | } | |
| 252 | + | ||
| 233 | 253 | /** Whether the question a job answers has been deleted by a clear. */ | |
| 234 | 254 | private async questionGone(job: AnswerJob): Promise<boolean> { | |
| 255 | + | if (await this.ctx.storage.get("erased") || await this.ctx.storage.get(`erased:user:${job.actor.id}`)) return true; | |
| 235 | 256 | if (!job.question_id) return false; | |
| 236 | 257 | const row = await this.env.DB.prepare("SELECT id FROM messages WHERE id = ?").bind(job.question_id).first<{ id: string }>(); | |
| 237 | 258 | return !row; | |
| ⋯ | |||
| 289 | 310 | ||
| 290 | 311 | override async webSocketMessage(socket: WebSocket, message: string | ArrayBuffer): Promise<void> { | |
| 291 | 312 | const who = socket.deserializeAttachment() as RoomMember | null; | |
| 292 | − | if (!who) return; | |
| 313 | + | if (!who || await this.ctx.storage.get("erased") || await this.ctx.storage.get(`erased:${principalKey(who.member)}`)) return; | |
| 293 | 314 | // The only thing a client says over the socket: it is typing | |
| 294 | 315 | // (`{"type":"typing","channel_id":"<id>"}`, at most every 3 s). | |
| 295 | 316 | // Everything else (posting, reading) goes through the site, which | |
| 1 | + | -- Purge work is committed with the destructive transaction, never after it. | |
| 2 | + | -- A successful publish means delivery to the event bus, not consumer completion. | |
| 3 | + | CREATE TABLE purge_jobs ( | |
| 4 | + | subject_id TEXT NOT NULL, | |
| 5 | + | kind TEXT NOT NULL CHECK (kind IN ('user', 'workspace')), | |
| 6 | + | target TEXT NOT NULL, | |
| 7 | + | payload TEXT NOT NULL, | |
| 8 | + | created_at TEXT NOT NULL, | |
| 9 | + | completed_at TEXT, | |
| 10 | + | attempts INTEGER NOT NULL DEFAULT 0, | |
| 11 | + | retry_at TEXT NOT NULL, | |
| 12 | + | last_error TEXT, | |
| 13 | + | PRIMARY KEY (subject_id, target) | |
| 14 | + | ); | |
| 15 | + | CREATE INDEX purge_jobs_pending ON purge_jobs(completed_at, retry_at); | |
| 16 | + | ||
| 17 | + | -- No FK: the case and minimal deletion ledger must survive account erasure. | |
| 18 | + | CREATE TABLE erasure_requests ( | |
| 19 | + | id TEXT PRIMARY KEY, | |
| 20 | + | user_id TEXT NOT NULL, | |
| 21 | + | username TEXT NOT NULL, | |
| 22 | + | requested_at TEXT NOT NULL, | |
| 23 | + | due_at TEXT NOT NULL, | |
| 24 | + | status TEXT NOT NULL DEFAULT 'received' CHECK (status IN ('received','reviewing','completed','refused')), | |
| 25 | + | scope TEXT NOT NULL, | |
| 26 | + | evidence TEXT NOT NULL DEFAULT '', | |
| 27 | + | updated_at TEXT NOT NULL, | |
| 28 | + | updated_by TEXT, | |
| 29 | + | closed_at TEXT | |
| 30 | + | ); | |
| 31 | + | CREATE UNIQUE INDEX erasure_requests_open ON erasure_requests(user_id) WHERE closed_at IS NULL; | |
| 32 | + | ||
| 33 | + | CREATE TABLE erasure_reviews ( | |
| 34 | + | id INTEGER PRIMARY KEY AUTOINCREMENT, | |
| 35 | + | request_id TEXT NOT NULL REFERENCES erasure_requests(id), | |
| 36 | + | at TEXT NOT NULL, | |
| 37 | + | staff TEXT NOT NULL, | |
| 38 | + | status TEXT NOT NULL, | |
| 39 | + | evidence TEXT NOT NULL | |
| 40 | + | ); |
| 61 | 61 | use g1t_contracts::Outcome; | |
| 62 | 62 | use g1t_contracts::User; | |
| 63 | 63 | use g1t_contracts::account_deletion::*; | |
| 64 | − | use g1t_contracts::events::{UserDeleted, UserDeleting, UserRestored}; | |
| 64 | + | use g1t_contracts::events::{UserDeleting, UserRestored}; | |
| 65 | 65 | use g1t_contracts::identity::{UserArgs, protected_names}; | |
| 66 | 66 | use g1t_contracts::time::rfc3339; | |
| 67 | 67 | use g1t_kit::now_ms; | |
| ⋯ | |||
| 884 | 884 | /// Removes a deleted account for good: see the module docs. | |
| 885 | 885 | async fn purge_account(&self, row: &DeletedRow) -> Result<()> { | |
| 886 | 886 | let now = rfc3339(now_ms()); | |
| 887 | − | let values = [row.id.as_str(), row.username.as_str(), row.deleted_at.as_str(), now.as_str()]; | |
| 888 | − | let mut batch = Vec::new(); | |
| 887 | + | let values = [ | |
| 888 | + | row.id.as_str(), | |
| 889 | + | row.username.as_str(), | |
| 890 | + | row.deleted_at.as_str(), | |
| 891 | + | now.as_str(), | |
| 892 | + | ]; | |
| 893 | + | let mut batch = self.purge_job_statements( | |
| 894 | + | "user", | |
| 895 | + | &row.id, | |
| 896 | + | serde_json::json!({"username": row.username, "avatar": row.avatar}), | |
| 897 | + | )?; | |
| 889 | 898 | for (sql, binds) in purge_statements() { | |
| 890 | − | let binds: Vec<JsValue> = values[..binds].iter().map(|value| JsValue::from(*value)).collect(); | |
| 899 | + | let binds: Vec<JsValue> = values[..binds] | |
| 900 | + | .iter() | |
| 901 | + | .map(|value| JsValue::from(*value)) | |
| 902 | + | .collect(); | |
| 891 | 903 | batch.push(self.db.prepare(sql).bind(&binds)?); | |
| 892 | 904 | } | |
| 893 | 905 | self.db.batch(batch).await?; | |
| 894 | − | if let Err(error) = self.forget_avatar(row.avatar.clone()).await { | |
| 895 | − | worker::console_error!("avatar of {} not removed: {error}", row.username); | |
| 896 | − | } | |
| 897 | − | self.announce("user.deleted", None, UserDeleted { user_id: row.id.clone(), username: row.username.clone() }) | |
| 898 | − | .await; | |
| 899 | 906 | Ok(()) | |
| 900 | 907 | } | |
| 901 | 908 | } | |
| 46 | 46 | AuditActor, AuditOutcome, AuditTarget, NewAuditEntry, RecordAuditArgs, Surface, | |
| 47 | 47 | }; | |
| 48 | 48 | use g1t_contracts::billing::CloseWorkspaceArgs; | |
| 49 | − | use g1t_contracts::events::{WorkspaceDeleted, WorkspaceDeleting, WorkspaceRestored}; | |
| 49 | + | use g1t_contracts::events::{WorkspaceDeleting, WorkspaceRestored}; | |
| 50 | 50 | use g1t_contracts::identity::*; | |
| 51 | 51 | use g1t_contracts::repos::NamespaceCountArgs; | |
| 52 | 52 | use g1t_contracts::time::rfc3339; | |
| ⋯ | |||
| 685 | 685 | async fn purge(&self, row: &DeletedRow) -> Result<()> { | |
| 686 | 686 | let id = row.id.as_str(); | |
| 687 | 687 | let by = row.deleted_by.as_deref().unwrap_or_default(); | |
| 688 | − | self.db | |
| 689 | − | .batch(vec![ | |
| 688 | + | #[derive(Deserialize)] | |
| 689 | + | struct Avatar { | |
| 690 | + | avatar: Option<String>, | |
| 691 | + | } | |
| 692 | + | let avatar = self | |
| 693 | + | .db | |
| 694 | + | .prepare("SELECT avatar FROM workspaces WHERE id = ?") | |
| 695 | + | .bind(&[id.into()])? | |
| 696 | + | .first::<Avatar>(None) | |
| 697 | + | .await? | |
| 698 | + | .and_then(|r| r.avatar); | |
| 699 | + | // Who notify must clear the workspace from, and every slug its | |
| 700 | + | // notifications may name: members as they are now (the batch below | |
| 701 | + | // removes them) and the names it had, unless another workspace holds | |
| 702 | + | // one now. | |
| 703 | + | #[derive(Deserialize)] | |
| 704 | + | struct Value { | |
| 705 | + | value: String, | |
| 706 | + | } | |
| 707 | + | let members = self | |
| 708 | + | .db | |
| 709 | + | .prepare("SELECT user_id AS value FROM workspace_members WHERE workspace_id = ?") | |
| 710 | + | .bind(&[id.into()])? | |
| 711 | + | .all() | |
| 712 | + | .await? | |
| 713 | + | .results::<Value>()? | |
| 714 | + | .into_iter() | |
| 715 | + | .map(|r| r.value) | |
| 716 | + | .collect::<Vec<_>>(); | |
| 717 | + | let old_slugs = self | |
| 718 | + | .db | |
| 719 | + | .prepare("SELECT old_slug AS value FROM workspace_redirects WHERE workspace_id = ?1 AND old_slug NOT IN (SELECT slug FROM workspaces WHERE id != ?1)") | |
| 720 | + | .bind(&[id.into()])? | |
| 721 | + | .all() | |
| 722 | + | .await? | |
| 723 | + | .results::<Value>()? | |
| 724 | + | .into_iter() | |
| 725 | + | .map(|r| r.value) | |
| 726 | + | .collect::<Vec<_>>(); | |
| 727 | + | let mut batch = self.purge_job_statements( | |
| 728 | + | "workspace", | |
| 729 | + | id, | |
| 730 | + | serde_json::json!({"slug": row.slug, "avatar": avatar, "members": members, "old_slugs": old_slugs}), | |
| 731 | + | )?; | |
| 732 | + | batch.extend(vec![ | |
| 690 | 733 | self.db | |
| 691 | 734 | .prepare( | |
| 692 | 735 | "INSERT OR REPLACE INTO deleted_workspaces (slug, workspace_id, name, deleted_by, deleted_at) | |
| 693 | − | VALUES (?, ?, ?, ?, ?)", | |
| 736 | + | SELECT ?1, ?2, ?3, ?4, ?5 WHERE EXISTS (SELECT 1 FROM workspaces WHERE id = ?2 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)", | |
| 694 | 737 | ) | |
| 695 | 738 | .bind(&[ | |
| 696 | 739 | row.slug.as_str().into(), | |
| ⋯ | |||
| 704 | 747 | self.db | |
| 705 | 748 | .prepare( | |
| 706 | 749 | "INSERT OR IGNORE INTO deleted_workspaces (slug, workspace_id, name, deleted_by, deleted_at) | |
| 707 | − | SELECT old_slug, workspace_id, ?, ?, ? FROM workspace_redirects WHERE workspace_id = ?", | |
| 750 | + | SELECT old_slug, workspace_id, ?, ?, ? FROM workspace_redirects WHERE workspace_id = ?4 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?4 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)", | |
| 708 | 751 | ) | |
| 709 | 752 | .bind(&[ | |
| 710 | 753 | row.name.as_str().into(), | |
| ⋯ | |||
| 713 | 756 | id.into(), | |
| 714 | 757 | ])?, | |
| 715 | 758 | self.db | |
| 716 | − | .prepare("DELETE FROM access_tokens WHERE workspace_id = ?") | |
| 759 | + | .prepare("DELETE FROM access_tokens WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 717 | 760 | .bind(&[id.into()])?, | |
| 718 | 761 | self.db | |
| 719 | − | .prepare("DELETE FROM workspace_members WHERE workspace_id = ?") | |
| 762 | + | .prepare("DELETE FROM workspace_members WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 720 | 763 | .bind(&[id.into()])?, | |
| 721 | 764 | // Each member's dock pins in it (dock.rs). | |
| 722 | 765 | self.db | |
| 723 | − | .prepare("DELETE FROM dock_pins WHERE workspace_id = ?") | |
| 766 | + | .prepare("DELETE FROM dock_pins WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 724 | 767 | .bind(&[id.into()])?, | |
| 725 | 768 | // Its teams, their people and the roles they gave (teams.rs). | |
| 726 | 769 | self.db | |
| 727 | − | .prepare("DELETE FROM team_members WHERE team_id IN (SELECT id FROM teams WHERE workspace_id = ?)") | |
| 770 | + | .prepare("DELETE FROM team_members WHERE team_id IN (SELECT id FROM teams WHERE workspace_id = ?1) AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 728 | 771 | .bind(&[id.into()])?, | |
| 729 | 772 | self.db | |
| 730 | − | .prepare("DELETE FROM repo_grants WHERE principal_kind = 'team' AND principal_id IN (SELECT id FROM teams WHERE workspace_id = ?)") | |
| 773 | + | .prepare("DELETE FROM repo_grants WHERE principal_kind = 'team' AND principal_id IN (SELECT id FROM teams WHERE workspace_id = ?1) AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 731 | 774 | .bind(&[id.into()])?, | |
| 732 | 775 | self.db | |
| 733 | − | .prepare("DELETE FROM teams WHERE workspace_id = ?") | |
| 776 | + | .prepare("DELETE FROM teams WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 734 | 777 | .bind(&[id.into()])?, | |
| 735 | 778 | self.db | |
| 736 | − | .prepare("DELETE FROM workspace_redirects WHERE workspace_id = ?") | |
| 779 | + | .prepare("DELETE FROM workspace_redirects WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 737 | 780 | .bind(&[id.into()])?, | |
| 738 | 781 | // Aliases staff pointed at it lead nowhere now (aliases.rs). | |
| 739 | 782 | self.db | |
| 740 | − | .prepare("DELETE FROM workspace_aliases WHERE workspace_id = ?") | |
| 783 | + | .prepare("DELETE FROM workspace_aliases WHERE workspace_id = ?1 AND EXISTS (SELECT 1 FROM workspaces WHERE id = ?1 AND deleted_at IS NOT NULL AND purge_after IS NOT NULL)") | |
| 741 | 784 | .bind(&[id.into()])?, | |
| 742 | 785 | // Only while it is still deleted: a restore a moment ago wins. | |
| 743 | 786 | self.db | |
| 744 | − | .prepare("DELETE FROM workspaces WHERE id = ? AND deleted_at IS NOT NULL") | |
| 787 | + | .prepare("DELETE FROM workspaces WHERE id = ? AND deleted_at IS NOT NULL AND purge_after IS NOT NULL") | |
| 745 | 788 | .bind(&[id.into()])?, | |
| 746 | − | ]) | |
| 747 | − | .await?; | |
| 748 | − | self.announce( | |
| 749 | − | "workspace.deleted", | |
| 750 | − | None, | |
| 751 | − | WorkspaceDeleted { | |
| 752 | − | workspace_id: row.id.clone(), | |
| 753 | − | slug: row.slug.clone(), | |
| 754 | − | }, | |
| 755 | − | ) | |
| 756 | − | .await; | |
| 789 | + | ]); | |
| 790 | + | self.db.batch(batch).await?; | |
| 757 | 791 | Ok(()) | |
| 758 | 792 | } | |
| 759 | 793 | ||
| 23 | 23 | mod oauth; | |
| 24 | 24 | mod paid; | |
| 25 | 25 | mod people; | |
| 26 | + | mod privacy; | |
| 26 | 27 | mod profiles; | |
| 28 | + | mod purge_jobs; | |
| 27 | 29 | mod rename; | |
| 28 | 30 | mod job_tokens; | |
| 29 | 31 | mod run_credentials; | |
| ⋯ | |||
| 856 | 858 | if let Err(error) = identity.purge_due_accounts().await { | |
| 857 | 859 | worker::console_error!("account purge: {error}"); | |
| 858 | 860 | } | |
| 861 | + | if let Err(error) = identity.run_purge_jobs().await { | |
| 862 | + | worker::console_error!("purge jobs: {error}"); | |
| 863 | + | } | |
| 859 | 864 | // Once: creators of repositories made before they got Admin (members.rs). | |
| 860 | 865 | if let Err(error) = identity.backfill_creator_grants().await { | |
| 861 | 866 | worker::console_error!("creator grants: {error}"); | |
| ⋯ | |||
| 1123 | 1128 | "check_account_deletion" => reply(&identity.check_account_deletion(args(body)?).await?), | |
| 1124 | 1129 | "delete_account" => reply(&identity.delete_account(args(body)?).await?), | |
| 1125 | 1130 | "admin_delete_account" => reply(&identity.admin_delete_account(args(body)?).await?), | |
| 1131 | + | "request_erasure" => reply(&identity.request_erasure(args(body)?).await?), | |
| 1132 | + | "my_erasure" => reply(&identity.my_erasure(args(body)?).await?), | |
| 1133 | + | "admin_intake_erasure" => reply(&identity.intake_erasure(args(body)?).await?), | |
| 1134 | + | "admin_erasure_requests" => reply(&identity.erasure_requests().await?), | |
| 1135 | + | "admin_review_erasure" => reply(&identity.review_erasure(args(body)?).await?), | |
| 1136 | + | "admin_pending_purges" => reply(&identity.pending_purge_jobs().await?), | |
| 1126 | 1137 | "admin_deleted_accounts" => reply(&identity.admin_deleted_accounts().await?), | |
| 1127 | 1138 | "admin_restore_account" => reply(&identity.admin_restore_account(args(body)?).await?), | |
| 1128 | 1139 | "admin_purge_account" => reply(&identity.admin_purge_account(args(body)?).await?), | |
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Comments
Sign in to comment.