Audit logs are kept by plan: a week on free, 90 days on the plan, and what staff set for an account in sudo
Free workspaces kept the same 90 days of audit log as paying ones. Now: - billing: FREE_AUDIT_RETENTION_DAYS (7) for free workspaces, AUDIT_RETENTION_DAYS (90) for the plan, comped and enterprise; an account's own number (allowances.audit_retention_days, 0029), set in sudo up to AUDIT_MAX_DAYS (400), wins either way. Without a Stripe key every workspace is on the plan, so self-hosting keeps 90. - billing: audit_retention RPC, each workspace's days; entitlements and the price book carry free and plan days. - events: a ceiling purge at AUDIT_MAX_DAYS, then a daily pass that asks billing (new BILLING binding) for each workspace with entries older than a week and deletes what its days do not keep, carrying on from a cursor. If billing cannot be reached nothing past the ceiling goes. 0004 indexes audit_entries by (workspace, time) and (time). - sudo: Audit log days on the account's allowances; the workspace page says when staff set it. - site: pricing shows a week free and 90 days on the plan; the audit log says how far back it goes. - docs: the audit log guide, usage and billing, workspaces.
| 75 | 75 | ||
| 76 | 76 | ## How long it is kept | |
| 77 | 77 | ||
| 78 | − | The log can be read and exported back **90 days**, on every workspace, | |
| 79 | − | with or without the [g1t plan](/guides/usage-and-billing/#the-g1t-plan). | |
| 80 | − | Entries older than 90 days are removed. | |
| 78 | + | How far back the log goes depends on the workspace's plan: | |
| 79 | + | ||
| 80 | + | | Workspace | Kept | | |
| 81 | + | | --- | --- | | |
| 82 | + | | Free | **7 days** | | |
| 83 | + | | On the [g1t plan](/guides/usage-and-billing/#the-g1t-plan) | **90 days** | | |
| 84 | + | | Paid for by an [enterprise](/guides/usage-and-billing/#enterprises-and-custom-terms) | **90 days** | | |
| 85 | + | | Longer, by arrangement | Up to **400 days** | | |
| 86 | + | ||
| 87 | + | The log can be read and exported back that far, and no further. Once a | |
| 88 | + | day, entries older than that are **deleted**, and cannot be brought back: | |
| 89 | + | export what you need to keep before then. Starting the plan keeps 90 days | |
| 90 | + | from then on; entries already deleted stay deleted. Ending it goes back to | |
| 91 | + | 7 days, and the next daily pass deletes what is older. | |
| 92 | + | ||
| 93 | + | For a longer log, such as for a compliance requirement, email | |
| 94 | + | [support@g1t.sh](mailto:support@g1t.sh). g1t can set the workspace's | |
| 95 | + | account to keep up to 400 days, and what is set there takes the place of | |
| 96 | + | the plan's. A [self-hosted](/guides/self-hosting/) g1t that does not charge | |
| 97 | + | keeps 90 days for every workspace. | |
| 81 | 98 | ||
| 82 | 99 | ## Export | |
| 83 | 100 |
| 20 | 20 | - [Security scans](#storage-search-embeddings-and-scans) of history and | |
| 21 | 21 | dependencies, which g1t pays for on a free workspace. | |
| 22 | 22 | - Search and Explore. | |
| 23 | − | - The [audit log](/guides/audit-log/), kept 90 days, with export. | |
| 23 | + | - The [audit log](/guides/audit-log/), with export: kept 7 days, or 90 on | |
| 24 | + | the plan. | |
| 24 | 25 | - Your own agent through [MCP](/reference/mcp/). | |
| 25 | 26 | - 1 GB of private repository storage, and 50,000 | |
| 26 | 27 | [git operations](#git-operations) a month. | |
| 52 | 53 | - **Unlimited** projects, previews and repositories. | |
| 53 | 54 | - Agents, workflows, the merge queue, | |
| 54 | 55 | [deployments](/guides/deployments/) and semantic search. | |
| 56 | + | - **90 days** of [audit log](/guides/audit-log/#how-long-it-is-kept), in | |
| 57 | + | place of a free workspace's 7. | |
| 55 | 58 | - Usage past $10 is charged at cost plus 20%, **up to your | |
| 56 | 59 | [spend limit](#limits)**. | |
| 57 | 60 | ||
| 576 | 579 | ## Security on every plan | |
| 577 | 580 | ||
| 578 | 581 | Security is never a paid extra. Every workspace, free or on the plan, has | |
| 579 | − | the same [audit log](/guides/audit-log/), kept 90 days, and the same CSV | |
| 580 | − | and JSON export. Single sign-on through your identity provider is not | |
| 582 | + | the [audit log](/guides/audit-log/), with the same CSV and JSON export, | |
| 583 | + | and secret push protection. What the plan changes is how long the log is | |
| 584 | + | kept: [7 days free, 90 on the plan](/guides/audit-log/#how-long-it-is-kept), | |
| 585 | + | and longer by arrangement. Single sign-on through your identity provider is not | |
| 581 | 586 | built yet; when it is, it will be on every plan. | |
| 582 | 587 | ||
| 583 | 588 | ## Enterprises and custom terms | |
| 597 | 602 | stays accurate. | |
| 598 | 603 | - **Custom**: a discount on usage, a limit of its own, or a larger share | |
| 599 | 604 | of the pools, sometimes until a date. | |
| 605 | + | - **A longer audit log**: up to 400 days for every workspace the account | |
| 606 | + | pays for, in place of the plan's 7 or 90. See | |
| 607 | + | [how long it is kept](/guides/audit-log/#how-long-it-is-kept). | |
| 600 | 608 | ||
| 601 | 609 | g1t's own workspaces and those of Flagon, Inc., the company that makes g1t, | |
| 602 | 610 | run comped. Their usage is recorded at cost, apart from what customers |
| 193 | 193 | | Webhooks, integrations, secrets and variables | The workspace's own are removed. | | |
| 194 | 194 | | Memory and guardrails | The workspace's own are removed. | | |
| 195 | 195 | | Statements, invoices and the ledger | Kept, for accounting. | | |
| 196 | − | | The audit log | Kept for its usual [90 days](/guides/audit-log/#how-long-it-is-kept), with the deletion as its last entry. With no owners left, ask support@g1t.sh for an export. | | |
| 196 | + | | The audit log | Kept as [long as its account keeps it](/guides/audit-log/#how-long-it-is-kept), with the deletion as its last entry: once the plan ends with the workspace, that is 7 days, unless an enterprise pays for it or longer was arranged. With no owners left, ask support@g1t.sh for an export. | | |
| 197 | 197 | | Recently deleted repositories | Purged with it, their git data with them. | | |
| 198 | 198 | | Old addresses | Redirects for repositories transferred out keep working. The workspace's own pages answer 404. | | |
| 199 | 199 |
| 284 | 284 | } | |
| 285 | 285 | if (allowances.runCapMicros != null) lines.push(`Run cap: ${usd(allowances.runCapMicros)} a run`); | |
| 286 | 286 | if (allowances.issueCapMicros != null) lines.push(`Issue cap: ${usd(allowances.issueCapMicros)} an issue`); | |
| 287 | + | if (allowances.auditRetentionDays != null) lines.push(`Audit log: ${allowances.auditRetentionDays} days, in place of the plan's`); | |
| 287 | 288 | if (allowances.hold) lines.push(`Held: ${allowances.hold}`); | |
| 288 | 289 | return lines; | |
| 289 | 290 | } | |
| 291 | 292 | /** | |
| 292 | 293 | * The g1t plan without its price, the account's share of g1t's pools, and | |
| 293 | 294 | * staff's overrides of what owners set: agents at once, the run and issue | |
| 294 | − | * caps, and a hold on new compute. Comped accounts have the plan anyway. | |
| 295 | + | * caps, the days of audit log kept (such as for an organization that pays | |
| 296 | + | * for longer), and a hold on new compute. Comped accounts have the plan | |
| 297 | + | * anyway. | |
| 295 | 298 | */ | |
| 296 | 299 | export function AllowancesForm({ | |
| 297 | 300 | allowances, | |
| 365 | 368 | <Field label="Issue cap $" hint="One issue's agents in all, over the owners'. Blank: theirs, or $10."> | |
| 366 | 369 | <Input name="issueCap" inputMode="decimal" placeholder="Owners'" defaultValue={values?.issueCap ?? dollarsField(current.issueCapMicros)} /> | |
| 367 | 370 | </Field> | |
| 371 | + | <Field label="Audit log days" hint="Kept, in place of the plan's, longer or shorter, up to 400. Blank: the plan's (7 free, 90 on the plan)."> | |
| 372 | + | <Input | |
| 373 | + | name="auditDays" | |
| 374 | + | inputMode="numeric" | |
| 375 | + | pattern="\d{1,3}" | |
| 376 | + | placeholder="Plan's" | |
| 377 | + | defaultValue={values?.auditDays ?? (current.auditRetentionDays != null ? String(current.auditRetentionDays) : "")} | |
| 378 | + | /> | |
| 379 | + | </Field> | |
| 368 | 380 | <Field label="Hold" hint="Pauses new compute and tells the owners why. Blank: no hold."> | |
| 369 | 381 | <Input name="hold" maxLength={200} placeholder="No hold" defaultValue={values?.hold ?? current.hold ?? ""} /> | |
| 370 | 382 | </Field> |
| 102 | 102 | test("allowances: the plan without its price, pool shares, and staff's overrides", () => { | |
| 103 | 103 | assert.deepEqual(parseAllowances(form({})), { | |
| 104 | 104 | ok: true, | |
| 105 | − | value: { plan: false, ossRepoMicros: null, trialMicros: null, maxConcurrentAgents: null, runCapMicros: null, issueCapMicros: null, hold: null }, | |
| 105 | + | value: { | |
| 106 | + | plan: false, | |
| 107 | + | ossRepoMicros: null, | |
| 108 | + | trialMicros: null, | |
| 109 | + | maxConcurrentAgents: null, | |
| 110 | + | runCapMicros: null, | |
| 111 | + | issueCapMicros: null, | |
| 112 | + | auditRetentionDays: null, | |
| 113 | + | hold: null, | |
| 114 | + | }, | |
| 106 | 115 | }); | |
| 107 | 116 | assert.deepEqual( | |
| 108 | − | parseAllowances(form({ plan: "on", oss: "5", trial: "2.50", agents: "20", runCap: "25", issueCap: "500", hold: " Card disputed;\nwaiting on the bank " })), | |
| 117 | + | parseAllowances(form({ plan: "on", oss: "5", trial: "2.50", agents: "20", runCap: "25", issueCap: "500", auditDays: "365", hold: " Card disputed;\nwaiting on the bank " })), | |
| 109 | 118 | { | |
| 110 | 119 | ok: true, | |
| 111 | 120 | value: { | |
| 115 | 124 | maxConcurrentAgents: 20, | |
| 116 | 125 | runCapMicros: 25_000_000, | |
| 117 | 126 | issueCapMicros: 500_000_000, | |
| 127 | + | auditRetentionDays: 365, | |
| 118 | 128 | hold: "Card disputed; waiting on the bank", | |
| 119 | 129 | }, | |
| 120 | 130 | }, | |
| 127 | 137 | assert.equal(parseAllowances(form({ runCap: "0.05" })).ok, false); | |
| 128 | 138 | assert.equal(parseAllowances(form({ runCap: "1000.01" })).ok, false); | |
| 129 | 139 | assert.equal(parseAllowances(form({ issueCap: "10000.01" })).ok, false); | |
| 140 | + | // Audit log days: shorter or longer than the plan's, up to 400. | |
| 141 | + | assert.equal((parseAllowances(form({ auditDays: "3" })) as { value: { auditRetentionDays: number } }).value.auditRetentionDays, 3); | |
| 142 | + | assert.equal((parseAllowances(form({ auditDays: "400" })) as { value: { auditRetentionDays: number } }).value.auditRetentionDays, 400); | |
| 143 | + | assert.equal(parseAllowances(form({ auditDays: "0" })).ok, false); | |
| 144 | + | assert.equal(parseAllowances(form({ auditDays: "401" })).ok, false); | |
| 145 | + | assert.equal(parseAllowances(form({ auditDays: "30.5" })).ok, false); | |
| 146 | + | assert.equal(parseAllowances(form({ auditDays: "a year" })).ok, false); | |
| 130 | 147 | assert.equal(parseAllowances(form({ hold: "x".repeat(201) })).ok, false); | |
| 131 | 148 | }); | |
| 132 | 149 |
| 184 | 184 | /** Staff's overrides of the owners' caps: one run, and one issue's agents in all. */ | |
| 185 | 185 | export const MAX_RUN_CAP_MICROS = 1_000 * MICROS_PER_DOLLAR; | |
| 186 | 186 | export const MAX_ISSUE_CAP_MICROS = 10_000 * MICROS_PER_DOLLAR; | |
| 187 | + | /** | |
| 188 | + | * The most days of audit log staff can give one account: billing's | |
| 189 | + | * AUDIT_MAX_DAYS. The events service deletes anything older for everyone. | |
| 190 | + | */ | |
| 191 | + | export const MAX_AUDIT_DAYS = 400; | |
| 187 | 192 | /** The smallest cap: ten cents, as owners may set. */ | |
| 188 | 193 | const MIN_CAP_MICROS = 100_000; | |
| 189 | 194 | const MAX_HOLD = 200; | |
| 191 | 196 | /** | |
| 192 | 197 | * Allowances from the plan-and-pools form: the g1t plan without its price, | |
| 193 | 198 | * the account's share of the open-source pool and of trials, and staff's | |
| 194 | − | * overrides of agents at once, the run cap and the issue cap. A blank | |
| 195 | − | * amount means the default (for a cap, no override). A hold is a line | |
| 196 | − | * saying why new compute is held; blank is no hold. | |
| 199 | + | * overrides of agents at once, the run cap, the issue cap and the days of | |
| 200 | + | * audit log kept. A blank amount means the default (for a cap, no | |
| 201 | + | * override; for the audit log, the plan's). A hold is a line saying why | |
| 202 | + | * new compute is held; blank is no hold. | |
| 197 | 203 | */ | |
| 198 | 204 | export function parseAllowances(form: FormData): Parsed<Allowances> { | |
| 199 | 205 | const amount = (name: string, what: string, max: number, maxText: string, min = 0): Parsed<number | null> => { | |
| 223 | 229 | maxConcurrentAgents = Number(rawAgents); | |
| 224 | 230 | } | |
| 225 | 231 | ||
| 232 | + | let auditRetentionDays: number | null = null; | |
| 233 | + | const rawAudit = text(form, "auditDays"); | |
| 234 | + | if (rawAudit) { | |
| 235 | + | if (!/^\d{1,3}$/.test(rawAudit) || Number(rawAudit) < 1 || Number(rawAudit) > MAX_AUDIT_DAYS) { | |
| 236 | + | return { ok: false, error: `Audit log days is a whole number from 1 to ${MAX_AUDIT_DAYS}, or blank for the plan's.` }; | |
| 237 | + | } | |
| 238 | + | auditRetentionDays = Number(rawAudit); | |
| 239 | + | } | |
| 240 | + | ||
| 226 | 241 | const hold = text(form, "hold").replace(/\s+/g, " "); | |
| 227 | 242 | if (hold.length > MAX_HOLD) return { ok: false, error: `Keep the hold's reason under ${MAX_HOLD} characters.` }; | |
| 228 | 243 | ||
| 235 | 250 | maxConcurrentAgents, | |
| 236 | 251 | runCapMicros: runCap.value, | |
| 237 | 252 | issueCapMicros: issueCap.value, | |
| 253 | + | auditRetentionDays, | |
| 238 | 254 | hold: hold || null, | |
| 239 | 255 | }, | |
| 240 | 256 | }; |
| 538 | 538 | ? `${e.gitOperations.toLocaleString("en-US")}${e.gitOperationsIncluded ? ` (${e.gitOperationsIncluded.toLocaleString("en-US")} free)` : ""}` | |
| 539 | 539 | : "—", | |
| 540 | 540 | ], | |
| 541 | − | ["Audit log", `${e.auditRetentionDays} days`], | |
| 541 | + | ["Audit log", `${e.auditRetentionDays} days${e.auditRetentionCustom ? ", set by staff" : ""}`], | |
| 542 | 542 | ]; | |
| 543 | 543 | return ( | |
| 544 | 544 | <Section |
| 157 | 157 | { | |
| 158 | 158 | icon: <ScrollText size={18} />, | |
| 159 | 159 | title: "Audit log on every workspace", | |
| 160 | − | about: "Every action by people, tokens and agents, with whether it was allowed and the rule that decided. Kept 90 days, with export.", | |
| 160 | + | about: "Every action by people, tokens and agents, with whether it was allowed and the rule that decided. Kept 90 days on the plan and 7 free, with export.", | |
| 161 | 161 | to: `${DOCS}/guides/audit-log/`, | |
| 162 | 162 | }, | |
| 163 | 163 | ]; |
| 11 | 11 | export const audit = auditClient(instrumented("events", env.EVENTS)); | |
| 12 | 12 | ||
| 13 | 13 | /** | |
| 14 | − | * How many days of the workspace's log are kept: 90, the same on every | |
| 15 | − | * plan. Null when billing cannot say, and then nothing is held back. | |
| 14 | + | * How many days of the workspace's log are kept: 7 for a free workspace, | |
| 15 | + | * 90 on the plan, or what g1t staff set for its account. Null when billing | |
| 16 | + | * cannot say, and then nothing is held back. | |
| 16 | 17 | */ | |
| 17 | 18 | export async function auditRetention(workspace: string): Promise<number | null> { | |
| 18 | 19 | const found = await billing.entitlements(workspace.toLowerCase()).catch(() => null); |
| 93 | 93 | ||
| 94 | 94 | test("the log reads back only as far as the plan keeps it", () => { | |
| 95 | 95 | const now = Date.parse("2026-10-31T00:00:00.000Z"); | |
| 96 | − | // 90 days, on every plan. | |
| 96 | + | // 90 days on the plan, 7 free. | |
| 97 | 97 | assert.equal(retainedSince(null, 90, now), "2026-08-02T00:00:00.000Z"); | |
| 98 | + | assert.equal(retainedSince(null, 7, now), "2026-10-24T00:00:00.000Z"); | |
| 98 | 99 | assert.equal(retainedSince(null, 30, now), "2026-10-01T00:00:00.000Z"); | |
| 99 | 100 | assert.equal(retainedSince("2026-01-01T00:00:00.000Z", 30, now), "2026-10-01T00:00:00.000Z"); | |
| 100 | 101 | // A later start than the window is kept. |
| 26 | 26 | ||
| 27 | 27 | /** | |
| 28 | 28 | * The earliest time a workspace's log can be read from, given how many | |
| 29 | − | * days are kept (90 on every plan): the later of what was asked | |
| 30 | − | * for and the start of the window. | |
| 29 | + | * days are kept (7 free, 90 on the plan, or what staff set): the later of | |
| 30 | + | * what was asked for and the start of the window. | |
| 31 | 31 | */ | |
| 32 | 32 | export function retainedSince(since: string | null | undefined, days: number, now = Date.now()): string { | |
| 33 | 33 | const start = new Date(now - days * 24 * 60 * 60 * 1000).toISOString(); |
| 49 | 49 | ossPoolMicros: 25_000_000, | |
| 50 | 50 | ossRepoMicros: 2_000_000, | |
| 51 | 51 | freePrivateStorageBytes: 1_000_000_000, | |
| 52 | − | auditRetentionDays: 90, | |
| 52 | + | auditRetentionDays: 7, | |
| 53 | + | planAuditRetentionDays: 90, | |
| 53 | 54 | minChargeMicros: 5_000_000, | |
| 54 | 55 | gitOperationsIncluded: 50_000, | |
| 55 | 56 | paidStartCeilingMicros: 100_000_000, | |
| 143 | 144 | { | |
| 144 | 145 | what: "Audit log", | |
| 145 | 146 | free: `${tier.auditRetentionDays} days, with export`, | |
| 146 | − | plan: `${tier.auditRetentionDays} days, with export`, | |
| 147 | + | plan: `${tier.planAuditRetentionDays} days, with export`, | |
| 148 | + | note: "Longer by arrangement. Older entries are deleted each day.", | |
| 147 | 149 | }, | |
| 148 | 150 | { what: "Secret push protection", free: "Included", plan: "Included" }, | |
| 149 | 151 | { what: "Single sign-on", free: "On every plan, once it is built", plan: "On every plan, once it is built" }, | |
| 460 | 462 | <li> | |
| 461 | 463 | <p className="font-medium">Security on every plan</p> | |
| 462 | 464 | <p className="text-muted"> | |
| 463 | − | The audit log for {tier.auditRetentionDays} days with export, and secret push protection, for every workspace. | |
| 465 | + | The audit log with export, {tier.planAuditRetentionDays} days or longer by arrangement, and secret push | |
| 466 | + | protection, for every workspace. | |
| 464 | 467 | </p> | |
| 465 | 468 | </li> | |
| 466 | 469 | <li> |
| 91 | 91 | ? " As an owner you see the whole workspace." | |
| 92 | 92 | : " As a member you see what was done to the workspace's projects, and what was done by you or on your behalf."} | |
| 93 | 93 | {retention != null && | |
| 94 | − | ` The log goes back ${retention} days, and exports the same, on every plan.`} | |
| 94 | + | ` The log goes back ${retention} days, and exports the same; older entries are deleted each day.`} | |
| 95 | 95 | </p> | |
| 96 | 96 | ||
| 97 | 97 | <Form method="get" className="mt-6 rounded-xl border border-line bg-surface p-4"> |
| 716 | 716 | /// it, the plan pays at cost plus the margin; a free workspace's pushes | |
| 717 | 717 | /// to private repositories stop instead. | |
| 718 | 718 | pub free_private_storage_bytes: i64, | |
| 719 | − | /// Days of audit log, the same on every plan. | |
| 719 | + | /// Days of audit log a free workspace keeps. | |
| 720 | 720 | pub audit_retention_days: u32, | |
| 721 | + | /// Days of audit log the g1t plan keeps, and g1t's own and enterprise | |
| 722 | + | /// workspaces. Longer is by arrangement, set per account in sudo. | |
| 723 | + | #[serde(default)] | |
| 724 | + | pub plan_audit_retention_days: u32, | |
| 721 | 725 | /// The smallest amount a card is charged when a month closes; less | |
| 722 | 726 | /// carries over. Charges at a limit always go through. | |
| 723 | 727 | pub min_charge_micros: i64, | |
| 788 | 792 | /// `ISSUE_CAP_MICROS` and the owners' own. None: theirs, or the default. | |
| 789 | 793 | #[serde(default)] | |
| 790 | 794 | pub issue_cap_micros: Option<i64>, | |
| 795 | + | /// Days of audit log its workspaces keep, in place of the plan's (7 | |
| 796 | + | /// free, 90 on the plan), longer or shorter. None: the plan's. | |
| 797 | + | #[serde(default)] | |
| 798 | + | pub audit_retention_days: Option<u32>, | |
| 791 | 799 | /// A hold g1t staff put on new compute, with why. None: no hold. | |
| 792 | 800 | #[serde(default)] | |
| 793 | 801 | pub hold: Option<String>, | |
| 921 | 929 | pub workspace: String, | |
| 922 | 930 | } | |
| 923 | 931 | ||
| 932 | + | /// `audit_retention`: how many days of audit log each workspace keeps, for | |
| 933 | + | /// the events service's daily purge. Takes `AuditRetentionArgs`; returns | |
| 934 | + | /// `Vec<AuditRetention>`, one for each workspace asked about. | |
| 935 | + | #[derive(Debug, Serialize, Deserialize)] | |
| 936 | + | pub struct AuditRetentionArgs { | |
| 937 | + | pub workspaces: Vec<String>, | |
| 938 | + | } | |
| 939 | + | ||
| 940 | + | #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] | |
| 941 | + | pub struct AuditRetention { | |
| 942 | + | pub workspace: String, | |
| 943 | + | pub days: u32, | |
| 944 | + | } | |
| 945 | + | ||
| 924 | 946 | #[derive(Clone, Debug, Serialize, Deserialize)] | |
| 925 | 947 | #[serde(rename_all = "camelCase")] | |
| 926 | 948 | pub struct Entitlements { | |
| 973 | 995 | pub included_micros: i64, | |
| 974 | 996 | #[serde(default)] | |
| 975 | 997 | pub included_used_micros: i64, | |
| 976 | − | /// How far back the audit log can be read and exported: the same on | |
| 977 | − | /// every plan. | |
| 998 | + | /// How far back the audit log can be read and exported, and what is | |
| 999 | + | /// kept: the plan's days, or what g1t staff set for the account. | |
| 978 | 1000 | pub audit_retention_days: u32, | |
| 1001 | + | /// Whether `audit_retention_days` is what staff set for the account | |
| 1002 | + | /// rather than the plan's. | |
| 1003 | + | #[serde(default)] | |
| 1004 | + | pub audit_retention_custom: bool, | |
| 979 | 1005 | /// Private repository storage that is free for every workspace: past | |
| 980 | 1006 | /// it, the plan pays for it and a free workspace's pushes stop. | |
| 981 | 1007 | pub free_private_storage_bytes: i64, |
| 127 | 127 | runCapMicros?: number | null; | |
| 128 | 128 | /** What one issue's agents may spend in all, in place of the owners' and the default $10; null for none. */ | |
| 129 | 129 | issueCapMicros?: number | null; | |
| 130 | + | /** Days of audit log its workspaces keep, in place of the plan's (7 free, 90 on the plan), longer or shorter; null for the plan's. */ | |
| 131 | + | auditRetentionDays?: number | null; | |
| 130 | 132 | /** A hold on new compute, with why; null for none. */ | |
| 131 | 133 | hold?: string | null; | |
| 132 | 134 | }; | |
| 209 | 211 | /** The plan's included usage each month, and what of it is used. */ | |
| 210 | 212 | includedMicros?: number; | |
| 211 | 213 | includedUsedMicros?: number; | |
| 212 | − | /** How far back the audit log can be read and exported: the same on every plan. */ | |
| 214 | + | /** How far back the audit log can be read and exported, and what is kept: the plan's days, or what g1t staff set for the account. */ | |
| 213 | 215 | auditRetentionDays: number; | |
| 216 | + | /** Whether `auditRetentionDays` is what staff set for the account rather than the plan's. */ | |
| 217 | + | auditRetentionCustom?: boolean; | |
| 214 | 218 | /** Private repository storage free for every workspace: past it, the plan pays and a free workspace's pushes stop. */ | |
| 215 | 219 | freePrivateStorageBytes: number; | |
| 216 | 220 | /** The last daily measure of the workspace's private repositories (a lower bound). */ | |
| 680 | 684 | ossRepoMicros: number; | |
| 681 | 685 | /** Private repository storage free for every workspace. Past it, the plan pays; a free workspace's pushes stop. */ | |
| 682 | 686 | freePrivateStorageBytes: number; | |
| 683 | − | /** Days of audit log, the same on every plan. */ | |
| 687 | + | /** Days of audit log a free workspace keeps. */ | |
| 684 | 688 | auditRetentionDays: number; | |
| 689 | + | /** Days of audit log the g1t plan keeps, and g1t's own and enterprise workspaces; longer by arrangement. */ | |
| 690 | + | planAuditRetentionDays?: number; | |
| 685 | 691 | /** The smallest amount a card is charged when a month closes; less carries over. */ | |
| 686 | 692 | minChargeMicros: number; | |
| 687 | 693 | /** Git operations free for every workspace each month. Past it, the plan pays; a free workspace is slowed down. */ |
| 1 | + | -- Days of audit log an account's workspaces keep, set by g1t staff in | |
| 2 | + | -- sudo in place of the plan's (7 free, 90 on the plan). NULL: the plan's. | |
| 3 | + | ALTER TABLE billing_accounts ADD COLUMN audit_retention_days INTEGER; |
| 56 | 56 | #[serde(default)] | |
| 57 | 57 | issue_cap_micros: Option<i64>, | |
| 58 | 58 | #[serde(default)] | |
| 59 | + | audit_retention_days: Option<u32>, | |
| 60 | + | #[serde(default)] | |
| 59 | 61 | hold: Option<String>, | |
| 60 | 62 | } | |
| 61 | 63 | ||
| 69 | 71 | max_concurrent_agents: self.max_concurrent_agents, | |
| 70 | 72 | run_cap_micros: self.run_cap_micros, | |
| 71 | 73 | issue_cap_micros: self.issue_cap_micros, | |
| 74 | + | audit_retention_days: self.audit_retention_days, | |
| 72 | 75 | hold: self.hold.clone().filter(|h| !h.trim().is_empty()), | |
| 73 | 76 | } | |
| 74 | 77 | } | |
| 505 | 508 | if a.allowances.max_concurrent_agents.is_some_and(|n| n == 0 || n > 1_000) { | |
| 506 | 509 | return Ok(Outcome::fail(FailureCode::Invalid, "Agents at once is between 1 and 1,000.")); | |
| 507 | 510 | } | |
| 511 | + | if let Some(why) = crate::retention::invalid_days(&self.plans, a.allowances.audit_retention_days) { | |
| 512 | + | return Ok(Outcome::fail(FailureCode::Invalid, why)); | |
| 513 | + | } | |
| 508 | 514 | let Some(account) = self.find_account(&a.id).await? else { | |
| 509 | 515 | return Ok(Outcome::fail(FailureCode::NotFound, "No such account.")); | |
| 510 | 516 | }; | |
| 514 | 520 | self.db | |
| 515 | 521 | .prepare( | |
| 516 | 522 | "INSERT INTO billing_accounts (id, kind, name, terms_kind, discount_percent, note, created_by, created_at, | |
| 517 | − | team_granted, oss_repo_micros, trial_micros, max_concurrent_agents, run_cap_micros, hold, issue_cap_micros) | |
| 518 | − | VALUES (?1, ?2, ?3, 'standard', 0, '', ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12) | |
| 523 | + | team_granted, oss_repo_micros, trial_micros, max_concurrent_agents, run_cap_micros, hold, issue_cap_micros, | |
| 524 | + | audit_retention_days) | |
| 525 | + | VALUES (?1, ?2, ?3, 'standard', 0, '', ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13) | |
| 519 | 526 | ON CONFLICT (id) DO UPDATE SET team_granted = ?6, oss_repo_micros = ?7, trial_micros = ?8, | |
| 520 | − | max_concurrent_agents = ?9, run_cap_micros = ?10, hold = ?11, issue_cap_micros = ?12", | |
| 527 | + | max_concurrent_agents = ?9, run_cap_micros = ?10, hold = ?11, issue_cap_micros = ?12, | |
| 528 | + | audit_retention_days = ?13", | |
| 521 | 529 | ) | |
| 522 | 530 | .bind(&[ | |
| 523 | 531 | account.id.as_str().into(), | |
| 532 | 540 | opt(a.allowances.run_cap_micros), | |
| 533 | 541 | optional(a.allowances.hold.as_deref().map(str::trim).filter(|h| !h.is_empty())), | |
| 534 | 542 | opt(a.allowances.issue_cap_micros), | |
| 543 | + | a.allowances.audit_retention_days.map_or(JsValue::NULL, JsValue::from), | |
| 535 | 544 | ])? | |
| 536 | 545 | .run() | |
| 537 | 546 | .await?; | |
| 713 | 722 | if let Some(m) = a.issue_cap_micros { | |
| 714 | 723 | parts.push(format!("issue cap {}", crate::features::dollars(m))); | |
| 715 | 724 | } | |
| 725 | + | if let Some(days) = a.audit_retention_days { | |
| 726 | + | parts.push(format!("audit log {days} days")); | |
| 727 | + | } | |
| 716 | 728 | if let Some(hold) = &a.hold { | |
| 717 | 729 | parts.push(format!("hold: {hold}")); | |
| 718 | 730 | } | |
| 764 | 776 | max_concurrent_agents: Some(4), | |
| 765 | 777 | run_cap_micros: Some(3_000_000), | |
| 766 | 778 | issue_cap_micros: None, | |
| 779 | + | audit_retention_days: Some(365), | |
| 767 | 780 | hold: Some("mining".into()), | |
| 768 | 781 | }; | |
| 769 | 782 | assert_eq!( | |
| 770 | 783 | describe_allowances(&given), | |
| 771 | − | "plan given, open-source share $5.00 a repository, trial $2.00, 4 agents at once, run cap $3.00, hold: mining" | |
| 784 | + | "plan given, open-source share $5.00 a repository, trial $2.00, 4 agents at once, run cap $3.00, audit log 365 days, hold: mining" | |
| 772 | 785 | ); | |
| 773 | 786 | } | |
| 774 | 787 |
| 506 | 506 | prepaid_micros: limit.prepaid_micros, | |
| 507 | 507 | included_micros: if has_plan { self.plans.plan_included_micros } else { 0 }, | |
| 508 | 508 | included_used_micros: included_used, | |
| 509 | − | audit_retention_days: self.plans.audit_days, | |
| 509 | + | audit_retention_days: crate::retention::effective_days(&self.plans, plan, account.allowances.audit_retention_days), | |
| 510 | + | audit_retention_custom: account.allowances.audit_retention_days.is_some(), | |
| 510 | 511 | free_private_storage_bytes: self.plans.free_storage_bytes, | |
| 511 | 512 | private_storage_bytes: stored, | |
| 512 | 513 | oss_paid_micros: oss, |
| 59 | 59 | /// free for every workspace. Past it, the plan pays at cost plus the | |
| 60 | 60 | /// margin; a free workspace's pushes to private repositories stop. | |
| 61 | 61 | pub free_storage_bytes: i64, | |
| 62 | − | /// `AUDIT_RETENTION_DAYS`: the same on every plan. | |
| 62 | + | /// `FREE_AUDIT_RETENTION_DAYS`: days of audit log a free workspace | |
| 63 | + | /// keeps. `AUDIT_RETENTION_DAYS`: the plan's, g1t's own and an | |
| 64 | + | /// enterprise's. `AUDIT_MAX_DAYS`: the most staff can set for an | |
| 65 | + | /// account; the events service deletes everything older regardless. | |
| 66 | + | pub free_audit_days: u32, | |
| 63 | 67 | pub audit_days: u32, | |
| 68 | + | pub audit_max_days: u32, | |
| 64 | 69 | /// `RUN_CAP_MICROS` and `ISSUE_CAP_MICROS`: one run's spend cap, and | |
| 65 | 70 | /// agents' spend on one issue in all. | |
| 66 | 71 | pub run_cap_micros: i64, | |
| 93 | 98 | trial_monthly_pool_micros: 100_000_000, | |
| 94 | 99 | min_charge_micros: 5_000_000, | |
| 95 | 100 | free_storage_bytes: 1_000_000_000, | |
| 101 | + | free_audit_days: 7, | |
| 96 | 102 | audit_days: 90, | |
| 103 | + | audit_max_days: 400, | |
| 97 | 104 | run_cap_micros: g1t_contracts::guardrails::DEFAULT_RUN_CAP_MICROS, | |
| 98 | 105 | issue_cap_micros: 10_000_000, | |
| 99 | 106 | paid_start_micros: 100_000_000, | |
| 120 | 127 | trial_monthly_pool_micros: number("TRIAL_MONTHLY_POOL_MICROS", d.trial_monthly_pool_micros), | |
| 121 | 128 | min_charge_micros: number("MIN_CHARGE_MICROS", d.min_charge_micros), | |
| 122 | 129 | free_storage_bytes: number("FREE_PRIVATE_STORAGE_BYTES", d.free_storage_bytes), | |
| 123 | − | audit_days: number("AUDIT_RETENTION_DAYS", d.audit_days.into()) as u32, | |
| 130 | + | free_audit_days: number("FREE_AUDIT_RETENTION_DAYS", d.free_audit_days.into()).max(1) as u32, | |
| 131 | + | audit_days: number("AUDIT_RETENTION_DAYS", d.audit_days.into()).max(1) as u32, | |
| 132 | + | audit_max_days: number("AUDIT_MAX_DAYS", d.audit_max_days.into()).max(1) as u32, | |
| 124 | 133 | run_cap_micros: number("RUN_CAP_MICROS", d.run_cap_micros), | |
| 125 | 134 | issue_cap_micros: number("ISSUE_CAP_MICROS", d.issue_cap_micros), | |
| 126 | 135 | paid_start_micros: number("LIMIT_PAID_START_MICROS", d.paid_start_micros), | |
| 760 | 769 | assert_eq!(c.trial_monthly_pool_micros, 100_000_000); | |
| 761 | 770 | assert_eq!(c.min_charge_micros, 5_000_000); | |
| 762 | 771 | assert_eq!(c.free_storage_bytes, 1_000_000_000); | |
| 772 | + | assert_eq!(c.free_audit_days, 7); | |
| 763 | 773 | assert_eq!(c.audit_days, 90); | |
| 774 | + | assert_eq!(c.audit_max_days, 400); | |
| 764 | 775 | assert_eq!(c.run_cap_micros, g1t_contracts::guardrails::DEFAULT_RUN_CAP_MICROS); | |
| 765 | 776 | assert_eq!(c.issue_cap_micros, 10_000_000); | |
| 766 | 777 | assert_eq!(c.paid_start_micros, 100_000_000); |
| 420 | 420 | oss_pool_micros: self.plans.oss_pool_micros, | |
| 421 | 421 | oss_repo_micros: self.plans.oss_repo_micros, | |
| 422 | 422 | free_private_storage_bytes: self.plans.free_storage_bytes, | |
| 423 | − | audit_retention_days: self.plans.audit_days, | |
| 423 | + | audit_retention_days: self.plans.free_audit_days, | |
| 424 | + | plan_audit_retention_days: self.plans.audit_days, | |
| 424 | 425 | min_charge_micros: self.plans.min_charge_micros, | |
| 425 | 426 | git_operations_included: self.plans.git_included, | |
| 426 | 427 | paid_start_ceiling_micros: self.plans.paid_start_micros, |
| 37 | 37 | mod keeper; | |
| 38 | 38 | mod limits; | |
| 39 | 39 | mod rename; | |
| 40 | + | mod retention; | |
| 40 | 41 | mod stripe; | |
| 41 | 42 | mod stripe_sync; | |
| 42 | 43 | ||
| 1153 | 1154 | "admin_credit" => reply(&billing.admin_credit(args(body)?).await?), | |
| 1154 | 1155 | "admin_set_allowances" => reply(&billing.admin_set_allowances(args(body)?).await?), | |
| 1155 | 1156 | "entitlements" => reply(&billing.entitlements(args(body)?).await?), | |
| 1157 | + | "audit_retention" => reply(&billing.audit_retention(args(body)?).await?), | |
| 1156 | 1158 | "reserve" => reply(&billing.reserve(args(body)?).await?), | |
| 1157 | 1159 | "settle" => reply(&billing.settle_reservation(args(body)?).await?), | |
| 1158 | 1160 | "card_check" => reply(&billing.card_check(args(body)?).await?), |
| 1 | + | //! How long each workspace's audit log is kept. | |
| 2 | + | //! | |
| 3 | + | //! A free workspace keeps `FREE_AUDIT_RETENTION_DAYS` (7); the g1t plan, | |
| 4 | + | //! g1t's own workspaces and an enterprise's keep `AUDIT_RETENTION_DAYS` | |
| 5 | + | //! (90). Staff can set an account's own number in sudo, such as for an | |
| 6 | + | //! organization that pays for longer, up to `AUDIT_MAX_DAYS` (400). What | |
| 7 | + | //! staff set wins over the plan's either way. The events service asks for | |
| 8 | + | //! these once a day (`audit_retention`) and deletes what is older. | |
| 9 | + | ||
| 10 | + | use futures_util::future::try_join_all; | |
| 11 | + | use g1t_contracts::billing::{AuditRetention, AuditRetentionArgs, PlanKind}; | |
| 12 | + | use worker::Result; | |
| 13 | + | ||
| 14 | + | use crate::Billing; | |
| 15 | + | use crate::credits::Config; | |
| 16 | + | ||
| 17 | + | /// Days of audit log a workspace keeps: what staff set for its account, | |
| 18 | + | /// or else its plan's. | |
| 19 | + | pub(crate) fn effective_days(plans: &Config, plan: PlanKind, custom: Option<u32>) -> u32 { | |
| 20 | + | custom.unwrap_or(match plan { | |
| 21 | + | PlanKind::Free => plans.free_audit_days, | |
| 22 | + | PlanKind::Paid | PlanKind::Internal | PlanKind::Enterprise => plans.audit_days, | |
| 23 | + | }) | |
| 24 | + | } | |
| 25 | + | ||
| 26 | + | /// Why a number staff typed cannot be an account's retention, or None | |
| 27 | + | /// when it can. | |
| 28 | + | pub(crate) fn invalid_days(plans: &Config, days: Option<u32>) -> Option<String> { | |
| 29 | + | days.filter(|d| !(1..=plans.audit_max_days).contains(d)) | |
| 30 | + | .map(|_| format!("Audit log days is between 1 and {}, or empty for the plan's.", plans.audit_max_days)) | |
| 31 | + | } | |
| 32 | + | ||
| 33 | + | impl Billing { | |
| 34 | + | /// `audit_retention`: each workspace's days, its account and plan read | |
| 35 | + | /// once and all of them at the same time. | |
| 36 | + | pub(crate) async fn audit_retention(&self, a: AuditRetentionArgs) -> Result<Vec<AuditRetention>> { | |
| 37 | + | try_join_all(a.workspaces.iter().map(|workspace| async move { | |
| 38 | + | let workspace = workspace.to_lowercase(); | |
| 39 | + | let account = self.account_of(&workspace).await?; | |
| 40 | + | let plan = self.plan_kind_for(&workspace, &account).await?; | |
| 41 | + | let days = effective_days(&self.plans, plan, account.allowances.audit_retention_days); | |
| 42 | + | Ok::<_, worker::Error>(AuditRetention { workspace, days }) | |
| 43 | + | })) | |
| 44 | + | .await | |
| 45 | + | } | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | #[cfg(test)] | |
| 49 | + | mod tests { | |
| 50 | + | use super::*; | |
| 51 | + | ||
| 52 | + | #[test] | |
| 53 | + | fn free_keeps_a_week_and_the_plan_ninety_days() { | |
| 54 | + | let plans = Config::default(); | |
| 55 | + | assert_eq!(effective_days(&plans, PlanKind::Free, None), 7); | |
| 56 | + | assert_eq!(effective_days(&plans, PlanKind::Paid, None), 90); | |
| 57 | + | assert_eq!(effective_days(&plans, PlanKind::Internal, None), 90); | |
| 58 | + | assert_eq!(effective_days(&plans, PlanKind::Enterprise, None), 90); | |
| 59 | + | } | |
| 60 | + | ||
| 61 | + | #[test] | |
| 62 | + | fn what_staff_set_wins_either_way() { | |
| 63 | + | let plans = Config::default(); | |
| 64 | + | assert_eq!(effective_days(&plans, PlanKind::Free, Some(30)), 30); | |
| 65 | + | assert_eq!(effective_days(&plans, PlanKind::Paid, Some(365)), 365); | |
| 66 | + | assert_eq!(effective_days(&plans, PlanKind::Enterprise, Some(14)), 14); | |
| 67 | + | } | |
| 68 | + | ||
| 69 | + | #[test] | |
| 70 | + | fn staff_set_between_one_day_and_the_most() { | |
| 71 | + | let plans = Config::default(); | |
| 72 | + | assert_eq!(invalid_days(&plans, None), None); | |
| 73 | + | assert_eq!(invalid_days(&plans, Some(1)), None); | |
| 74 | + | assert_eq!(invalid_days(&plans, Some(400)), None); | |
| 75 | + | assert_eq!( | |
| 76 | + | invalid_days(&plans, Some(0)).as_deref(), | |
| 77 | + | Some("Audit log days is between 1 and 400, or empty for the plan's.") | |
| 78 | + | ); | |
| 79 | + | assert!(invalid_days(&plans, Some(401)).is_some()); | |
| 80 | + | } | |
| 81 | + | } |
| 54 | 54 | "PLAN_INCLUDED_MICROS": "10000000", | |
| 55 | 55 | // 1 GB of private storage free for every workspace: past it, the plan | |
| 56 | 56 | // pays at cost plus the margin, and a free workspace's pushes to | |
| 57 | − | // private repositories stop. The audit log is kept 90 days on every plan. | |
| 57 | + | // private repositories stop. | |
| 58 | 58 | "FREE_PRIVATE_STORAGE_BYTES": "1000000000", | |
| 59 | + | // The audit log (src/retention.rs): 7 days for a free workspace, 90 on | |
| 60 | + | // the plan, for g1t's own and for an enterprise. Staff can set an | |
| 61 | + | // account's own days in sudo, up to AUDIT_MAX_DAYS; keep that at most | |
| 62 | + | // the events service's AUDIT_MAX_DAYS, which deletes anything older. | |
| 63 | + | "FREE_AUDIT_RETENTION_DAYS": "7", | |
| 59 | 64 | "AUDIT_RETENTION_DAYS": "90", | |
| 65 | + | "AUDIT_MAX_DAYS": "400", | |
| 60 | 66 | // The trial (src/credits.rs, src/cards.rs): $5 of usage once per new | |
| 61 | 67 | // workspace, granted after a card check (one per card), out of a pool | |
| 62 | 68 | // for everyone ($100) that resets each calendar month (UTC). Either at |
| 1 | + | -- Audit entries are kept by plan (src/audit.rs): 7 days for a free | |
| 2 | + | -- workspace, 90 on the plan, or what g1t staff set for its account, and | |
| 3 | + | -- never past AUDIT_MAX_DAYS. The daily purge deletes everything older than | |
| 4 | + | -- the ceiling by time, finds the workspaces with entries older than the | |
| 5 | + | -- shortest retention, and deletes each one's older than its own days; these | |
| 6 | + | -- indexes keep each of those a seek rather than a scan. | |
| 7 | + | CREATE INDEX IF NOT EXISTS audit_workspace_time ON audit_entries (workspace, time); | |
| 8 | + | CREATE INDEX IF NOT EXISTS audit_time ON audit_entries (time); | |
| 9 | + | ||
| 10 | + | -- Where the last daily run stopped, so the next carries on from there: one | |
| 11 | + | -- run looks at a bounded number of workspaces. Empty: from the start. | |
| 12 | + | CREATE TABLE IF NOT EXISTS audit_purge_cursor ( | |
| 13 | + | id INTEGER PRIMARY KEY CHECK (id = 1), | |
| 14 | + | after TEXT NOT NULL DEFAULT '' | |
| 15 | + | ); |
| 8 | 8 | AuditEntry, AuditPage, AuditVisibility, ListAuditArgs, MAX_AUDIT_PAGE, NewAuditEntry, | |
| 9 | 9 | RecordAuditArgs, | |
| 10 | 10 | }; | |
| 11 | + | use g1t_contracts::billing::{AuditRetention, AuditRetentionArgs}; | |
| 11 | 12 | use g1t_contracts::events::{Event, WorkspaceRenamed}; | |
| 12 | 13 | use g1t_contracts::new_id; | |
| 13 | 14 | use g1t_contracts::time::rfc3339; | |
| 14 | 15 | use g1t_kit::now_ms; | |
| 15 | 16 | use serde::Deserialize; | |
| 16 | 17 | use worker::wasm_bindgen::JsValue; | |
| 17 | − | use worker::{D1Database, Result}; | |
| 18 | + | use worker::{D1Database, Fetcher, Result}; | |
| 18 | 19 | ||
| 19 | 20 | const DEFAULT_PAGE: u32 = 100; | |
| 20 | 21 | /// More than one request ever records. | |
| 277 | 278 | Ok(AuditPage { entries, next }) | |
| 278 | 279 | } | |
| 279 | 280 | ||
| 280 | − | /// Entries are kept this many days unless `AUDIT_KEEP_DAYS` says otherwise: | |
| 281 | − | /// what the audit log reads back, the same on every plan (90 days). | |
| 282 | − | pub const DEFAULT_KEEP_DAYS: u32 = 90; | |
| 281 | + | /// No entry is kept longer than this, whatever its workspace's plan, unless | |
| 282 | + | /// `AUDIT_MAX_DAYS` says otherwise. Keep it at least billing's | |
| 283 | + | /// `AUDIT_MAX_DAYS`, the most staff can set for an account. | |
| 284 | + | pub const DEFAULT_MAX_DAYS: u32 = 400; | |
| 285 | + | /// The shortest any workspace keeps (`AUDIT_MIN_DAYS`, a free workspace's | |
| 286 | + | /// 7 days): only workspaces with entries older than this are asked about. | |
| 287 | + | pub const DEFAULT_MIN_DAYS: u32 = 7; | |
| 288 | + | /// Workspaces looked at in one daily run, so one run never runs long; the | |
| 289 | + | /// next run carries on after the last one. | |
| 290 | + | pub const WORKSPACES_PER_RUN: u32 = 200; | |
| 291 | + | /// Workspaces asked about in one call to billing, which reads each one's | |
| 292 | + | /// account and plan. | |
| 293 | + | const ASK_AT_ONCE: usize = 50; | |
| 283 | 294 | /// Rows removed per statement, so one purge never runs long. | |
| 284 | 295 | const PURGE_BATCH: u32 = 5_000; | |
| 285 | 296 | ||
| 288 | 299 | g1t_contracts::time::rfc3339(now_ms.saturating_sub(u64::from(keep_days) * 24 * 60 * 60 * 1000)) | |
| 289 | 300 | } | |
| 290 | 301 | ||
| 302 | + | /// Each workspace with the time its entries are kept from. Its days are | |
| 303 | + | /// held between the shortest and the longest any workspace keeps: fewer | |
| 304 | + | /// than the shortest would not be looked for, and more than the longest | |
| 305 | + | /// are deleted anyway. | |
| 306 | + | pub fn cutoffs( | |
| 307 | + | now_ms: u64, | |
| 308 | + | retention: &[AuditRetention], | |
| 309 | + | min_days: u32, | |
| 310 | + | max_days: u32, | |
| 311 | + | ) -> Vec<(String, String)> { | |
| 312 | + | retention | |
| 313 | + | .iter() | |
| 314 | + | .map(|r| { | |
| 315 | + | let days = r.days.max(min_days).min(max_days); | |
| 316 | + | (r.workspace.clone(), keep_from(now_ms, days)) | |
| 317 | + | }) | |
| 318 | + | .collect() | |
| 319 | + | } | |
| 320 | + | ||
| 291 | 321 | /// Removes entries older than every plan keeps, a batch at a time, up to | |
| 292 | 322 | /// `rounds` batches. Returns how many went. | |
| 293 | 323 | pub async fn purge(db: &D1Database, before: &str, rounds: u32) -> Result<u32> { | |
| 310 | 340 | Ok(removed) | |
| 311 | 341 | } | |
| 312 | 342 | ||
| 343 | + | /// Removes one workspace's entries older than `before`, the same way. | |
| 344 | + | async fn purge_workspace( | |
| 345 | + | db: &D1Database, | |
| 346 | + | workspace: &str, | |
| 347 | + | before: &str, | |
| 348 | + | rounds: u32, | |
| 349 | + | ) -> Result<u32> { | |
| 350 | + | let mut removed = 0; | |
| 351 | + | for _ in 0..rounds { | |
| 352 | + | let result = db | |
| 353 | + | .prepare( | |
| 354 | + | "DELETE FROM audit_entries WHERE id IN | |
| 355 | + | (SELECT id FROM audit_entries WHERE workspace = ? AND time < ? ORDER BY time LIMIT ?)", | |
| 356 | + | ) | |
| 357 | + | .bind(&[workspace.into(), before.into(), PURGE_BATCH.into()])? | |
| 358 | + | .run() | |
| 359 | + | .await?; | |
| 360 | + | let changed = result.meta()?.and_then(|meta| meta.changes).unwrap_or(0) as u32; | |
| 361 | + | removed += changed; | |
| 362 | + | if changed < PURGE_BATCH { | |
| 363 | + | break; | |
| 364 | + | } | |
| 365 | + | } | |
| 366 | + | Ok(removed) | |
| 367 | + | } | |
| 368 | + | ||
| 369 | + | /// Up to `limit` workspaces after `after`, in order, that have entries | |
| 370 | + | /// older than `before`. Each step seeks the next workspace in the index on | |
| 371 | + | /// (workspace, time) rather than reading every row, which a DISTINCT over | |
| 372 | + | /// the rows older than a week would: a workspace on the plan always has | |
| 373 | + | /// weeks of them. | |
| 374 | + | async fn workspaces_past( | |
| 375 | + | db: &D1Database, | |
| 376 | + | after: &str, | |
| 377 | + | before: &str, | |
| 378 | + | limit: u32, | |
| 379 | + | ) -> Result<Vec<String>> { | |
| 380 | + | #[derive(Deserialize)] | |
| 381 | + | struct Found { | |
| 382 | + | workspace: String, | |
| 383 | + | } | |
| 384 | + | Ok(db | |
| 385 | + | .prepare( | |
| 386 | + | "WITH RECURSIVE w(workspace) AS ( | |
| 387 | + | SELECT (SELECT MIN(workspace) FROM audit_entries WHERE workspace > ?1) | |
| 388 | + | UNION ALL | |
| 389 | + | SELECT (SELECT MIN(e.workspace) FROM audit_entries e WHERE e.workspace > w.workspace) | |
| 390 | + | FROM w WHERE w.workspace IS NOT NULL | |
| 391 | + | ) | |
| 392 | + | SELECT workspace FROM w | |
| 393 | + | WHERE workspace IS NOT NULL | |
| 394 | + | AND EXISTS (SELECT 1 FROM audit_entries a WHERE a.workspace = w.workspace AND a.time < ?2) | |
| 395 | + | LIMIT ?3", | |
| 396 | + | ) | |
| 397 | + | .bind(&[after.into(), before.into(), limit.into()])? | |
| 398 | + | .all() | |
| 399 | + | .await? | |
| 400 | + | .results::<Found>()? | |
| 401 | + | .into_iter() | |
| 402 | + | .map(|found| found.workspace) | |
| 403 | + | .collect()) | |
| 404 | + | } | |
| 405 | + | ||
| 406 | + | /// Removes each workspace's entries older than its plan keeps, for up to | |
| 407 | + | /// `WORKSPACES_PER_RUN` workspaces after where the last run stopped. Their | |
| 408 | + | /// days come from billing; if it cannot be reached, nothing is removed and | |
| 409 | + | /// the next run tries the same workspaces again. Returns how many went. | |
| 410 | + | pub async fn purge_by_plan( | |
| 411 | + | db: &D1Database, | |
| 412 | + | billing: &Fetcher, | |
| 413 | + | now_ms: u64, | |
| 414 | + | min_days: u32, | |
| 415 | + | max_days: u32, | |
| 416 | + | ) -> Result<u32> { | |
| 417 | + | #[derive(Deserialize)] | |
| 418 | + | struct Cursor { | |
| 419 | + | after: String, | |
| 420 | + | } | |
| 421 | + | let after = db | |
| 422 | + | .prepare("SELECT after FROM audit_purge_cursor WHERE id = 1") | |
| 423 | + | .first::<Cursor>(None) | |
| 424 | + | .await? | |
| 425 | + | .map_or_else(String::new, |cursor| cursor.after); | |
| 426 | + | let found = | |
| 427 | + | workspaces_past(db, &after, &keep_from(now_ms, min_days), WORKSPACES_PER_RUN).await?; | |
| 428 | + | // Every workspace's days are asked for before anything is removed, so a | |
| 429 | + | // billing that cannot be reached removes nothing at all. | |
| 430 | + | let mut retention: Vec<AuditRetention> = Vec::with_capacity(found.len()); | |
| 431 | + | for chunk in found.chunks(ASK_AT_ONCE) { | |
| 432 | + | let args = AuditRetentionArgs { | |
| 433 | + | workspaces: chunk.to_vec(), | |
| 434 | + | }; | |
| 435 | + | let answered: Vec<AuditRetention> = | |
| 436 | + | g1t_kit::call(billing, "audit_retention", &args).await?; | |
| 437 | + | retention.extend(answered); | |
| 438 | + | } | |
| 439 | + | let mut removed = 0; | |
| 440 | + | for (workspace, before) in cutoffs(now_ms, &retention, min_days, max_days) { | |
| 441 | + | removed += purge_workspace(db, &workspace, &before, 4).await?; | |
| 442 | + | } | |
| 443 | + | // A short page means the end was reached: the next run starts over. | |
| 444 | + | let next = if found.len() < WORKSPACES_PER_RUN as usize { | |
| 445 | + | String::new() | |
| 446 | + | } else { | |
| 447 | + | found.last().cloned().unwrap_or_default() | |
| 448 | + | }; | |
| 449 | + | db.prepare( | |
| 450 | + | "INSERT INTO audit_purge_cursor (id, after) VALUES (1, ?1) | |
| 451 | + | ON CONFLICT (id) DO UPDATE SET after = ?1", | |
| 452 | + | ) | |
| 453 | + | .bind(&[next.into()])? | |
| 454 | + | .run() | |
| 455 | + | .await?; | |
| 456 | + | Ok(removed) | |
| 457 | + | } | |
| 458 | + | ||
| 313 | 459 | /// Moves a renamed workspace's rows to its new slug. | |
| 314 | 460 | pub async fn follow_renames(db: &D1Database, events: &[Event]) -> Result<()> { | |
| 315 | 461 | for event in events | |
| 352 | 498 | // 2026-10-05T00:00:00Z, a year back. | |
| 353 | 499 | let now = 1_791_158_400_000; | |
| 354 | 500 | assert_eq!(keep_from(now, 365), "2025-10-05T00:00:00.000Z"); | |
| 355 | − | assert_eq!(DEFAULT_KEEP_DAYS, 90); | |
| 501 | + | assert_eq!(DEFAULT_MAX_DAYS, 400); | |
| 502 | + | assert_eq!(DEFAULT_MIN_DAYS, 7); | |
| 503 | + | } | |
| 504 | + | ||
| 505 | + | #[test] | |
| 506 | + | fn each_workspace_is_cut_off_at_its_own_days() { | |
| 507 | + | // 2026-10-05T00:00:00Z. | |
| 508 | + | let now = 1_791_158_400_000; | |
| 509 | + | let kept = |workspace: &str, days: u32| AuditRetention { | |
| 510 | + | workspace: workspace.into(), | |
| 511 | + | days, | |
| 512 | + | }; | |
| 513 | + | let retention = [ | |
| 514 | + | kept("free", 7), | |
| 515 | + | kept("plan", 90), | |
| 516 | + | kept("longer", 365), | |
| 517 | + | kept("shorter", 1), | |
| 518 | + | kept("past-the-most", 1_000), | |
| 519 | + | ]; | |
| 520 | + | let expected = [ | |
| 521 | + | ("free", "2026-09-28T00:00:00.000Z"), | |
| 522 | + | ("plan", "2026-07-07T00:00:00.000Z"), | |
| 523 | + | ("longer", "2025-10-05T00:00:00.000Z"), | |
| 524 | + | // Fewer days than the shortest are never looked for. | |
| 525 | + | ("shorter", "2026-09-28T00:00:00.000Z"), | |
| 526 | + | // More than the most are deleted by the ceiling anyway. | |
| 527 | + | ("past-the-most", "2025-08-31T00:00:00.000Z"), | |
| 528 | + | ]; | |
| 529 | + | let expected: Vec<(String, String)> = expected | |
| 530 | + | .iter() | |
| 531 | + | .map(|(w, t)| ((*w).to_owned(), (*t).to_owned())) | |
| 532 | + | .collect(); | |
| 533 | + | assert_eq!(cutoffs(now, &retention, 7, 400), expected); | |
| 356 | 534 | } | |
| 357 | 535 | ||
| 358 | 536 | fn args() -> ListAuditArgs { |
| 209 | 209 | Ok(()) | |
| 210 | 210 | } | |
| 211 | 211 | ||
| 212 | − | /// Once a day: audit entries older than the audit log keeps are removed | |
| 213 | − | /// (`AUDIT_KEEP_DAYS`, 90 days by default, the same on every plan). | |
| 212 | + | /// Once a day, old audit entries are removed: first everything older than | |
| 213 | + | /// any workspace keeps (`AUDIT_MAX_DAYS`, 400 days), then each workspace's | |
| 214 | + | /// entries older than its own plan keeps, as billing says (7 days free, 90 | |
| 215 | + | /// on the plan, or what staff set). Workspaces with nothing older than the | |
| 216 | + | /// shortest (`AUDIT_MIN_DAYS`, 7) are left alone. | |
| 214 | 217 | #[event(scheduled)] | |
| 215 | 218 | async fn scheduled(_event: worker::ScheduledEvent, env: Env, _ctx: worker::ScheduleContext) { | |
| 216 | − | let keep_days = env | |
| 217 | − | .var("AUDIT_KEEP_DAYS") | |
| 218 | − | .ok() | |
| 219 | − | .and_then(|v| v.to_string().parse().ok()) | |
| 220 | − | .unwrap_or(audit::DEFAULT_KEEP_DAYS); | |
| 219 | + | let days = |name: &str, default: u32| { | |
| 220 | + | env.var(name) | |
| 221 | + | .ok() | |
| 222 | + | .and_then(|v| v.to_string().trim().parse::<u32>().ok()) | |
| 223 | + | .filter(|days| *days > 0) | |
| 224 | + | .unwrap_or(default) | |
| 225 | + | }; | |
| 226 | + | let max_days = days("AUDIT_MAX_DAYS", audit::DEFAULT_MAX_DAYS); | |
| 227 | + | let min_days = days("AUDIT_MIN_DAYS", audit::DEFAULT_MIN_DAYS).min(max_days); | |
| 221 | 228 | let Ok(db) = env.d1("DB") else { return }; | |
| 222 | − | match audit::purge(&db, &audit::keep_from(now_ms(), keep_days), 20).await { | |
| 223 | − | Ok(removed) if removed > 0 => worker::console_log!("removed {removed} audit entries older than {keep_days} days"), | |
| 229 | + | let now = now_ms(); | |
| 230 | + | match audit::purge(&db, &audit::keep_from(now, max_days), 20).await { | |
| 231 | + | Ok(removed) if removed > 0 => { | |
| 232 | + | worker::console_log!("removed {removed} audit entries older than {max_days} days") | |
| 233 | + | } | |
| 224 | 234 | Ok(_) => {} | |
| 225 | 235 | Err(error) => worker::console_error!("could not remove old audit entries: {error}"), | |
| 226 | 236 | } | |
| 237 | + | // Without billing nobody's plan is known, so nothing younger than the | |
| 238 | + | // ceiling is removed. | |
| 239 | + | let billing = match env.service("BILLING") { | |
| 240 | + | Ok(billing) => billing, | |
| 241 | + | Err(error) => { | |
| 242 | + | worker::console_error!( | |
| 243 | + | "audit entries kept past their plan's days: no billing: {error}" | |
| 244 | + | ); | |
| 245 | + | return; | |
| 246 | + | } | |
| 247 | + | }; | |
| 248 | + | match audit::purge_by_plan(&db, &billing, now, min_days, max_days).await { | |
| 249 | + | Ok(removed) if removed > 0 => { | |
| 250 | + | worker::console_log!("removed {removed} audit entries older than their plan keeps") | |
| 251 | + | } | |
| 252 | + | Ok(_) => {} | |
| 253 | + | Err(error) => worker::console_error!("audit entries kept past their plan's days: {error}"), | |
| 254 | + | } | |
| 227 | 255 | } |
| 38 | 38 | ], | |
| 39 | 39 | "consumers": [{ "queue": "g1t-events", "max_batch_size": 100, "max_batch_timeout": 1 }] | |
| 40 | 40 | }, | |
| 41 | − | // Once a day, audit entries older than the audit log reads back are | |
| 42 | − | // removed. The same on every plan: keep it at least billing's | |
| 43 | − | // AUDIT_RETENTION_DAYS. | |
| 41 | + | // Each workspace's audit log keeps as many days as billing says (7 free, | |
| 42 | + | // 90 on the plan, or what staff set), asked for over this binding. | |
| 43 | + | "services": [{ "binding": "BILLING", "service": "g1t-billing" }], | |
| 44 | + | // Once a day, audit entries older than AUDIT_MAX_DAYS are removed for | |
| 45 | + | // everyone (keep it at least billing's AUDIT_MAX_DAYS), then each | |
| 46 | + | // workspace's older than its own days. Only workspaces with entries | |
| 47 | + | // older than AUDIT_MIN_DAYS, the shortest any plan keeps, are asked about. | |
| 44 | 48 | "triggers": { "crons": ["41 3 * * *"] }, | |
| 45 | − | "vars": { "AUDIT_KEEP_DAYS": "90" }, | |
| 49 | + | "vars": { "AUDIT_MAX_DAYS": "400", "AUDIT_MIN_DAYS": "7" }, | |
| 46 | 50 | "observability": { "enabled": true } | |
| 47 | 51 | } |