Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 1 | /** |
| 2 | * The front door's rate limits (workers/app.ts), per request before it is | |
| 3 | * answered. The bindings and their limits are in `RATE_LIMITS` | |
| The docs folder is gone, and what it held lives where people read it: how a self-hosted g1t runs and how to deploy g1t to Cloudflare are pages on docs.g1t.sh under Run g1t yourself, and speed, rate limits and operating g1t.sh are sections of CONTRIBUTING.md; code that cited a file in docs/ now points to the page or section that covers it, or says what it means itself, and applied migrations and the runner images are left as they were. | 4 | * (packages/contracts/src/rate-limits.ts); CONTRIBUTING.md, "Rate limits", |
| 5 | * says how they are kept. | |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 6 | * |
| 7 | * - Git over HTTPS: without credentials, by address; with them, by a hash | |
| 8 | * of the credential, much higher. A clone is about three requests. | |
| 9 | * - Pages: every request that reaches the Worker counts against a ceiling | |
| 10 | * per address. Signed out, by address, with a tighter limit on what is | |
| 11 | * costly to answer (archives, run pages, logs, search). Signed in, by a | |
| 12 | * hash of the session cookie, higher: the session is not checked here, | |
| 13 | * which would cost a call to identity, and the ceiling per address keeps | |
| Merge main into Artifacts Phase 2 | 14 | * made-up cookies from getting round the signed-out limit. With an |
| 15 | * access token (`Authorization: Bearer`, app/lib/website-token.ts), by a | |
| 16 | * hash of the token, at the API's limit for a token; the address ceiling | |
| 17 | * applies as for cookies. | |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 18 | * |
| 19 | * Static assets never reach the Worker (the assets binding answers them), | |
| 20 | * and the few files it serves itself are left out here too. Every limit | |
| 21 | * fails open. | |
| 22 | */ | |
| 23 | import { | |
| 24 | type RateLimitBinding, | |
| 25 | checkLimit, | |
| 26 | clientAddress, | |
| 27 | secretKey, | |
| 28 | tooManyRequests, | |
| 29 | } from "@g1t/contracts/rate-limits"; | |
| 30 | ||
| 31 | export type FrontDoorLimits = { | |
| 32 | WEB_ANONYMOUS_LIMIT?: RateLimitBinding; | |
| 33 | WEB_HEAVY_LIMIT?: RateLimitBinding; | |
| 34 | WEB_SESSION_LIMIT?: RateLimitBinding; | |
| 35 | WEB_ADDRESS_LIMIT?: RateLimitBinding; | |
| Merge main into Artifacts Phase 2 | 36 | WEB_TOKEN_LIMIT?: RateLimitBinding; |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 37 | GIT_ANONYMOUS_LIMIT?: RateLimitBinding; |
| 38 | GIT_SIGNED_LIMIT?: RateLimitBinding; | |
| 39 | }; | |
| 40 | ||
| 41 | /** Files the Worker serves that are never limited: build output, fonts, and top-level files such as robots.txt. */ | |
| 42 | const UNLIMITED = /^\/(?:assets\/|fonts\/|favicon|[^/]+\.(?:ico|png|svg|txt|xml|webmanifest)$)/; | |
| 43 | ||
| A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so. | 44 | /** |
| Merge remote-tracking branch 'origin/claude/cool-newton-7nm8r9' into claude/cool-newton-7nm8r9 | 45 | * What is costly to answer for a signed-out visitor, or must not be flooded: |
| 46 | * a repository's archives, a run's page, log downloads and artifacts, | |
| 47 | * search, and the form that reports a vulnerability to a repository's | |
| 48 | * maintainers privately, which takes no account. A running job's live log | |
| 49 | * (`actions/jobs/:job/log`, a small read the page makes about once a | |
| 50 | * second) counts as a page: with the run's page refreshing beside it, it | |
| 51 | * would use up this limit in half a minute. | |
| A running job's log is live, as on GitHub. A step's output reaches its run's page within about two seconds: the runner sends what is printed every second, and the page asks every second while a step runs, every three while none does; an answer that fails (a 429, a 5xx, the network) is asked again after a wait that doubles to half a minute, or as long as Retry-After says, where before one such answer stopped the log for good and the step said "No output yet." until the page was reloaded, and a signed-out viewer's run page, refreshing beside its log, ran into the 30-a-minute limit within a minute; the live log now counts as a page, and only log downloads as costly. A hidden tab stops asking and catches up when shown, and the page says when it is reconnecting. The running step opens by itself, its timer counts, its log follows the newest line while its end is on screen and offers Jump to latest once the viewer scrolls up, a step that has printed nothing for a while says "Running · no output for 2m 10s", and the job's header names the step it is on. Colour that tools print (bold, dim, italic, underline, the 16 colours as theme tokens for light and dark, 256 colours and 24-bit colour, text and background) is drawn as printed, other escape codes are dropped and links keep their text, and search reads the plain text and keeps the colours around what it marks. Every line is stamped with the time it was printed, as GitHub's raw logs are, so log.txt and the run's zip have a time at the start of each line, and a clock button beside Search logs shows the times, remembered by the browser. Set up job now gives the runner's version and name, the operating system, the runner image, the machine's architecture, CPUs and memory, the token's permissions under G1T_TOKEN Permissions, the secret source and the actions it downloads; each run step opens with GitHub's folded Run group, the script in cyan, the shell's whole path and the step's env, each uses step with its with inputs (an action's defaults among them) and env; cache and artifact transfers say how far they have got every five seconds; and a last Complete job step sets the job's outputs and stops what the steps left running, found by RUNNER_TRACKING_ID as GitHub's runner finds it. Two batches of a job's log written at once can no longer reach for the same seq and lose one. The Actions guide and the rate limits page say so. | 52 | */ |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 53 | const HEAVY = |
| Merge remote-tracking branch 'origin/claude/cool-newton-7nm8r9' into claude/cool-newton-7nm8r9 | 54 | /^\/(?:search(?:\.data)?$|[^/]+\/[^/]+\/(?:archive\/|actions\/runs\/[^/]+(?:\.data|\/logs\.zip|\/artifacts\/[^/]+)?$|actions\/jobs\/[^/]+\/log\.txt$|security\/report(?:\.data)?$))/; |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 55 | |
| 56 | export function unlimited(pathname: string): boolean { | |
| 57 | return UNLIMITED.test(pathname); | |
| 58 | } | |
| 59 | ||
| 60 | export function heavy(pathname: string): boolean { | |
| 61 | return HEAVY.test(pathname); | |
| 62 | } | |
| 63 | ||
| 64 | /** The session cookie's value, or null when signed out. */ | |
| 65 | export function sessionCookie(cookie: string | null): string | null { | |
| 66 | const match = /(?:^|;\s*)g1t_session=([^;]+)/.exec(cookie ?? ""); | |
| 67 | return match?.[1] ?? null; | |
| 68 | } | |
| 69 | ||
| 70 | const GIT_MESSAGE_ANONYMOUS = | |
| 71 | "Too many git requests from your network. Wait a minute and try again, or use credentials for a higher limit: https://docs.g1t.sh/reference/rate-limits/\n"; | |
| 72 | const GIT_MESSAGE_SIGNED = "Too many git requests with these credentials. Wait a minute and try again: https://docs.g1t.sh/reference/rate-limits/\n"; | |
| 73 | const PAGE_MESSAGE = "Too many requests from your network. Wait a minute and try again.\n"; | |
| Merge main into Artifacts Phase 2 | 74 | const TOKEN_PAGE_MESSAGE = "Too many requests with this access token. Wait a minute and try again: https://docs.g1t.sh/reference/rate-limits/\n"; |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 75 | |
| 76 | /** | |
| 77 | * The 429 for a git request past its limit, or null to go on. Git shows a | |
| 78 | * plain-text answer's body to the person running it. | |
| 79 | */ | |
| 80 | export async function gitLimited(env: FrontDoorLimits, request: Request): Promise<Response | null> { | |
| 81 | const credentials = request.headers.get("authorization"); | |
| 82 | if (credentials) { | |
| 83 | const verdict = await checkLimit(env.GIT_SIGNED_LIMIT, await secretKey("git", credentials)); | |
| 84 | return verdict === "limited" ? tooManyRequests(GIT_MESSAGE_SIGNED) : null; | |
| 85 | } | |
| 86 | const verdict = await checkLimit(env.GIT_ANONYMOUS_LIMIT, `ip:${clientAddress(request)}`); | |
| 87 | return verdict === "limited" ? tooManyRequests(GIT_MESSAGE_ANONYMOUS) : null; | |
| 88 | } | |
| 89 | ||
| Merge main into Artifacts Phase 2 | 90 | /** |
| 91 | * The token in a page request's `Authorization: Bearer` header, or null | |
| 92 | * (app/lib/website-token.ts). Not checked here either. | |
| 93 | */ | |
| 94 | export function websiteToken(authorization: string | null): string | null { | |
| 95 | return /^\s*bearer\s+(\S+)\s*$/i.exec(authorization ?? "")?.[1] ?? null; | |
| 96 | } | |
| 97 | ||
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 98 | /** The 429 for a page or data request past its limit, or null to go on. */ |
| 99 | export async function pageLimited(env: FrontDoorLimits, request: Request, pathname: string): Promise<Response | null> { | |
| 100 | if (unlimited(pathname)) return null; | |
| 101 | const address = `ip:${clientAddress(request)}`; | |
| Merge main into Artifacts Phase 2 | 102 | const token = websiteToken(request.headers.get("authorization")); |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 103 | const session = sessionCookie(request.headers.get("cookie")); |
| 104 | const checks: Promise<string>[] = [checkLimit(env.WEB_ADDRESS_LIMIT, address)]; | |
| Merge main into Artifacts Phase 2 | 105 | if (token) { |
| 106 | // A token on the website: per token, as the API counts it. | |
| 107 | checks.push(secretKey("token", token).then((key) => checkLimit(env.WEB_TOKEN_LIMIT, key))); | |
| 108 | } else if (session) { | |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 109 | checks.push(secretKey("session", session).then((key) => checkLimit(env.WEB_SESSION_LIMIT, key))); |
| 110 | } else { | |
| 111 | checks.push(checkLimit(env.WEB_ANONYMOUS_LIMIT, address)); | |
| 112 | if (heavy(pathname)) checks.push(checkLimit(env.WEB_HEAVY_LIMIT, address)); | |
| 113 | } | |
| 114 | const verdicts = await Promise.all(checks); | |
| 115 | if (!verdicts.includes("limited")) return null; | |
| Merge main into Artifacts Phase 2 | 116 | if (token && verdicts[1] === "limited") return tooManyRequests(TOKEN_PAGE_MESSAGE); |
| 117 | return tooManyRequests(token || session ? PAGE_MESSAGE : `${PAGE_MESSAGE.trimEnd()} Signed-in accounts have a higher limit.\n`); | |
| Merge branch 'worktree-agent-a8752162fea25f63f' into spend-guardrails | 118 | } |
| Merge g1tusercontent.com: registry answers run nothing in a browser, the site's pages run only their own scripts, repository files and avatars on their own origin, raw files rate limited per address | 119 | |
| 120 | /** | |
| 121 | * The 429 for a repository file on the usercontent origin past its limit, | |
| 122 | * or null to go on. Nothing there is signed in (it never sees the session | |
| 123 | * cookie), so a file counts as a signed-out page from its address. Avatars | |
| Merge the workspace shell: navigation and phone shell, g1t as orchestrator, agents in roles with audience-checked reads, reactions and custom emoji, live notifications and browser push, the homepage tour (agents 0002, chat 0002) | 124 | * and custom emoji are answered from cache and are not limited. |
| Merge g1tusercontent.com: registry answers run nothing in a browser, the site's pages run only their own scripts, repository files and avatars on their own origin, raw files rate limited per address | 125 | */ |
| 126 | export async function usercontentLimited(env: FrontDoorLimits, request: Request, path: string): Promise<Response | null> { | |
| Merge the workspace shell: navigation and phone shell, g1t as orchestrator, agents in roles with audience-checked reads, reactions and custom emoji, live notifications and browser push, the homepage tour (agents 0002, chat 0002) | 127 | if (path.startsWith("/avatars/") || path.startsWith("/emoji/")) return null; |
| Merge g1tusercontent.com: registry answers run nothing in a browser, the site's pages run only their own scripts, repository files and avatars on their own origin, raw files rate limited per address | 128 | const verdict = await checkLimit(env.WEB_ANONYMOUS_LIMIT, `ip:${clientAddress(request)}`); |
| 129 | return verdict === "limited" ? tooManyRequests(PAGE_MESSAGE) : null; | |
| 130 | } |
This file's history is long; its oldest lines are credited to the oldest commit read.