Pick any line to see why it is the way it is: the commit, the pull request and issue it came from, and what the agent was thinking.
| Large pull requests, commits and comparisons open quickly and no longer risk failing: the Files changed tab of a 206-file pull request sent every file's diff at once, 14,366 table rows drawn on the server (6.7 MB of its 8.7 MB of HTML) and the same lines again in the data the page hydrates from (1.6 MB), with 300 KB of highlighting sent twice. Now the page arrives with the whole file list, tree and counts, and the lines of its first files, about 2,000; every later file holds its place at its height and loads as you scroll near it, eight or so to a request (diff-files.json, cached by the two commits compared, here and in the browser), and a refresh keeps what was loaded. A file with a diff over 1,000 lines (a lockfile, generated code) waits for Load diff; files with comments on their lines come with the page wherever they are; a link to a file (#file-<path>, or #diff-<path>) opens it, loads it at once however large, and keeps it at the top while the files above it fill in. Small changes are sent exactly as before. Highlighting done on the server stops after 16,000 characters rather than 40,000; the rest is coloured in the browser as it comes near, as it already was. The repos service listed the files past its 20,000-line read as binary with no counts, so 126 of this pull request's files said "Binary or large file"; they are now marked omitted, show no counts until read ("126 files not counted" in the toolbar), and load like the rest, read on their own by compare's new paths (at most 50); truncated now means only that more than 300 files changed. get_pull_request_changes and GET /repos/{owner}/{name}/pulls/{number}/changes take paths too, and the mobile app says an omitted file is too large to show there. The pull requests guide gains Files changed, and limitations says how large changes are shown. Checked: the site's typecheck, its 914 tests (new: the first-screens budget, large and commented files, omitted and binary ones, links to a file, and fetching in batches, once, with retries) and a production build; cargo test for contracts, repos (new: omitted files and reading by path), actions, security, work and api; the mobile typecheck; the docs build. Measured on flagon-io/g1t pull 1's real response, before → after: single-fetch data 1,592,101 → 236,252 bytes; the diff's server-rendered HTML 6,694,038 → 1,698,300 bytes (renderToString 62 → 15 ms), so the page's HTML about 8.7 MB → 2.0 MB; in a harness mounting the real routes in Chrome at 1280 px, first render 600 → 270 ms, settled 2,250 → 750 ms, long tasks 1,700 → 500 ms, 84,605 → 17,982 elements; scrolling to the bottom loads all 72 waiting files in 9 requests, and refreshes with the same data remove no elements, before and after files load. Screenshots at 1280 and 390 px looked at. | 1 | import type { FileDiff } from "@g1t/contracts"; |
| 2 | ||
| 3 | /** | |
| 4 | * Fetching the files a large change was sent without (lib/diff.ts | |
| 5 | * firstScreens), from routes/repo/diff-files.ts. Files asked for in the | |
| 6 | * same moment, as several scroll near together, go in one request, and | |
| 7 | * each is fetched once per page: a refresh or a file folded and unfolded | |
| 8 | * finds it here. | |
| 9 | */ | |
| 10 | ||
| 11 | /** As many as the route takes in one request. */ | |
| 12 | const PER_REQUEST = 20; | |
| 13 | /** How long asks are gathered before they are sent. */ | |
| 14 | const GATHER_MS = 30; | |
| 15 | ||
| 16 | type Ask = { path: string; resolve: (file: FileDiff | null) => void; reject: (error: unknown) => void }; | |
| 17 | ||
| 18 | const loaded = new Map<string, FileDiff | null>(); | |
| 19 | const loading = new Map<string, Promise<FileDiff | null>>(); | |
| 20 | const queued = new Map<string, Ask[]>(); | |
| 21 | let timer: ReturnType<typeof setTimeout> | null = null; | |
| 22 | ||
| 23 | const keyOf = (source: string, path: string) => `${source}\n${path}`; | |
| 24 | ||
| 25 | /** A file fetched already, so it is drawn at once rather than after a frame of its placeholder. */ | |
| 26 | export function peekFile(source: string, path: string): FileDiff | null | undefined { | |
| 27 | return loaded.get(keyOf(source, path)); | |
| 28 | } | |
| 29 | ||
| 30 | /** | |
| 31 | * The file at `path` of the change `source` names (the route's address with | |
| 32 | * its `base` and `head`), with its lines; null when the change no longer has it. | |
| 33 | * The files in `also` (the ones after it) come in the same request, so a | |
| 34 | * reader scrolling down finds them fetched rather than asking one by one. | |
| 35 | */ | |
| 36 | export function loadFile( | |
| 37 | source: string, | |
| 38 | path: string, | |
| 39 | { also = [], fetcher = fetch }: { also?: string[]; fetcher?: typeof fetch } = {}, | |
| 40 | ): Promise<FileDiff | null> { | |
| 41 | const key = keyOf(source, path); | |
| 42 | // Fetched already, or on its way (with the file before it): nothing to send. | |
| 43 | if (loaded.has(key)) return Promise.resolve(loaded.get(key)!); | |
| 44 | if (loading.has(key)) return loading.get(key)!; | |
| 45 | for (const next of also) { | |
| 46 | if (!loaded.has(keyOf(source, next))) void ask(source, next, fetcher).catch(() => {}); | |
| 47 | } | |
| 48 | return ask(source, path, fetcher); | |
| 49 | } | |
| 50 | ||
| 51 | function ask(source: string, path: string, fetcher: typeof fetch): Promise<FileDiff | null> { | |
| 52 | const key = keyOf(source, path); | |
| 53 | if (loaded.has(key)) return Promise.resolve(loaded.get(key)!); | |
| 54 | const pending = loading.get(key); | |
| 55 | if (pending) return pending; | |
| 56 | const promise = new Promise<FileDiff | null>((resolve, reject) => { | |
| 57 | const asks = queued.get(source) ?? []; | |
| 58 | asks.push({ path, resolve, reject }); | |
| 59 | queued.set(source, asks); | |
| 60 | timer ??= setTimeout(() => flush(fetcher), GATHER_MS); | |
| 61 | }).then( | |
| 62 | (file) => { | |
| 63 | loaded.set(key, file); | |
| 64 | loading.delete(key); | |
| 65 | return file; | |
| 66 | }, | |
| 67 | (error: unknown) => { | |
| 68 | // Asked again, it is fetched again. | |
| 69 | loading.delete(key); | |
| 70 | throw error; | |
| 71 | }, | |
| 72 | ); | |
| 73 | loading.set(key, promise); | |
| 74 | return promise; | |
| 75 | } | |
| 76 | ||
| 77 | function flush(fetcher: typeof fetch) { | |
| 78 | timer = null; | |
| 79 | const all = [...queued]; | |
| 80 | queued.clear(); | |
| 81 | for (const [source, asks] of all) { | |
| 82 | for (let start = 0; start < asks.length; start += PER_REQUEST) { | |
| 83 | const batch = asks.slice(start, start + PER_REQUEST); | |
| 84 | const url = `${source}${source.includes("?") ? "&" : "?"}${batch.map((ask) => `path=${encodeURIComponent(ask.path)}`).join("&")}`; | |
| 85 | fetcher(url, { headers: { accept: "application/json" } }) | |
| 86 | .then(async (response) => { | |
| 87 | if (!response.ok) throw new Error(`The diff could not be loaded (${response.status}).`); | |
| 88 | const { files } = (await response.json()) as { files: FileDiff[] }; | |
| 89 | const byPath = new Map(files.map((file) => [file.path, file])); | |
| 90 | for (const ask of batch) ask.resolve(byPath.get(ask.path) ?? null); | |
| 91 | }) | |
| 92 | .catch((error: unknown) => { | |
| 93 | for (const ask of batch) ask.reject(error); | |
| 94 | }); | |
| 95 | } | |
| 96 | } | |
| 97 | } | |
| 98 | ||
| 99 | /** For tests: forget everything fetched. */ | |
| 100 | export function forgetFiles() { | |
| 101 | loaded.clear(); | |
| 102 | loading.clear(); | |
| 103 | queued.clear(); | |
| 104 | if (timer) clearTimeout(timer); | |
| 105 | timer = null; | |
| 106 | } |
