Skip to content

Where people and agents ship software together. The open-source git platform for the whole job: issues, agents, checks and deploys to the edge.

Read a pushed pack in place, so a large push is scanned instead of answering 500 #177

Mergedsyntaqx merged pack-memory-budget into main
syntaqxopened this pull request

Why

A force push of 359 rewritten commits (9.8 MB) to flagon-io/g1t got a bare HTTP 500. Pushes under 24 MB are held and checked whole in the Worker, and Pack::parse inflated every object and kept them all. Its 48 MB cap added up each entry's header size, which for a delta is the delta's size, not the size of the object it makes. So this push passed the cap and still inflated to ~200 MB, past the Worker's 128 MB. Cloudflare ended the request, and git got a 500 with no message.

What changes

  • Pack keeps the pack as it came, compressed, with an index of entries. An object is inflated when someone asks for it: walk down its delta chain to an object already at hand, then apply each delta on the way back up.
  • A FIFO cache keeps the objects inflated most recently, up to 24 MB, as bases for the deltas that follow.
  • MAX_INFLATED now counts each object at its full size, so it bounds the real work of reading the pack. It's raised to 512 MB because memory no longer grows with it.
  • get/blob return Rc<[u8]> instead of &[u8]. Call sites in rule_facts, workflow_gate and git_http borrow it.

Evidence

Measured on the real push that failed, natively, with a counting allocator:

beforeafter
held inflated~200 MB44 MB peak
parse time357 ms372 ms
objects vs git index-pack—8070 / 8070 match

New tests cover reading a delta chain back with the cache at zero (so every object is rebuilt from the pack) and the cache dropping its oldest entries first. g1t-scan (75) and g1t-repos (371) pass, and the wasm build of g1t-repos checks clean.

Closes #176

syntaqxmerged this·

All checks have passed

2 passed · on 73ab873

Workflows

CI / pull_request — CI passedDetails
Dependency review — No dependency changesDetails

Merged and closed: the branch can be deleted safely

Everything on it is in its base now.

Sign in to comment.