Merge pull request #334 from flagon-io/319-push-answer-loose-ends
A push answered with a fatal error stays ended for staff, and a push records only the refs its report did not refuse
8 files+296−590/8 viewed
| 116 | 116 | back and counts nothing: refused before the store took any of it, or in | |
| 117 | 117 | the store's report (an `ng` for its ref, most often because the branch | |
| 118 | 118 | moved meanwhile, or a failed unpack; for a push over git, every ref it | |
| 119 | − | reports on refused). Only bytes the store took reach `stored_bytes`. | |
| 119 | + | reports on refused). Only bytes the store took reach `stored_bytes`. A | |
| 120 | + | push and a write of one ref read the store's report the same way, one | |
| 121 | + | parser for both (`land.rs` `read_answer`, report-status v1 or v2). A push | |
| 122 | + | records no ref its report refused: that ref starts no event, webhook or | |
| 123 | + | workflow, and a push whose unpack failed, or whose every ref was refused, | |
| 124 | + | records nothing. A ref the report does not mention is checked against | |
| 125 | + | the store, as every recorded ref is. | |
| 126 | + | ||
| 127 | + | g1t's own copies are not reserved or counted: a pull request's test merge | |
| 128 | + | (`refs/pull/N/merge`, a merge commit g1t makes), its head copied in from | |
| 129 | + | where the change is, the head kept when a working copy is removed and the | |
| 130 | + | branch a removed working copy is made again with. Each holds what g1t | |
| 131 | + | wrote, or what was counted when it was pushed: to the repository a working | |
| 132 | + | copy belongs to, or, for a pull request opened from a branch of someone's | |
| 133 | + | own fork, to that fork. The repository the pull request is in is never | |
| 134 | + | charged for objects its owner did not push. | |
| 120 | 135 | ||
| 121 | 136 | A write that stops without knowing whether the store kept it (the store | |
| 122 | − | answered with an error, stopped answering, or answered a write of one ref | |
| 137 | + | answered with an error, stopped answering, answered a push with a fatal | |
| 138 | + | error, on side-band channel 3 or as `ERR`, or answered a write of one ref | |
| 123 | 139 | without saying whether it moved) marks its row `ended`, with the refs it | |
| 124 | 140 | asked for. A row also carries a lease, an hour from when it | |
| 125 | 141 | was taken (`LEASE_MS`): a push is at most 100 MB, which uploads well within |
| 490 | 490 | the packs pushed through g1t's git endpoints to each repository and its | |
| 491 | 491 | pull requests' working copies, so what is charged is never more than what | |
| 492 | 492 | is stored. A push, merge or update that storage refused, for example | |
| 493 | − | because its branch moved meanwhile, is not counted. Public repositories | |
| 493 | + | because its branch moved meanwhile, is not counted. Neither are the copies | |
| 494 | + | g1t makes for its own use, such as a pull request's test merge, or its head | |
| 495 | + | copied in from someone's fork: g1t wrote them, or they were counted where | |
| 496 | + | they were pushed (a fork's to the fork), never to a repository whose owner | |
| 497 | + | did not push them. Public repositories | |
| 494 | 498 | are never charged, and neither is a free workspace: its pushes to private | |
| 495 | 499 | repositories stop at 1 GB instead. On the plan they never stop. | |
| 496 | 500 |
| 85 | 85 | default branch's commit) and what g1t counts (stored, reserved, how much | |
| 86 | 86 | of it the ledger accounts for). For a push that ended, each ref it asked | |
| 87 | 87 | for says **landed** when the store has it: the push may have been stored | |
| 88 | − | though its answer was an error, or a later push may have set the same | |
| 89 | − | ref. | |
| 88 | + | though its answer was an error (a 500, or a 200 with a fatal error, | |
| 89 | + | whose outcome quotes it), or a later push may have set the same ref. | |
| 90 | 90 | 3. **Release it**, with a reason that says what you saw ("ended with a 500; | |
| 91 | 91 | main is not at the commit it asked for"). The reason and the inspection | |
| 92 | 92 | are kept in the audit row, listed under **Released**. The release is | |
| ⋯ | |||
| 116 | 116 | plus what each pull request's working copy holds beyond it, beside | |
| 117 | 117 | `stored_bytes`. It reads only. Above zero, g1t counts less than the store | |
| 118 | 118 | holds: pushes made with a credential straight to the store (sandboxes' | |
| 119 | − | `git_access`), and ended pushes that landed, are not counted. Below zero, | |
| 119 | + | `git_access`), ended pushes that landed, g1t's own refs (a pull | |
| 120 | + | request's test merge, `refs/pull/N/merge`), and a pull request's head | |
| 121 | + | copied in from someone's fork are not counted. Below zero, | |
| 120 | 122 | g1t counted objects that a later push made unreachable (a force push), or | |
| 121 | 123 | packs that sent objects the store already had: g1t asks for whole packs. | |
| 122 | 124 | Before #307, g1t also counted writes the store refused (a merge or update | |
| 335 | 335 | let fork_read = self.store.open(&store_key(fork)).await?.access(Scope::Read).await?; | |
| 336 | 336 | land::fetch_pack(&fork_read, head, base.as_deref()).await? | |
| 337 | 337 | }; | |
| 338 | + | // Not reserved or counted (`stored_bytes`): the pack holds only the | |
| 339 | + | // working copy's commits, which were counted to this parent (the | |
| 340 | + | // working copy's root, registry.rs `add_stored_bytes`) when they | |
| 341 | + | // were pushed to it, and stay counted after its git data goes. | |
| 342 | + | // Counting them again here would count them twice. | |
| 338 | 343 | let parent_write = parent_git.access(Scope::Write).await?; | |
| 339 | 344 | let kept = land::push_ref(&parent_write, &reference, existing.as_deref(), head, Some(pack)).await; | |
| 340 | 345 | self.refs_moved(&parent.id).await; | |
| ⋯ | |||
| 361 | 366 | let git = self.store.open(&key).await?; | |
| 362 | 367 | let current = git.log(&fork.default_branch, 1).await?.into_iter().next().map(|commit| commit.hash); | |
| 363 | 368 | if current.as_deref() != Some(head.as_str()) { | |
| 369 | + | // Not reserved or counted (`stored_bytes`): the pack is the | |
| 370 | + | // parent's own objects (the head `keep_head` kept), already | |
| 371 | + | // counted to the parent, which is this working copy's root too. | |
| 364 | 372 | let pack = land::fetch_pack(&parent_git.access(Scope::Read).await?, &head, current.as_deref()).await?; | |
| 365 | 373 | let moved = land::push_ref( | |
| 366 | 374 | &git.access(Scope::Write).await?, | |
| 216 | 216 | } | |
| 217 | 217 | } | |
| 218 | 218 | ||
| 219 | − | /// The lines of a receive-pack report-status, plain or inside side-band | |
| 220 | − | /// channel 1, as the store sends either. `None` when the answer is not a | |
| 221 | − | /// report this can read, or carries a fatal error (channel 3). | |
| 222 | − | pub(crate) fn report_lines(answer: &[u8]) -> Option<Vec<String>> { | |
| 219 | + | /// A receive-pack's answer, read once for everything that settles or | |
| 220 | + | /// announces by it: [`verdict`] for a write of one ref, and a push over | |
| 221 | + | /// git (public_storage.rs `push_settlement`, lib.rs `answer_git`), so they | |
| 222 | + | /// never read one answer two ways. | |
| 223 | + | #[derive(Debug, PartialEq, Eq)] | |
| 224 | + | pub(crate) enum Answer { | |
| 225 | + | /// A report-status, v1 or v2. | |
| 226 | + | Report(Report), | |
| 227 | + | /// A fatal error, on side-band channel 3 or as an `ERR` packet, with | |
| 228 | + | /// what it said: the store stopped without a word on any ref, most | |
| 229 | + | /// likely before it took anything, but that is not known. | |
| 230 | + | Fatal(String), | |
| 231 | + | /// Nothing this can read as a report: none was asked for, or the | |
| 232 | + | /// answer is not git's. | |
| 233 | + | Unread, | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | /// What a report-status says. v1's lines are `unpack <status>`, then | |
| 237 | + | /// `ok <ref>` or `ng <ref> <reason>` for each command; v2 may follow an | |
| 238 | + | /// `ok` with `option <key> <value>` lines about it, which say nothing of | |
| 239 | + | /// whether the ref moved and are passed over. A ref name holds no space, | |
| 240 | + | /// so an `ok` line names exactly one ref. | |
| 241 | + | #[derive(Debug, Default, PartialEq, Eq)] | |
| 242 | + | pub(crate) struct Report { | |
| 243 | + | /// Its lines, as sent: the outcome staff read. | |
| 244 | + | pub lines: Vec<String>, | |
| 245 | + | /// It said `unpack ok`. | |
| 246 | + | pub unpacked: bool, | |
| 247 | + | /// It said the unpack failed: nothing of the pack was kept. | |
| 248 | + | pub unpack_failed: bool, | |
| 249 | + | /// The refs it says moved (`ok`). | |
| 250 | + | pub moved: Vec<String>, | |
| 251 | + | /// The refs it says it refused (`ng`). | |
| 252 | + | pub refused: Vec<String>, | |
| 253 | + | } | |
| 254 | + | ||
| 255 | + | impl Report { | |
| 256 | + | fn read(lines: Vec<String>) -> Report { | |
| 257 | + | let mut report = Report::default(); | |
| 258 | + | for line in &lines { | |
| 259 | + | if let Some(status) = line.strip_prefix("unpack ") { | |
| 260 | + | if status == "ok" { | |
| 261 | + | report.unpacked = true; | |
| 262 | + | } else { | |
| 263 | + | report.unpack_failed = true; | |
| 264 | + | } | |
| 265 | + | } else if let Some(reference) = line.strip_prefix("ok ") { | |
| 266 | + | report.moved.push(reference.to_owned()); | |
| 267 | + | } else if let Some(rest) = line.strip_prefix("ng ") { | |
| 268 | + | report.refused.push(rest.split(' ').next().unwrap_or_default().to_owned()); | |
| 269 | + | } | |
| 270 | + | } | |
| 271 | + | report.lines = lines; | |
| 272 | + | report | |
| 273 | + | } | |
| 274 | + | ||
| 275 | + | /// Whether it says `reference` moved. | |
| 276 | + | pub fn moved(&self, reference: &str) -> bool { | |
| 277 | + | !self.unpack_failed && self.moved.iter().any(|moved| moved == reference) | |
| 278 | + | } | |
| 279 | + | } | |
| 280 | + | ||
| 281 | + | impl Answer { | |
| 282 | + | /// Whether a push records `git_ref` from this answer: not when the | |
| 283 | + | /// report refused it (`ng`), nor any ref when its unpack failed. A ref | |
| 284 | + | /// the report does not mention, or an answer that says nothing either | |
| 285 | + | /// way (a fatal error, no report), is left to the store's refs, which | |
| 286 | + | /// record_push checks (pushed.rs `pushed_to`, lib.rs `publish_deleted`): | |
| 287 | + | /// a report cut short never hides a ref that moved. | |
| 288 | + | pub(crate) fn records(&self, git_ref: &str) -> bool { | |
| 289 | + | match self { | |
| 290 | + | Answer::Report(report) => !report.unpack_failed && !report.refused.iter().any(|refused| refused == git_ref), | |
| 291 | + | Answer::Fatal(_) | Answer::Unread => true, | |
| 292 | + | } | |
| 293 | + | } | |
| 294 | + | } | |
| 295 | + | ||
| 296 | + | /// Reads a receive-pack's answer, its report-status plain or inside | |
| 297 | + | /// side-band channel 1, as the store sends either. | |
| 298 | + | pub(crate) fn read_answer(answer: &[u8]) -> Answer { | |
| 223 | 299 | let (packets, _) = read_pkt_lines(answer); | |
| 224 | 300 | let banded = packets.first().is_some_and(|packet| matches!(packet.first(), Some(1..=3))); | |
| 225 | 301 | let status = if banded { | |
| ⋯ | |||
| 228 | 304 | match packet.first() { | |
| 229 | 305 | Some(1) => status.extend_from_slice(&packet[1..]), | |
| 230 | 306 | Some(2) => {} | |
| 231 | − | _ => return None, | |
| 307 | + | Some(3) => return Answer::Fatal(String::from_utf8_lossy(&packet[1..]).trim().to_owned()), | |
| 308 | + | _ => return Answer::Unread, | |
| 232 | 309 | } | |
| 233 | 310 | } | |
| 234 | 311 | status | |
| ⋯ | |||
| 237 | 314 | }; | |
| 238 | 315 | let (lines, _) = read_pkt_lines(&status); | |
| 239 | 316 | let lines: Vec<String> = lines.into_iter().map(|line| String::from_utf8_lossy(line).trim_end().to_owned()).collect(); | |
| 240 | − | (!lines.is_empty()).then_some(lines) | |
| 317 | + | if let Some(said) = lines.iter().find_map(|line| line.strip_prefix("ERR ")) { | |
| 318 | + | return Answer::Fatal(said.to_owned()); | |
| 319 | + | } | |
| 320 | + | if lines.is_empty() { Answer::Unread } else { Answer::Report(Report::read(lines)) } | |
| 241 | 321 | } | |
| 242 | 322 | ||
| 243 | 323 | /// What a report-status says of `reference`. | |
| 244 | − | pub(crate) fn verdict(report: &[u8], reference: &str, sent_pack: bool) -> Verdict { | |
| 324 | + | pub(crate) fn verdict(answer: &[u8], reference: &str, sent_pack: bool) -> Verdict { | |
| 245 | 325 | // What was unclear, said whole: it is the outcome staff read on an | |
| 246 | 326 | // ended reservation. | |
| 247 | 327 | let unclear = |said: &str| { | |
| 248 | 328 | let said = if said.is_empty() { "it said nothing" } else { said }; | |
| 249 | 329 | Verdict::Unclear(format!("The store's answer said neither that {reference} moved nor that it was refused: {said}")) | |
| 250 | 330 | }; | |
| 251 | − | let Some(lines) = report_lines(report) else { | |
| 252 | − | return unclear(String::from_utf8_lossy(report).trim()); | |
| 331 | + | let report = match read_answer(answer) { | |
| 332 | + | Answer::Report(report) => report, | |
| 333 | + | Answer::Fatal(said) => return unclear(&said), | |
| 334 | + | Answer::Unread => return unclear(String::from_utf8_lossy(answer).trim()), | |
| 253 | 335 | }; | |
| 254 | − | let reason = lines.join("; "); | |
| 255 | − | if lines.iter().any(|line| line.starts_with("unpack ") && line != "unpack ok") { | |
| 336 | + | let reason = report.lines.join("; "); | |
| 337 | + | if report.unpack_failed { | |
| 256 | 338 | return Verdict::Refused(reason); | |
| 257 | 339 | } | |
| 258 | 340 | // With nothing to unpack a server may not say so. | |
| 259 | − | let unpacked = !sent_pack || lines.iter().any(|line| line == "unpack ok"); | |
| 260 | − | if unpacked && lines.iter().any(|line| *line == format!("ok {reference}")) { | |
| 341 | + | if (!sent_pack || report.unpacked) && report.moved(reference) { | |
| 261 | 342 | return Verdict::Moved; | |
| 262 | 343 | } | |
| 263 | − | let ng = format!("ng {reference}"); | |
| 264 | − | if lines.iter().any(|line| *line == ng || line.starts_with(&format!("{ng} "))) { | |
| 344 | + | if report.refused.iter().any(|refused| refused == reference) { | |
| 265 | 345 | return Verdict::Refused(reason); | |
| 266 | 346 | } | |
| 267 | 347 | unclear(&reason) | |
| 2453 | 2453 | } else { | |
| 2454 | 2454 | None | |
| 2455 | 2455 | }; | |
| 2456 | + | // The store's answer, read once: its reservation settles by it, and | |
| 2457 | + | // the refs it refused are not recorded. | |
| 2458 | + | let reply = answered.as_ref().map(|(_, report)| land::read_answer(report)); | |
| 2456 | 2459 | if let Some(reserved) = &public_reserved { | |
| 2457 | − | let report = answered.as_ref().map(|(_, report)| report.as_slice()); | |
| 2458 | − | match public_storage::push_settlement(status, forwarded.rate_limited, report) { | |
| 2460 | + | match public_storage::push_settlement(status, forwarded.rate_limited, reply.as_ref()) { | |
| 2459 | 2461 | // Successful uploads replace it with the actual admitted pack. | |
| 2460 | 2462 | public_storage::Settlement::Finish => self.registry.finish_public(reserved, forwarded.pack_bytes).await?, | |
| 2461 | 2463 | // Refused for the store's rate limit, or a report saying | |
| 2462 | 2464 | // the store refused all of it (its unpack failed, or every | |
| 2463 | 2465 | // ref was refused): it took nothing. | |
| 2464 | 2466 | public_storage::Settlement::Release => self.registry.release_public(reserved).await?, | |
| 2465 | − | // On any other answer the store may have kept the push, so | |
| 2466 | − | // the whole reservation stays, ended, with the refs it asked | |
| 2467 | − | // for: staff release it once they have looked | |
| 2468 | − | // (storage_holds.rs). | |
| 2467 | + | // On any other answer (an error, or a 200 with a fatal | |
| 2468 | + | // error) the store may have kept the push, so the whole | |
| 2469 | + | // reservation stays, ended, with the refs it asked for: | |
| 2470 | + | // staff release it once they have looked (storage_holds.rs). | |
| 2469 | 2471 | public_storage::Settlement::End => { | |
| 2470 | − | self.registry.end_public(reserved, &format!("The store answered the push with {status}."), &asked).await; | |
| 2472 | + | let outcome = public_storage::push_outcome(status, reply.as_ref()); | |
| 2473 | + | self.registry.end_public(reserved, &outcome, &asked).await; | |
| 2471 | 2474 | } | |
| 2472 | 2475 | } | |
| 2473 | 2476 | } | |
| ⋯ | |||
| 2527 | 2530 | } | |
| 2528 | 2531 | } | |
| 2529 | 2532 | after.ended(status, None); | |
| 2530 | − | if status == 200 && (forwarded.pack_bytes > 0 || !forwarded.pushed.is_empty() || !forwarded.deleted.is_empty()) { | |
| 2533 | + | // A ref the report refused is not recorded: it starts no event, | |
| 2534 | + | // webhook, workflow or rule, and a push whose unpack failed records | |
| 2535 | + | // nothing. A ref the report does not mention, or an answer that | |
| 2536 | + | // says neither (a fatal error, no report), is left to record_push, | |
| 2537 | + | // which checks where it is in the store. | |
| 2538 | + | let (mut pushed, mut deleted) = (forwarded.pushed, forwarded.deleted); | |
| 2539 | + | if let Some(reply) = &reply { | |
| 2540 | + | pushed.retain(|pushed| reply.records(&pushed.git_ref)); | |
| 2541 | + | deleted.retain(|(git_ref, _)| reply.records(git_ref)); | |
| 2542 | + | } | |
| 2543 | + | // A reserved push is counted by its reservation, settled above. | |
| 2544 | + | let pack_bytes = if public_reserved.is_some() { 0 } else { forwarded.pack_bytes }; | |
| 2545 | + | if status == 200 && (pack_bytes > 0 || !pushed.is_empty() || !deleted.is_empty()) { | |
| 2531 | 2546 | after.push = Some(PushDone { | |
| 2532 | 2547 | repo, | |
| 2533 | − | pushed: forwarded.pushed, | |
| 2534 | − | deleted: forwarded.deleted, | |
| 2535 | − | pack_bytes: if public_reserved.is_some() { 0 } else { forwarded.pack_bytes }, | |
| 2548 | + | pushed, | |
| 2549 | + | deleted, | |
| 2550 | + | pack_bytes, | |
| 2536 | 2551 | caused_by_job: viewer.as_ref().and_then(g1t_contracts::events::job_run_of).map(str::to_owned), | |
| 2537 | 2552 | // The calendar credits people, never a job's, an agent's or a workspace's token. | |
| 2538 | 2553 | credit: viewer | |
| 35 | 35 | use serde::Deserialize; | |
| 36 | 36 | use worker::Result; | |
| 37 | 37 | ||
| 38 | − | use crate::land::Verdict; | |
| 38 | + | use crate::land::{Answer, Report, Verdict}; | |
| 39 | 39 | use crate::registry::Registry; | |
| 40 | 40 | ||
| 41 | 41 | /// How long an operation holding a reservation may send to the store. A | |
| ⋯ | |||
| 442 | 442 | End, | |
| 443 | 443 | } | |
| 444 | 444 | ||
| 445 | − | /// A push the store answered with `status` and `report`, its answer read | |
| 446 | − | /// whole; `rate_limited` when the store refused it for its rate limit | |
| 447 | − | /// (git_http.rs `Forwarded`), which takes nothing. A 200 whose report | |
| 448 | − | /// says the store refused all of it ([`refused_whole`]) takes nothing | |
| 449 | − | /// either; any other 200 is counted. Any other answer than 200 is | |
| 450 | − | /// ambiguous. | |
| 451 | − | pub(crate) fn push_settlement(status: u16, rate_limited: bool, report: Option<&[u8]>) -> Settlement { | |
| 452 | − | match (status, rate_limited) { | |
| 453 | − | (200, _) if report.is_some_and(refused_whole) => Settlement::Release, | |
| 454 | − | (200, _) => Settlement::Finish, | |
| 455 | − | (_, true) => Settlement::Release, | |
| 445 | + | /// A push the store answered with `status` and `answer`, its answer read | |
| 446 | + | /// whole (land.rs `read_answer`); `rate_limited` when the store refused it | |
| 447 | + | /// for its rate limit (git_http.rs `Forwarded`), which takes nothing. A | |
| 448 | + | /// 200 whose report says the store refused all of it ([`refused_whole`]) | |
| 449 | + | /// takes nothing either. A 200 with a fatal error (side-band channel 3, | |
| 450 | + | /// or `ERR`) says nothing of any ref: the store most likely took nothing, | |
| 451 | + | /// but that is not known, so it ends, as a write of one ref answered so | |
| 452 | + | /// does (`write_settlement`). Any other 200 is counted, a report that | |
| 453 | + | /// moved a ref or none at all (a client that asked for none). Any other | |
| 454 | + | /// answer than 200 is ambiguous. | |
| 455 | + | pub(crate) fn push_settlement(status: u16, rate_limited: bool, answer: Option<&Answer>) -> Settlement { | |
| 456 | + | match (status, rate_limited, answer) { | |
| 457 | + | (200, _, Some(Answer::Report(report))) if refused_whole(report) => Settlement::Release, | |
| 458 | + | (200, _, Some(Answer::Fatal(_))) => Settlement::End, | |
| 459 | + | (200, ..) => Settlement::Finish, | |
| 460 | + | (_, true, _) => Settlement::Release, | |
| 456 | 461 | _ => Settlement::End, | |
| 457 | 462 | } | |
| 458 | 463 | } | |
| 459 | 464 | ||
| 465 | + | /// What an ended push's reservation says of how it ended, for staff. | |
| 466 | + | pub(crate) fn push_outcome(status: u16, answer: Option<&Answer>) -> String { | |
| 467 | + | match answer { | |
| 468 | + | Some(Answer::Fatal(said)) => format!("The store answered the push with {status} and a fatal error: {said}"), | |
| 469 | + | _ => format!("The store answered the push with {status}."), | |
| 470 | + | } | |
| 471 | + | } | |
| 472 | + | ||
| 460 | 473 | /// Whether a push's report-status says the store kept none of it: its | |
| 461 | 474 | /// unpack failed, or it refused (`ng`) every ref it reports on and moved | |
| 462 | − | /// none. A report this cannot read says neither. | |
| 463 | − | pub(crate) fn refused_whole(report: &[u8]) -> bool { | |
| 464 | − | let Some(lines) = crate::land::report_lines(report) else { | |
| 465 | − | return false; | |
| 466 | − | }; | |
| 467 | − | if lines.iter().any(|line| line.starts_with("unpack ") && line != "unpack ok") { | |
| 468 | − | return true; | |
| 469 | − | } | |
| 470 | − | lines.iter().any(|line| line == "unpack ok") | |
| 471 | − | && lines.iter().any(|line| line.starts_with("ng ")) | |
| 472 | − | && !lines.iter().any(|line| line.starts_with("ok ")) | |
| 475 | + | /// none. | |
| 476 | + | pub(crate) fn refused_whole(report: &Report) -> bool { | |
| 477 | + | report.unpack_failed || (report.unpacked && !report.refused.is_empty() && report.moved.is_empty()) | |
| 473 | 478 | } | |
| 474 | 479 | ||
| 475 | 480 | /// What the store's answer to a write of one ref (land.rs `Verdict`) | |
| ⋯ | |||
| 864 | 869 | ||
| 865 | 870 | #[test] | |
| 866 | 871 | fn a_push_the_store_refused_whole_counts_nothing() { | |
| 867 | − | let settle = |answer: Vec<u8>| push_settlement(200, false, Some(&answer)); | |
| 872 | + | let settle = |answer: Vec<u8>| push_settlement(200, false, Some(&crate::land::read_answer(&answer))); | |
| 868 | 873 | for answer in [ | |
| 869 | 874 | // Every ref refused: the store moved none of them. | |
| 870 | 875 | report(&["unpack ok", "ng refs/heads/main non-fast-forward", "ng refs/tags/v1 already exists"]), | |
| ⋯ | |||
| 883 | 888 | assert_eq!(settle(b"0000".to_vec()), Settlement::Finish); | |
| 884 | 889 | assert_eq!(settle(Vec::new()), Settlement::Finish); | |
| 885 | 890 | // An answer other than 200 is ambiguous whatever it says. | |
| 886 | − | assert_eq!(push_settlement(500, false, Some(&report(&["unpack ok", "ng refs/heads/main x"]))), Settlement::End); | |
| 891 | + | let refused = crate::land::read_answer(&report(&["unpack ok", "ng refs/heads/main x"])); | |
| 892 | + | assert_eq!(push_settlement(500, false, Some(&refused)), Settlement::End); | |
| 887 | 893 | // The push path reads the report before it settles. | |
| 888 | 894 | let answer = body_of(include_str!("lib.rs"), "answer_git"); | |
| 889 | 895 | let read = answer.find("match response.bytes().await").expect("the report is read"); | |
| 890 | − | let settles = answer.find("public_storage::push_settlement(status, forwarded.rate_limited, report)").expect("settled by it"); | |
| 891 | − | assert!(read < settles); | |
| 896 | + | let parsed = answer.find("land::read_answer(report)").expect("read once"); | |
| 897 | + | let settles = answer.find("public_storage::push_settlement(status, forwarded.rate_limited, reply.as_ref())").expect("settled by it"); | |
| 898 | + | assert!(read < parsed && parsed < settles); | |
| 899 | + | } | |
| 900 | + | ||
| 901 | + | #[test] | |
| 902 | + | fn a_push_the_store_answered_with_a_fatal_error_stays_ended_for_staff() { | |
| 903 | + | let fatal = [crate::git_http::pkt_line(b"\x03fatal: the remote end hung up unexpectedly\n"), b"0000".to_vec()].concat(); | |
| 904 | + | let after_progress = [crate::git_http::pkt_line(b"\x02Receiving objects: 40%\n"), fatal.clone()].concat(); | |
| 905 | + | let err = [crate::git_http::pkt_line(b"ERR pack exceeds the store's limit\n"), b"0000".to_vec()].concat(); | |
| 906 | + | for answer in [&fatal, &after_progress, &err] { | |
| 907 | + | let read = crate::land::read_answer(answer); | |
| 908 | + | assert!(matches!(read, Answer::Fatal(_)), "{read:?}"); | |
| 909 | + | // Ended with the refs it asked for, as a write of one ref | |
| 910 | + | // answered so is (Unclear). | |
| 911 | + | assert_eq!(push_settlement(200, false, Some(&read)), Settlement::End); | |
| 912 | + | assert_eq!(settled(answer, "refs/heads/main"), Settlement::End); | |
| 913 | + | // Every ref is left to record_push, which checks the store. | |
| 914 | + | assert!(read.records("refs/heads/main")); | |
| 915 | + | } | |
| 916 | + | let outcome = push_outcome(200, Some(&crate::land::read_answer(&fatal))); | |
| 917 | + | assert_eq!(outcome, "The store answered the push with 200 and a fatal error: fatal: the remote end hung up unexpectedly"); | |
| 918 | + | assert_eq!(push_outcome(500, None), "The store answered the push with 500."); | |
| 919 | + | // The push path ends it through end_public with what it asked for. | |
| 920 | + | let answer = body_of(include_str!("lib.rs"), "answer_git"); | |
| 921 | + | assert!(answer.contains("let outcome = public_storage::push_outcome(status, reply.as_ref());")); | |
| 922 | + | assert!(answer.contains("self.registry.end_public(reserved, &outcome, &asked).await;")); | |
| 923 | + | } | |
| 924 | + | ||
| 925 | + | #[test] | |
| 926 | + | fn a_push_and_a_write_of_one_ref_read_a_report_the_same() { | |
| 927 | + | // v1, and v2 with option lines after an ok, plain and banded. | |
| 928 | + | let answers = [ | |
| 929 | + | report(&["unpack ok", "ok refs/heads/main"]), | |
| 930 | + | report(&["unpack ok", "ok refs/heads/main", "option refname refs/heads/main", "option old-oid abc", "option new-oid def"]), | |
| 931 | + | banded(&["unpack ok", "ok refs/heads/main", "option forced-update"]), | |
| 932 | + | report(&["unpack ok", "ng refs/heads/main non-fast-forward"]), | |
| 933 | + | report(&["unpack ok", "ng refs/heads/main"]), | |
| 934 | + | report(&["unpack error: bad pack", "ok refs/heads/main"]), | |
| 935 | + | report(&["unpack ok"]), | |
| 936 | + | report(&["unpack ok", "ok refs/heads/main2"]), | |
| 937 | + | report(&["unpack ok", "okrefs/heads/main"]), | |
| 938 | + | ]; | |
| 939 | + | for answer in &answers { | |
| 940 | + | let read = crate::land::read_answer(answer); | |
| 941 | + | let verdict = crate::land::verdict(answer, "refs/heads/main", true); | |
| 942 | + | let Answer::Report(report) = &read else { panic!("{read:?}") }; | |
| 943 | + | // A write of one ref is counted when the report says it moved, | |
| 944 | + | // and a ref refused there is one a push does not record. | |
| 945 | + | assert_eq!(verdict == Verdict::Moved, report.moved("refs/heads/main"), "{report:?}"); | |
| 946 | + | assert_eq!(matches!(verdict, Verdict::Refused(_)), !read.records("refs/heads/main"), "{report:?}"); | |
| 947 | + | if refused_whole(report) { | |
| 948 | + | assert!(matches!(verdict, Verdict::Refused(_)), "{report:?}"); | |
| 949 | + | } | |
| 950 | + | } | |
| 951 | + | // v2's options are not refs. | |
| 952 | + | let Answer::Report(v2) = crate::land::read_answer(&answers[1]) else { panic!() }; | |
| 953 | + | assert_eq!(v2.moved, ["refs/heads/main"]); | |
| 954 | + | assert!(v2.refused.is_empty()); | |
| 955 | + | // One parser: neither reads the lines itself. | |
| 956 | + | let code = include_str!("public_storage.rs").split("#[cfg(test)]\nmod tests").next().unwrap_or_default(); | |
| 957 | + | assert!(!code.contains("starts_with(\"ok \")") && !code.contains("starts_with(\"ng \")")); | |
| 958 | + | } | |
| 959 | + | ||
| 960 | + | #[test] | |
| 961 | + | fn a_push_records_every_ref_but_those_its_report_refused() { | |
| 962 | + | let (main, tag, pull) = ("refs/heads/main", "refs/tags/v1", "refs/pull/1/head"); | |
| 963 | + | let recorded = |answer: Vec<u8>| { | |
| 964 | + | let read = crate::land::read_answer(&answer); | |
| 965 | + | [main, tag, pull].into_iter().filter(|git_ref| read.records(git_ref)).collect::<Vec<_>>() | |
| 966 | + | }; | |
| 967 | + | // Every ref refused, a tag among them: none. | |
| 968 | + | assert!(recorded(report(&["unpack ok", "ng refs/heads/main non-fast-forward", "ng refs/tags/v1 already exists", "ng refs/pull/1/head x"])).is_empty()); | |
| 969 | + | // The unpack failed: none, whatever the refs say. | |
| 970 | + | assert!(recorded(report(&["unpack error: invalid pack", "ok refs/heads/main"])).is_empty()); | |
| 971 | + | // One moved beside one refused (a hidden ref): all but the refused. | |
| 972 | + | assert_eq!(recorded(banded(&["unpack ok", "ok refs/heads/main", "ng refs/pull/1/head deny updating a hidden ref"])), [main, tag]); | |
| 973 | + | assert_eq!(recorded(report(&["unpack ok", "ok refs/heads/main", "option refname refs/heads/main", "ng refs/tags/v1 already exists"])), [main, pull]); | |
| 974 | + | // A report that mentions only some refs, or none (just `unpack ok`, | |
| 975 | + | // or cut short): the rest are left to record_push's store checks. | |
| 976 | + | assert_eq!(recorded(report(&["unpack ok", "ok refs/heads/main"])), [main, tag, pull]); | |
| 977 | + | assert_eq!(recorded(report(&["unpack ok"])), [main, tag, pull]); | |
| 978 | + | assert_eq!(recorded(banded(&["unpack ok", "ng refs/tags/v1 already exists"])), [main, pull]); | |
| 979 | + | // No report, or a fatal error: left to the store too. | |
| 980 | + | assert_eq!(recorded(Vec::new()), [main, tag, pull]); | |
| 981 | + | assert_eq!(recorded([crate::git_http::pkt_line(b"\x03fatal: hung up\n"), b"0000".to_vec()].concat()), [main, tag, pull]); | |
| 982 | + | // The push path filters what it records by it, pushed and deleted. | |
| 983 | + | let answer = body_of(include_str!("lib.rs"), "answer_git"); | |
| 984 | + | let filtered = answer.find("if let Some(reply) = &reply {").expect("filtered by the report"); | |
| 985 | + | let done = answer.find("after.push = Some(PushDone {").expect("recorded"); | |
| 986 | + | assert!(filtered < done); | |
| 987 | + | assert!(answer.contains("pushed.retain(|pushed| reply.records(&pushed.git_ref));")); | |
| 988 | + | assert!(answer.contains("deleted.retain(|(git_ref, _)| reply.records(git_ref));")); | |
| 989 | + | assert!(!answer[done..].contains("forwarded.pushed") && !answer[done..].contains("forwarded.deleted")); | |
| 892 | 990 | } | |
| 893 | 991 | ||
| 894 | 992 | #[test] | |
| 354 | 354 | let head = a.head.clone(); | |
| 355 | 355 | ||
| 356 | 356 | // The head, kept in the repository, its objects brought from the fork. | |
| 357 | + | // Not reserved or counted here. They come from the pull request's | |
| 358 | + | // source (contracts `Pull::source_id`): its working copy, whose | |
| 359 | + | // pushes were counted to this repository (its root, registry.rs | |
| 360 | + | // `add_stored_bytes`) when they were made, so counting the copy too | |
| 361 | + | // would count them twice; or a branch of someone's own fork | |
| 362 | + | // (`head_repo`, `fork_of` NULL), whose pushes were counted to that | |
| 363 | + | // fork, not to this repository. Either way this repository's owner | |
| 364 | + | // did not push them, so they are not billed here; Drift counts the | |
| 365 | + | // fork's as the store holding more than g1t counts. | |
| 357 | 366 | let held_head = find(&refs, &head_ref); | |
| 358 | 367 | if held_head.as_deref() != Some(head.as_str()) { | |
| 359 | 368 | let write = git.access(Scope::Write).await?; | |
| ⋯ | |||
| 396 | 405 | Ok(PullMergeRef::Merged { head, base, merge: commit }) | |
| 397 | 406 | } | |
| 398 | 407 | Made::Merged { commit, objects } => { | |
| 408 | + | // Not reserved or counted (`stored_bytes`): the test merge is | |
| 409 | + | // g1t's own ref, made again whenever the base or head moves, | |
| 410 | + | // from a merge commit and the trees g1t wrote for it. Nobody | |
| 411 | + | // pushed it, so nobody is charged for it; Drift counts it as | |
| 412 | + | // the store holding more than g1t counts (storage-holds.md). | |
| 399 | 413 | let write = git.access(Scope::Write).await?; | |
| 400 | 414 | let pushed = land::push_ref(&write, &merge_ref, held_merge.as_deref(), &commit, Some(write_pack(&objects))).await?; | |
| 401 | 415 | self.refs_moved(&target.id).await; | |
Comments
Sign in to comment.