Skip to content

Commit

Merge branch 'worktree-agent-a633ac0f7f66d419d'

syntaqxcommitted Parents32894a4e1824a1Browse files
16 files+838−600/16 viewed
+6−2
259259 difference appears on the statement as a correction, such as
260260 *Correction to "Work on acme/api#12": AI Gateway priced its 41 model
261261 requests at $0.0312, not $0.0298*. A run whose sandbox stopped without
262− reporting is charged from the gateway's logs instead of not at all.
262+ reporting is charged from the gateway's logs instead of not at all. When
263+ the gateway has no price for a model a run used, the run is never settled
264+ below what its sandbox reported, and the correction says so.
263265 - **Cloudflare.** Every day, g1t checks what Cloudflare billed its account
264266 against what was used: Containers and the Durable Objects behind them
265267 against the seconds containers ran, Workers for Platforms per request
615617 charged, and usage is still recorded at what it cost, so the Usage page
616618 stays accurate.
617619 - **Custom**: a discount on usage, a limit of its own, or a larger share
618− of the pools, sometimes until a date.
620+ of the pools, sometimes until a date. A discount comes off each usage
621+ charge, and the statement line says how much off (*(20% off)*); prices
622+ themselves stay the public ones.
619623 - **A longer audit log**: up to 400 days for every workspace the account
620624 pays for, in place of the plan's 7 or 90. See
621625 [how long it is kept](/guides/audit-log/#how-long-it-is-kept).
+22−1
15321532 TermsKind::Standard => charge_micros,
15331533 }
15341534 }
1535+
1536+ /// What a charge at cost plus the margin becomes under these terms, and
1537+ /// how much of it g1t gives away by a discount: a sold charge is never
1538+ /// below its cost plus the margin unless the difference is counted as
1539+ /// given (`ledger.discount_micros`), never lost. Comped terms give it
1540+ /// all, and are counted as comped elsewhere, so their given part is 0
1541+ /// here.
1542+ pub fn discounted(&self, charge_micros: i64) -> (i64, i64) {
1543+ let charged = self.apply(charge_micros);
1544+ match self.kind {
1545+ TermsKind::Custom => (charged, (charge_micros - charged).max(0)),
1546+ TermsKind::Comped | TermsKind::Standard => (charged, 0),
1547+ }
1548+ }
15351549 }
15361550
15371551 #[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
24532467 pub given_trial_micros: i64,
24542468 #[serde(default)]
24552469 pub given_pool_micros: i64,
2470+ /// What discounts on an account's terms took below cost plus the
2471+ /// margin: given, so a discounted sale is not margin lost.
2472+ #[serde(default)]
2473+ pub given_discount_micros: i64,
24562474 /// `cost_micros` by who g1t pays: Cloudflare's bill (billed amounts,
24572475 /// after the included allowances), and model providers (the ledger's
24582476 /// cost of the tokens, which Cloudflare's bill does not show).
24742492 pub bucket: String,
24752493 pub title: String,
24762494 /// `count` (units g1t counted against Cloudflare's), `cost` (the bill
2477− /// against the price book's cost of the same usage), or `leak`.
2495+ /// against the price book's cost of the same usage; for models, what AI
2496+ /// Gateway priced g1t's provider traffic at against the ledger's model
2497+ /// cost), `unpriced` (model usage AI Gateway put no price on, so its
2498+ /// cost is not the providers'), or `leak`.
24782499 pub kind: String,
24792500 pub ours: f64,
24802501 pub cloudflare: f64,
+77−6
2121 | --- | --- | --- |
2222 | Billable usage, `GET /accounts/{account}/billable-usage?from=&to=` | One row per service per day in FOCUS columns: `ServiceFamilyName`, `ServiceName`, `ChargePeriodStart`, `PricingQuantity`, `ContractedCost` / `BilledCost` / `ListCost`. Every product g1t uses appears once it is used: Workers, Workers for Platforms, D1, KV, R2, Queues, Containers, Durable Objects, Artifacts, Browser Rendering, Workers AI, Vectorize, Cloudflare for SaaS, Email. Inside an included amount the cost is 0. | `cost_lines`, source `billable_usage` |
2323 | GraphQL `artifactsEventsAdaptiveGroups` | Artifacts' own count by `date`, `eventType` and `repositoryName`. Operations are `create`, `fork`, `push`, `pull`, `delete`; errors (`rateLimited`, `serverError`, …) are kept but not counted. | `cost_lines`, source `artifacts_events`; per workspace (from the store key `<workspace>--<repo>`; a pull request's working copy, `pulls--<id>`, is its repository's workspace's, from repos' `pull_owners`) in `own_counts` as `cloudflare_git` |
24+| GraphQL `aiGatewayRequestsAdaptiveGroups`, filtered to `AI_GATEWAY_ID` | What AI Gateway priced g1t's own provider traffic at, by `date`, `provider`, `model` and `wholesale`: `count`, `sum.cost` (dollars), `sum.tokensIn`/`tokensOut`/`cacheReadTokens`/`cacheWriteTokens`. Field names checked against Cloudflare's schema (introspection of `AccountAiGatewayRequestsAdaptiveGroups{Sum,Dimensions,Filter_InputObject}`). An adaptive (sampled) dataset: an estimate, close at g1t's volumes. Only g1t's hosted models go through this gateway: a workspace's own provider is called at its own address, never here. | `cost_lines`, source `ai_gateway`, product `ai_gateway_requests`: per day and model a line `<provider>_<model>` (requests, at the gateway's cost), and at no cost `…__tokens`, `…__cache_read_tokens`, `…__cache_write_tokens`; Cloudflare-billed (unified billing) requests are prefixed `wholesale__`. Mapped to `models` (migration 0036). A re-read day replaces all its gateway lines. |
2425 | The ledger | Every charge: its cost at the price book's cost, what it was charged at price, what paid for it. | read, never written |
2526 | `pending_usage` | Month-end meters (git, storage, scans, embeddings, the cache) as they stand. | snapshotted daily into `pending_days` |
2627 | `plan_payments` | The plan's $20. | read |
3940 | Secret on g1t-billing | Permissions | Used for |
4041 | --- | --- | --- |
4142 | `CLOUDFLARE_BILLING_TOKEN` (optional) | Account: **Billing Read**, Account: **Account Analytics Read**, for the g1t account only | Reading the bill, the Artifacts events and the subscriptions |
42−| `CLOUDFLARE_USAGE_TOKEN` (exists) | Billing Read, Account Analytics Read, AI Gateway Read | The keeper; also the bill when `CLOUDFLARE_BILLING_TOKEN` is not set |
43+| `CLOUDFLARE_USAGE_TOKEN` (exists) | Billing Read, Account Analytics Read, AI Gateway Read | The keeper (settling runs from the gateway's logs); also the bill when `CLOUDFLARE_BILLING_TOKEN` is not set. AI Gateway's analytics are read with the bill's token first and, if that is refused, with this one |
4344
4445 With neither, the daily run reconciles only what g1t counted itself, and
4546 the page says the bill cannot be read. Nothing fails. To set the scoped one:
6061 1. The keeper's measurements (sandbox seconds, app requests and CPU), each
6162 a proposal now, not a direct change.
6263 2. `costs_daily`:
63− 1. Read the bill and the Artifacts events. The first run reads the last
64+ 1. Read the bill, the Artifacts events and AI Gateway's analytics. The first run reads the last
6465 31 days (GraphQL keeps 31); later runs the last 4, since Cloudflare
6566 restates recent days, or back to the last day read after a gap.
6667 Lines are upserted on `(day, source, product, meter)`, so a re-read
9798 | `embeddings` | Workers AI, Vectorize | `context` |
9899 | `security` | (Workers CPU, under `platform`) | `security` |
99100 | `domains` | Cloudflare for SaaS | `domains` |
100−| `models` | not Cloudflare: AI Gateway's settled cost on the ledger | every other task (agent runs) |
101+| `models` | not Cloudflare: AI Gateway's settled cost on the ledger; AI Gateway's own daily total (`ai_gateway_requests`) beside it, to check it | every other task (agent runs) |
101102 | `platform` | Workers, D1, KV, Queues, Email, Browser Rendering, other Durable Objects | the plan's price |
102103
103104 For each day and bucket:
106107 included allowances, so a month inside them costs $0 here as on
107108 Cloudflare's Billable usage page. `models` uses the ledger's cost of the
108109 tokens instead; that is paid to the model providers and is not on
109− Cloudflare's bill.
110+ Cloudflare's bill. Its "Cloudflare" column is what AI Gateway priced the
111+ same traffic at, which drift compares with the ledger (below); it is
112+ never added to the cost.
110113 - **Own cost** = Σ the ledger's `cost_micros` for the bucket's keys (the
111114 price book's cost when charged), plus month-end deltas. A workspace's own
112115 model provider is no cost to g1t.
121124 (`ledger.given_micros`), and all of a workspace's cost on a day it had
122125 nothing priced (free allowances);
123126 - **trial** and **open-source pool**: what `trial_micros` and
124− `oss_micros` paid.
127+ `oss_micros` paid;
128+ - **discount**: what a discount on an account's custom terms took below
129+ cost plus the margin (`ledger.discount_micros`, see
130+ [Margin floor](#margin-floor)). The usage is valued at its price, so a
131+ discounted sale never reads as margin lost.
125132
126133 Otherwise a workspace's day is split by those shares of its value at
127134 price, and the same shares of each of its buckets' cost are given, its
128135 part of running g1t included. The Team plan's included usage is sold:
129136 the plan's price paid for it. Stored on `margin_days` (`given_micros`
130− and `given_<why>_micros`) and `workspace_costs` (`given_micros`).
137+ and `given_<why>_micros`, `given_discount_micros` from migration 0036)
138+ and `workspace_costs` (`given_micros`). Sudo's Bill & pricing page lists
139+ comped, free use, trial and pool by name; the discount part is in the
140+ total until the page names it (`givenDiscountMicros`).
131141 - **Month-end meters**: a day's figure is that day's `pending_days`
132142 snapshot less the day before's, within a month. Their month-end ledger
133143 entries are left out, so nothing is counted twice.
168178 | --- | --- | --- |
169179 | Count | g1t's count and Cloudflare's differ by more than the mapping's `drift_percent` (10%) | Find out what Cloudflare counts: compare its events with `own_counts` `artifacts_*` and `cost_operations`. If it counts more (binding reads, `ls-refs`), either change repos' `operation_mapping` so customers are charged for what Cloudflare counts, or leave it and let the per-unit cost rise (below). |
170180 | Cost | Cloudflare charged more than `drift_percent` away from the price book's cost of the same usage, with at least `min_daily_cost` | A price is stale: check the proposals. |
181+| Cost, on `models` | What AI Gateway priced g1t's own provider traffic at over the 7 days, against the ledger's model cost for the same days (billed to g1t: comped, free and trial use included, a workspace's own provider not), more than the `ai_gateway_requests` mapping's `drift_percent` (10%) apart, with at least `min_daily_cost`. Compared once the gateway has been read; then a ledger with none of it is drift too | The gateway higher: model calls g1t paid for and charged no one: runs not settled yet (they catch up within the hour), runs with no session, a run started without a billing ticket, or something else on g1t's gateway. The ledger higher: runs that reached a provider without the gateway. The detail adds why the gateway's own figure may be off: prompt-cache read and write tokens (the gateway prices them at its rates for cache tokens, which can lag the provider's; check against the provider's invoice), requests Cloudflare billed itself (unified billing: on Cloudflare's bill, not a provider's), and models with no price. Days are UTC by when a request ran (gateway) and when a charge was entered (ledger), so a run across midnight shifts a little between days; the 7-day sum absorbs it. |
182+| Unpriced | Over the 7 days, a model in AI Gateway's analytics with tokens and $0 cost, or runs settled with `runs.gateway_note` (the gateway could not price all of a run) | The gateway has no price for a model g1t runs: add it in the gateway (custom cost) or route away from it. Until then those runs are charged no less than the sandbox reported (Claude Code's own price table), never $0 silently. |
171183 | Leak | Cost of at least `min_daily_cost` and nothing charged for it (never for `platform`), or a meter in `unmapped` | Map the meter (below), or decide it is overhead (`platform`). |
172184
173185 ## Prices: versions, proposals, notice
248260 emailed again weekly. The red bar on every sudo page shows margin,
249261 overall and leak alerts.
250262
263+## Model costs
264+
265+Every model call g1t pays for is an agent run's (the `claude` CLI in the
266+sandbox, `crates/runner`); the only other model is Workers AI's embeddings,
267+which are on Cloudflare's bill (`embeddings`). How each reaches the ledger:
268+
269+| Call | Who pays | Run and session | Ledger cost | Settled to the gateway |
270+| --- | --- | --- | --- | --- |
271+| Agent run through the model proxy (`services/models`) on g1t's hosted models | g1t | `runs` row; session `ms_…` in `cf-aig-metadata` | On finish, the sandbox's figure (Claude Code's `total_cost_usd`, at its own price table, cache tokens included) | Yes, every 15 minutes |
272+| Agent run straight to the gateway (no `MODELS_URL`) | g1t | `runs` row; session `rs_…` in `cf-aig-metadata` (`services/runner` `gatewaySession`) | As above | Yes |
273+| Agent run with no gateway (`AI_GATEWAY_ID` empty, self-hosting) | g1t's key | `runs` row, no session | The sandbox's figure | No: nothing to settle against |
274+| Agent run on a workspace's own provider | The workspace | `runs` row, `billed_to = 'workspace'`, no session | None (no cost to g1t) | No; never on g1t's gateway |
275+| A sandbox that died before reporting | g1t | as its route | Charged from the gateway when settled | Yes |
276+| Embeddings (indexing) | g1t | none (Workers AI) | Month-end `context` meter | No: Cloudflare's bill, `embeddings` bucket |
277+| Embeddings (queries, search and agent context) | g1t | none | None: not charged, by design | No: in Cloudflare's `embeddings` line, shared out |
278+
279+**Settling.** A run's charge is corrected to what AI Gateway priced its
280+session's requests at (`settled_cost` in `keeper.rs`). The gateway's
281+figure is trusted in full: it is not held to the $100 cap on a sandbox's
282+own report. It is never taken below what the sandbox reported when it
283+cannot be the whole cost: a request with tokens and no cost (a model the
284+gateway has no price for) or more logs than are read (2,000). Such a run
285+keeps `runs.gateway_note`, its correction says why, and it raises the
286+**Unpriced** drift. g1t keeps no token rates of its own: the first figure
287+is Claude Code's, the final one the gateway's.
288+
289+**The daily total.** AI Gateway's analytics for the day (above) against
290+the ledger's model cost is the check that nothing slips past: a model call
291+with no run, or a run never settled, shows as **Cost** drift on `models`.
292+The gateway's per-request `cost` is its estimate from its own price list:
293+it can be off for prompt-cache tokens, for requests Cloudflare bills
294+itself, and for models it has no price for. The drift's detail says when
295+any of those were in the window; the provider's invoice is the last word.
296+
297+### Margin floor
298+
299+A sold charge is cost × (1 + `MARGIN_PERCENT`), rounded up (`margin_on`;
300+`charge_micros` for a sandbox's own report). Terms change it only as
301+follows (`Terms::discounted`, `Billing::charged`):
302+
303+- **Standard**: charged in full.
304+- **Comped**, `FREE_WHILE_BUILDING`, the plan's included usage, the trial,
305+ the open-source pool, and overruns g1t covers: given, and counted by why
306+ (above).
307+- **Custom, with a discount**: the discount comes off, and what it took
308+ below cost plus the margin is written on the entry as
309+ `ledger.discount_micros` and counted as given (**discount**), so the sale
310+ is valued at its price and charged plus given is never under cost plus the
311+ margin. On a settlement correction it moves with the charge (less than
312+ nothing when the charge comes down).
313+- **Goodwill credits** (overages) are separate, given by staff on purpose:
314+ their margin part first, the cost only up to the cap, each audited.
315+
316+Every usage path goes through this: `finish_run`, settling, sandbox time,
317+features and builds (`charge_feature`), and the month-end meters.
318+
251319 ## Token usage
252320
253321 The model proxy (`services/models`) reads Anthropic's `usage` from every
276344 Every create is `IF NOT EXISTS` and every seed `INSERT OR IGNORE`; the one
277345 `ALTER` is applied once by D1's migration tracking. Migration
278346 `0023_one_operation_mapping.sql` drops `billable_units` (see above).
347+Migration `0036_model_costs_in_full.sql` adds `ledger.discount_micros`,
348+`margin_days.given_discount_micros`, `runs.gateway_note` and the
349+`ai_gateway_requests` → `models` mapping.
279350
280351 ## Spend caps
281352
+2−0
10821082 givenFreeMicros?: number;
10831083 givenTrialMicros?: number;
10841084 givenPoolMicros?: number;
1085+ /** What discounts on an account's terms took below cost plus the margin: given, not margin lost. */
1086+ givenDiscountMicros?: number;
10851087 /** costMicros by who g1t pays: Cloudflare's bill (billed, after included allowances) and model providers (tokens, not on Cloudflare's bill). */
10861088 /** What the plan's included usage paid for, at price: money in for usage, paid out of plansMicros. */
10871089 includedMicros?: number;
+19−0
1+-- Model costs counted in full, and margin kept on every sale.
2+--
3+-- What a discount on an account's terms took below cost plus the margin:
4+-- counted as given (why "discount"), so a discounted sale is valued at its
5+-- price and never reads as margin lost. Negative on a correction down.
6+ALTER TABLE ledger ADD COLUMN discount_micros INTEGER NOT NULL DEFAULT 0;
7+ALTER TABLE margin_days ADD COLUMN given_discount_micros INTEGER NOT NULL DEFAULT 0;
8+
9+-- Why a run was settled at no less than the sandbox reported instead of
10+-- at AI Gateway's figure: a model the gateway has no price for, or more
11+-- logs than were read. Null when the gateway's figure was the whole cost.
12+ALTER TABLE runs ADD COLUMN gateway_note TEXT;
13+
14+-- AI Gateway's analytics (cost_lines source ai_gateway): what the gateway
15+-- priced g1t's own provider traffic at, per day and model. Its cost is the
16+-- models bucket's "Cloudflare" side, checked against the ledger's model
17+-- cost (which stays the bucket's cost).
18+INSERT OR IGNORE INTO cost_map (product, meter, bucket, price_meter, own_meter, scale_to_own, note, updated_at, updated_by) VALUES
19+ ('ai_gateway_requests', '*', 'models', NULL, NULL, 0, 'What AI Gateway priced g1t''s own model traffic at, per model', '2026-10-07T00:00:00Z', 'migration');
+23−0
760760 }
761761
762762 #[test]
763+ fn a_discount_below_cost_plus_the_margin_is_counted_as_given() {
764+ // $1 of model cost at 20%: $1.20 is the floor of what a sale is worth.
765+ let base = crate::charge_micros(1.0, 20);
766+ assert_eq!(base, 1_200_000);
767+ // Standard: all of it sold, nothing given.
768+ assert_eq!(terms(TermsKind::Standard, 0).discounted(base), (1_200_000, 0));
769+ // 30% off: charged $0.84, under cost; the $0.36 below the floor is
770+ // given, so charged plus given is never under cost plus the margin.
771+ let (charged, given) = terms(TermsKind::Custom, 30).discounted(base);
772+ assert_eq!((charged, given), (840_000, 360_000));
773+ assert_eq!(charged + given, base);
774+ // Over 100% is everything given, never a negative charge.
775+ assert_eq!(terms(TermsKind::Custom, 250).discounted(base), (0, base));
776+ // Comped is counted as comped by the reconciliation, not here.
777+ assert_eq!(terms(TermsKind::Comped, 0).discounted(base), (0, 0));
778+ // Every discount: charged plus given is the whole charge.
779+ for percent in 0..=100 {
780+ let (charged, given) = terms(TermsKind::Custom, percent).discounted(base);
781+ assert_eq!(charged + given, base, "{percent}% off");
782+ }
783+ }
784+
785+ #[test]
763786 fn terms_read_plainly_in_the_audit_log() {
764787 let custom = Terms { ceiling_micros: Some(50_000_000), note: "Design partner".into(), ..terms(TermsKind::Custom, 20) };
765788 assert_eq!(describe(&custom), "custom (20% off, ceiling $50.00): Design partner");
+200−0
238238 ))
239239 }
240240
241+/// Where a cost line came from: AI Gateway's analytics, what it priced
242+/// g1t's own provider traffic at.
243+pub(crate) const SOURCE_GATEWAY: &str = "ai_gateway";
244+/// The product of AI Gateway's lines. Not `ai_gateway`, which is what a
245+/// billable-usage line from Cloudflare for AI Gateway itself would slug to.
246+pub(crate) const GATEWAY_PRODUCT: &str = "ai_gateway_requests";
247+/// Meter suffixes of a model's token lines (no cost; for the drift).
248+pub(crate) const GATEWAY_TOKENS: &str = "__tokens";
249+pub(crate) const GATEWAY_CACHE_READ: &str = "__cache_read_tokens";
250+pub(crate) const GATEWAY_CACHE_WRITE: &str = "__cache_write_tokens";
251+/// The meter prefix of requests Cloudflare billed itself (unified billing),
252+/// which are on Cloudflare's bill as well as here.
253+pub(crate) const GATEWAY_WHOLESALE: &str = "wholesale__";
254+
255+/// What AI Gateway priced each day's requests at, per provider and model,
256+/// for g1t's gateway only: GraphQL `aiGatewayRequestsAdaptiveGroups`, with
257+/// `sum.cost` (dollars), the tokens it priced and whether Cloudflare billed
258+/// the request itself (`wholesale`). Field names checked against the
259+/// schema (`AccountAiGatewayRequestsAdaptiveGroupsSum` and `…Dimensions`).
260+pub(crate) const GATEWAY_QUERY: &str = "query ($account: String!, $gateway: String!, $since: Date!, $until: Date!) {
261+ viewer { accounts(filter: { accountTag: $account }) {
262+ aiGatewayRequestsAdaptiveGroups(limit: 10000, filter: { date_geq: $since, date_leq: $until, gateway: $gateway }) {
263+ count
264+ sum { cost tokensIn tokensOut cacheReadTokens cacheWriteTokens }
265+ dimensions { date provider model wholesale }
266+ }
267+ } }
268+}";
269+
270+pub(crate) fn gateway_variables(account: &str, gateway: &str, since: &str, until: &str) -> Value {
271+ json!({ "query": GATEWAY_QUERY, "variables": { "account": account, "gateway": gateway, "since": since, "until": until } })
272+}
273+
274+/// AI Gateway's analytics as lines: per day and model, the requests at
275+/// what the gateway priced them (meter `<provider>_<model>`), and their
276+/// tokens, cache reads and cache writes at no cost (`…__tokens`, …).
277+/// Errors when GraphQL does.
278+pub(crate) fn lines_from_gateway(body: &Value) -> std::result::Result<Vec<CostLine>, String> {
279+ if let Some(errors) = body["errors"].as_array().filter(|e| !e.is_empty()) {
280+ return Err(format!("AI Gateway analytics failed: {}", Value::Array(errors.clone())));
281+ }
282+ let groups = body["data"]["viewer"]["accounts"][0]["aiGatewayRequestsAdaptiveGroups"].as_array().cloned().unwrap_or_default();
283+ let mut lines = Vec::new();
284+ for g in &groups {
285+ let d = &g["dimensions"];
286+ let Some(day) = d["date"].as_str().filter(|day| day.len() >= 10) else { continue };
287+ let provider = d["provider"].as_str().unwrap_or("unknown");
288+ let model = d["model"].as_str().unwrap_or("unknown");
289+ let wholesale = d["wholesale"].as_u64().unwrap_or(0) == 1;
290+ let name = format!("{}{}", if wholesale { GATEWAY_WHOLESALE } else { "" }, slug(&format!("{provider} {model}")));
291+ let sum = |key: &str| g["sum"][key].as_f64().unwrap_or(0.0);
292+ let line = |meter: String, unit: &str, quantity: f64, cost_usd: f64| CostLine {
293+ day: day[..10].to_owned(),
294+ source: SOURCE_GATEWAY,
295+ product: GATEWAY_PRODUCT.to_owned(),
296+ meter,
297+ unit: unit.to_owned(),
298+ quantity,
299+ cost_usd,
300+ raw_name: format!("AI Gateway / {provider} / {model}{}", if wholesale { " (billed by Cloudflare)" } else { "" }),
301+ };
302+ lines.push(line(name.clone(), "requests", g["count"].as_f64().unwrap_or(0.0), sum("cost").max(0.0)));
303+ lines.push(line(format!("{name}{GATEWAY_TOKENS}"), "tokens", sum("tokensIn") + sum("tokensOut"), 0.0));
304+ for (suffix, key) in [(GATEWAY_CACHE_READ, "cacheReadTokens"), (GATEWAY_CACHE_WRITE, "cacheWriteTokens")] {
305+ if sum(key) > 0.0 {
306+ lines.push(line(format!("{name}{suffix}"), "tokens", sum(key), 0.0));
307+ }
308+ }
309+ }
310+ Ok(aggregate(lines))
311+}
312+
313+/// What the gateway's lines over a window say about whether its cost can
314+/// be taken as what the providers bill g1t.
315+#[derive(Clone, Debug, Default, PartialEq)]
316+pub(crate) struct GatewayCaveats {
317+ /// Models the gateway put no price on although they used tokens.
318+ pub unpriced: Vec<String>,
319+ pub cache_read_tokens: f64,
320+ pub cache_write_tokens: f64,
321+ /// What Cloudflare billed itself (unified billing), in dollars: on its
322+ /// bill too, so not a cost paid to a provider.
323+ pub wholesale_usd: f64,
324+ /// Runs settled with the gateway's figure short (see `keeper::settled_cost`).
325+ pub short_runs: u32,
326+}
327+
328+/// The caveats in AI Gateway's lines (any days, any order).
329+pub(crate) fn gateway_caveats(lines: &[(String, f64, f64)]) -> GatewayCaveats {
330+ let mut out = GatewayCaveats::default();
331+ let mut cost: BTreeMap<&str, f64> = BTreeMap::new();
332+ let mut tokens: BTreeMap<&str, f64> = BTreeMap::new();
333+ for (meter, quantity, cost_usd) in lines {
334+ if let Some(model) = meter.strip_suffix(GATEWAY_TOKENS) {
335+ *tokens.entry(model).or_default() += quantity;
336+ } else if meter.ends_with(GATEWAY_CACHE_READ) {
337+ out.cache_read_tokens += quantity;
338+ } else if meter.ends_with(GATEWAY_CACHE_WRITE) {
339+ out.cache_write_tokens += quantity;
340+ } else {
341+ *cost.entry(meter.as_str()).or_default() += cost_usd;
342+ if meter.starts_with(GATEWAY_WHOLESALE) {
343+ out.wholesale_usd += cost_usd;
344+ }
345+ }
346+ }
347+ for (model, used) in tokens {
348+ if used > 0.0 && cost.get(model).copied().unwrap_or(0.0) <= 0.0 {
349+ out.unpriced.push(model.to_owned());
350+ }
351+ }
352+ out
353+}
354+
241355 /// The workspace a repository in the store belongs to: keys are
242356 /// `<workspace>--<repo>`; a pull request's working copy (`pulls--<id>`) is
243357 /// its repository's workspace's, from `owners` (repos' `pull_owners`), else
463577 },
464578 Err(error) => problems.push(format!("Artifacts events could not be read: {error}")),
465579 }
580+ // What AI Gateway priced g1t's own provider traffic at, each day:
581+ // the total the ledger's model cost is checked against (`margin`).
582+ if !keeper.gateway().is_empty() {
583+ match keeper
584+ .graphql_either(gateway_variables(keeper.account(), keeper.gateway(), &since, &until))
585+ .await
586+ .map_err(|e| e.to_string())
587+ .and_then(|body| lines_from_gateway(&body))
588+ {
589+ Ok(lines) => {
590+ // A day's models are read whole: one that is gone from a
591+ // re-read day must not keep its old line.
592+ self.db
593+ .prepare("DELETE FROM cost_lines WHERE source = ?1 AND day >= ?2 AND day <= ?3")
594+ .bind(&[SOURCE_GATEWAY.into(), since.as_str().into(), until.as_str().into()])?
595+ .run()
596+ .await?;
597+ written += self.upsert_lines(&lines, &fetched_at).await?;
598+ }
599+ Err(error) => problems.push(format!("AI Gateway's analytics could not be read: {error}")),
600+ }
601+ }
466602 Ok(Some((since, until, written)))
467603 }
468604
645781 mod tests {
646782 use super::*;
647783
784+ /// AI Gateway's analytics in the shape of the schema
785+ /// (`aiGatewayRequestsAdaptiveGroups`: `count`, `sum`, `dimensions`).
786+ fn gateway_fixture() -> Value {
787+ let group = |day: &str, provider: &str, model: &str, wholesale: u8, count: u64, cost: f64, tokens: (f64, f64, f64, f64)| {
788+ json!({
789+ "count": count,
790+ "sum": { "cost": cost, "tokensIn": tokens.0, "tokensOut": tokens.1, "cacheReadTokens": tokens.2, "cacheWriteTokens": tokens.3 },
791+ "dimensions": { "date": day, "provider": provider, "model": model, "wholesale": wholesale }
792+ })
793+ };
794+ json!({ "data": { "viewer": { "accounts": [{ "aiGatewayRequestsAdaptiveGroups": [
795+ group("2026-10-05", "anthropic", "claude-sonnet-5-5", 0, 120, 4.25, (900_000.0, 40_000.0, 3_000_000.0, 200_000.0)),
796+ group("2026-10-05", "anthropic", "claude-haiku-4-5-20251001", 0, 300, 0.40, (400_000.0, 20_000.0, 0.0, 0.0)),
797+ group("2026-10-06", "anthropic", "claude-new-1", 0, 12, 0.0, (80_000.0, 4_000.0, 0.0, 0.0)),
798+ group("2026-10-06", "openai", "gpt-x", 1, 5, 0.10, (1_000.0, 100.0, 0.0, 0.0)),
799+ ] }] } }, "errors": null })
800+ }
801+
802+ #[test]
803+ fn the_gateway_query_names_the_fields_its_schema_has() {
804+ // As checked against Cloudflare's GraphQL schema (introspection of
805+ // AccountAiGatewayRequestsAdaptiveGroups{,Sum,Dimensions,Filter}).
806+ for field in ["aiGatewayRequestsAdaptiveGroups", "date_geq", "date_leq", "gateway: $gateway", "count", "cost", "tokensIn", "tokensOut", "cacheReadTokens", "cacheWriteTokens", "date", "provider", "model", "wholesale"] {
807+ assert!(GATEWAY_QUERY.contains(field), "{field}");
808+ }
809+ let body = gateway_variables("acct", "g1t", "2026-10-01", "2026-10-07");
810+ assert_eq!(body["variables"]["gateway"], "g1t");
811+ }
812+
813+ #[test]
814+ fn what_the_gateway_priced_is_kept_per_day_and_model_with_its_tokens() {
815+ let lines = lines_from_gateway(&gateway_fixture()).unwrap();
816+ let get = |day: &str, meter: &str| lines.iter().find(|l| l.day == day && l.meter == meter).unwrap_or_else(|| panic!("{day} {meter}"));
817+ let sonnet = get("2026-10-05", "anthropic_claude_sonnet_5_5");
818+ assert_eq!((sonnet.source, sonnet.product.as_str(), sonnet.quantity, sonnet.cost_usd), (SOURCE_GATEWAY, GATEWAY_PRODUCT, 120.0, 4.25));
819+ assert_eq!(get("2026-10-05", "anthropic_claude_sonnet_5_5__tokens").quantity, 940_000.0);
820+ assert_eq!(get("2026-10-05", "anthropic_claude_sonnet_5_5__cache_read_tokens").quantity, 3_000_000.0);
821+ assert_eq!(get("2026-10-05", "anthropic_claude_sonnet_5_5__cache_write_tokens").cost_usd, 0.0);
822+ assert_eq!(get("2026-10-06", "wholesale__openai_gpt_x").cost_usd, 0.10);
823+ // Only the request lines carry cost: the day's total is the gateway's.
824+ let day5: f64 = lines.iter().filter(|l| l.day == "2026-10-05").map(|l| l.cost_usd).sum();
825+ assert!((day5 - 4.65).abs() < 1e-9);
826+ // Errors are a problem for the run, not lines.
827+ assert!(lines_from_gateway(&json!({ "errors": [{ "message": "not authorized for that account" }] })).is_err());
828+ }
829+
830+ #[test]
831+ fn the_gateway_lines_say_when_its_cost_cannot_be_the_providers() {
832+ let rows: Vec<(String, f64, f64)> =
833+ lines_from_gateway(&gateway_fixture()).unwrap().into_iter().map(|l| (l.meter, l.quantity, l.cost_usd)).collect();
834+ let caveats = gateway_caveats(&rows);
835+ assert_eq!(caveats.unpriced, vec!["anthropic_claude_new_1".to_owned()]);
836+ assert_eq!((caveats.cache_read_tokens, caveats.cache_write_tokens), (3_000_000.0, 200_000.0));
837+ assert!((caveats.wholesale_usd - 0.10).abs() < 1e-9);
838+ // A model priced on one day and not another is priced.
839+ let mixed = vec![
840+ ("m".to_owned(), 3.0, 0.5),
841+ ("m__tokens".to_owned(), 10.0, 0.0),
842+ ("m".to_owned(), 3.0, 0.0),
843+ ("m__tokens".to_owned(), 10.0, 0.0),
844+ ];
845+ assert!(gateway_caveats(&mixed).unpriced.is_empty());
846+ }
847+
648848 /// Billable usage as Cloudflare answered g1t on 2026-10-06 (two days,
649849 /// trimmed), plus rows past the included amounts, which cost money.
650850 fn billable_fixture() -> Value {
+1−1
669669
670670 /// A charge in millionths of a dollar for `micros` of cost plus `margin`.
671671 pub(crate) fn with_margin(cost_micros: i64, margin_percent: u32) -> i64 {
672− crate::charge_micros(cost_micros.max(0) as f64 / g1t_contracts::billing::MICROS_PER_DOLLAR as f64, margin_percent)
672+ crate::margin_on(cost_micros, margin_percent)
673673 }
674674
675675 #[cfg(test)]
+2−1
597597 // and only the account's terms change it. The plan's included usage
598598 // pays what it can; the trial and the open-source pool never pay for
599599 // deployments.
600− let charge = self.terms_of(&workspace).await?.apply(crate::charge_micros(cost, self.margin_percent));
600+ let (charge, discount) = self.terms_of(&workspace).await?.discounted(crate::charge_micros(cost, self.margin_percent));
601601 let drawn = self.draw(&workspace, charge, &month, &crate::credits::Eligible::default()).await?;
602602 description.push_str(&drawn.note());
603603 self.post_usage(crate::storage::UsageLine {
612612 drawn,
613613 })
614614 .await?;
615+ self.record_discount(&a.reference, discount).await?;
615616 self.count_spend(&workspace, cost_micros, charge - drawn.total(), &drawn).await;
616617 Ok(Outcome::Ok(true))
617618 }
+202−27
3030 use serde_json::{Value, json};
3131 use worker::{Env, Fetch, Headers, Method, Request, RequestInit, Result};
3232
33−use crate::{Billing, RunRow, charge_micros};
33+use crate::{Billing, RunRow};
3434
3535 /// The cron that also checks costs against Cloudflare's bill.
3636 pub(crate) const DAILY: &str = "17 4 * * *";
110110 format!("https://api.cloudflare.com/client/v4/accounts/{}{path}", self.account)
111111 }
112112
113− /// What AI Gateway priced a session's requests at, in dollars, and how
114− /// many there were.
115− async fn session_cost(&self, session: &str) -> Result<(f64, u32)> {
116− let mut cost = 0.0;
117− let mut count = 0;
118− for page in 1..=40 {
113+ /// AI Gateway's id, empty when there is none.
114+ pub(crate) fn gateway(&self) -> &str {
115+ &self.gateway
116+ }
117+
118+ /// A GraphQL query with the bill's token, and on failure with the
119+ /// keeper's (AI Gateway Read), when that is a different token.
120+ pub(crate) async fn graphql_either(&self, body: Value) -> Result<Value> {
121+ match self.graphql(body.clone()).await {
122+ Ok(answer) => Ok(answer),
123+ Err(error) => match &self.token {
124+ Some(token) if Some(token) != self.billing_token.as_ref() => {
125+ send_with(token, Method::Post, "https://api.cloudflare.com/client/v4/graphql", Some(body)).await
126+ }
127+ _ => Err(error),
128+ },
129+ }
130+ }
131+
132+ /// What AI Gateway priced a session's requests at, and how many there
133+ /// were, with the requests it had no price for.
134+ async fn session_cost(&self, session: &str) -> Result<SessionCost> {
135+ let mut total = SessionCost { complete: true, ..SessionCost::default() };
136+ for page in 1..=MAX_LOG_PAGES {
119137 // The filter goes as URL-encoded JSON; the bracket form is
120138 // ignored, and would sum every log there is. Session ids are
121139 // [a-z0-9_], which need no escaping inside it.
129147 let body = self.send(Method::Get, &url, None).await?;
130148 let logs = body["result"].as_array().cloned().unwrap_or_default();
131149 for log in &logs {
132− cost += log["cost"].as_f64().unwrap_or(0.0);
133− count += 1;
150+ total.add(log);
134151 }
135152 if logs.len() < 50 {
136− break;
153+ return Ok(total);
137154 }
138155 }
139− Ok((cost, count))
156+ // More logs than were read: what was read is less than the run.
157+ total.complete = false;
158+ Ok(total)
140159 }
141160
142161 /// The account's billable usage, one row per service per day, as
362381 if delta >= 0 { delta } else { delta.max(-first_charged.max(0)) }
363382 }
364383
384+/// A session's logs are read 50 at a time, up to this many pages.
385+const MAX_LOG_PAGES: u32 = 40;
386+
387+/// What AI Gateway's logs say a session cost.
388+#[derive(Clone, Debug, Default, PartialEq)]
389+pub(crate) struct SessionCost {
390+ /// What the gateway priced the requests at, in dollars.
391+ pub cost_usd: f64,
392+ pub requests: u32,
393+ /// Requests that used tokens but that the gateway put no price on: a
394+ /// model it has no price for. Their cost is not in `cost_usd`.
395+ pub unpriced: u32,
396+ /// The models of those, for the statement and the drift.
397+ pub unpriced_models: Vec<String>,
398+ /// False when there were more logs than were read.
399+ pub complete: bool,
400+}
401+
402+impl SessionCost {
403+ /// Adds one log, read leniently: `cost` in dollars, `tokens_in` and
404+ /// `tokens_out`, `cached` for an answer from the gateway's own cache
405+ /// (which costs nothing).
406+ pub(crate) fn add(&mut self, log: &Value) {
407+ self.requests += 1;
408+ let number = |key: &str| log[key].as_f64().or_else(|| log[key].as_str().and_then(|s| s.parse().ok()));
409+ let cost = number("cost").filter(|c| c.is_finite() && *c > 0.0);
410+ let tokens = number("tokens_in").unwrap_or(0.0) + number("tokens_out").unwrap_or(0.0);
411+ let cached = log["cached"].as_bool().unwrap_or(false);
412+ match cost {
413+ Some(cost) => self.cost_usd += cost,
414+ None if tokens > 0.0 && !cached => {
415+ self.unpriced += 1;
416+ let model = log["model"].as_str().unwrap_or("an unnamed model").to_owned();
417+ if !self.unpriced_models.contains(&model) {
418+ self.unpriced_models.push(model);
419+ }
420+ }
421+ None => {}
422+ }
423+ }
424+
425+ /// Whether the gateway's figure is the whole of what the run cost.
426+ pub(crate) fn whole(&self) -> bool {
427+ self.complete && self.unpriced == 0
428+ }
429+}
430+
431+/// The cost a run is settled at, in millionths: the gateway's figure when
432+/// it priced every request; otherwise (a model it has no price for, or
433+/// more logs than were read) never less than the sandbox reported, since
434+/// the gateway's sum is then short of what the provider bills. With a
435+/// reason for the statement and the drift when it is not the gateway's
436+/// figure alone.
437+pub(crate) fn settled_cost(reported_micros: i64, gateway: &SessionCost) -> (i64, Option<String>) {
438+ // Not held to MAX_RUN_COST_USD: the gateway's figure is trusted.
439+ let priced = if gateway.cost_usd.is_finite() { (gateway.cost_usd.max(0.0) * MICROS_PER_DOLLAR as f64).ceil() as i64 } else { 0 };
440+ if gateway.whole() {
441+ return (priced, None);
442+ }
443+ let mut why = Vec::new();
444+ if gateway.unpriced > 0 {
445+ why.push(format!(
446+ "AI Gateway has no price for {} of its {} requests ({})",
447+ gateway.unpriced,
448+ gateway.requests,
449+ gateway.unpriced_models.join(", ")
450+ ));
451+ }
452+ if !gateway.complete {
453+ why.push(format!("more than {} of its requests were logged", gateway.requests));
454+ }
455+ (priced.max(reported_micros.max(0)), Some(why.join("; ")))
456+}
457+
365458 fn ms(timestamp: &str) -> u64 {
366459 // RFC 3339 in UTC, as g1t writes them.
367460 worker::js_sys::Date::parse(timestamp) as u64
483576 .await?
484577 .results::<Unsettled>()?;
485578 for run in runs {
486− let (cost_usd, requests) = match keeper.session_cost(&run.session_id).await {
579+ let gateway = match keeper.session_cost(&run.session_id).await {
487580 Ok(found) => found,
488581 Err(error) => {
489582 worker::console_error!("could not read gateway logs for {}: {error}", run.id);
491584 }
492585 };
493586 let since = ms(run.finished_at.as_deref().unwrap_or(&run.created_at));
494− if requests == 0 && now.saturating_sub(since) < GIVE_UP_AFTER_MS {
587+ if gateway.requests == 0 && now.saturating_sub(since) < GIVE_UP_AFTER_MS {
495588 continue;
496589 }
497− self.settle(&run, cost_usd, requests).await?;
590+ self.settle(&run, &gateway).await?;
498591 }
499592 Ok(())
500593 }
501594
502− async fn settle(&self, run: &Unsettled, cost_usd: f64, requests: u32) -> Result<()> {
595+ async fn settle(&self, run: &Unsettled, gateway: &SessionCost) -> Result<()> {
596+ let requests = gateway.requests;
503597 let row = RunRow {
504598 workspace: run.workspace.clone(),
505599 repo: run.repo.clone(),
509603 token_hash: run.token_hash.clone(),
510604 billed_to: run.billed_to.clone(),
511605 };
512− let gateway_micros = charge_micros(cost_usd, 0);
513606 let charged = self
514607 .db
515608 .prepare("SELECT cost_micros, description, amount_micros FROM ledger WHERE reference = ?")
516609 .bind(&[run.id.as_str().into()])?
517610 .first::<Charged>(None)
518611 .await?;
612+ let reported = charged.as_ref().and_then(|c| c.cost_micros).unwrap_or(0);
613+ // The gateway's figure; never under what the sandbox reported when
614+ // the gateway could not price all of it (see `settled_cost`).
615+ let (gateway_micros, short) = settled_cost(reported, gateway);
519616 let terms = self.terms_of(&run.workspace).await?;
617+ // A cost's charge on the account's terms, and what a discount gave
618+ // below cost plus the margin (counted as given, see `charged`).
520619 let charge_for = |micros: i64| {
521− if self.free {
522− 0
523− } else {
524− terms.apply(charge_micros(micros as f64 / MICROS_PER_DOLLAR as f64, self.margin_percent))
525− }
620+ if self.free { (0, 0) } else { terms.discounted(crate::margin_on(micros, self.margin_percent)) }
526621 };
527622 let settled_at = rfc3339(now_ms());
528623 // Claim it, so two crons never settle it twice.
529624 let claimed = self
530625 .db
531− .prepare("UPDATE runs SET settled_at = ?, gateway_cost_micros = ?, finished_at = COALESCE(finished_at, ?) WHERE id = ? AND settled_at IS NULL RETURNING id")
626+ .prepare(
627+ "UPDATE runs SET settled_at = ?, gateway_cost_micros = ?, gateway_note = ?, finished_at = COALESCE(finished_at, ?)
628+ WHERE id = ? AND settled_at IS NULL RETURNING id",
629+ )
532630 .bind(&[
533631 settled_at.as_str().into(),
534632 (gateway_micros as f64).into(),
633+ short.as_deref().map_or(worker::wasm_bindgen::JsValue::NULL, Into::into),
535634 settled_at.as_str().into(),
536635 run.id.as_str().into(),
537636 ])?
540639 if claimed.is_none() || requests == 0 {
541640 return Ok(());
542641 }
642+ if let Some(why) = &short {
643+ worker::console_warn!("run {} settled at no less than reported: {why}", run.id);
644+ }
645+ let short_note = short.as_ref().map_or(String::new(), |why| format!(" ({why}; charged at no less than the sandbox reported)"));
543646 let free_note = if self.free { " (free while g1t is being built out)" } else { "" };
544647 match charged {
545648 // Never reported: charged now, from the gateway's figure.
546649 None => {
547− let charge = charge_for(gateway_micros);
650+ let (charge, discount) = charge_for(gateway_micros);
548651 let eligible = crate::credits::eligible_for(Some(g1t_contracts::billing::ComputeKind::Agent), None);
549652 let drawn = self.draw(&run.workspace, charge, &settled_at[..7], &eligible).await?;
550653 let description = format!(
551− "Work on {}#{}, settled from AI Gateway after the sandbox stopped without reporting{free_note}{}",
654+ "Work on {}#{}, settled from AI Gateway after the sandbox stopped without reporting{short_note}{free_note}{}",
552655 run.repo,
553656 run.number,
554657 drawn.note()
556659 self.enter(&run.workspace, EntryKind::Usage, -(charge - drawn.total()), &description, &run.id, Some(&row), Some(gateway_micros), None, None)
557660 .await?;
558661 self.record_drawn(&run.id, &drawn).await?;
662+ self.record_discount(&run.id, discount).await?;
559663 self.count_spend(&run.workspace, gateway_micros, charge - drawn.total(), &drawn).await;
560664 }
561665 Some(charged) => {
562− let reported = charged.cost_micros.unwrap_or(0);
563666 let delta = gateway_micros - reported;
564667 if delta == 0 {
565668 return Ok(());
566669 }
567− let change = correction(charge_for(reported), charge_for(gateway_micros), -charged.amount_micros);
670+ let ((was, was_given), (now, now_given)) = (charge_for(reported), charge_for(gateway_micros));
671+ let change = correction(was, now, -charged.amount_micros);
672+ // What the discount gives moves with the charge.
673+ let discount = now_given - was_given;
568674 // A charge up is paid for like any other charge.
569675 let drawn = if change > 0 {
570676 let eligible = crate::credits::eligible_for(Some(g1t_contracts::billing::ComputeKind::Agent), None);
573679 crate::credits::Drawn::default()
574680 };
575681 let description = format!(
576− "Correction to “{}”: AI Gateway priced its {requests} model requests at {}, not {}{}",
682+ "Correction to “{}”: AI Gateway priced its {requests} model requests at {}, not {}{short_note}{}",
577683 charged.description,
578684 crate::features::dollars(gateway_micros),
579685 crate::features::dollars(reported),
593699 )
594700 .await?;
595701 self.record_drawn(&reference, &drawn).await?;
702+ self.record_discount(&reference, discount).await?;
596703 self.count_spend(&run.workspace, delta, change - drawn.total(), &drawn).await;
597704 }
598705 }
727834 assert_eq!(correction(120_000, 100_000, 0), 0);
728835 }
729836
837+ fn logs(entries: &[Value]) -> SessionCost {
838+ let mut total = SessionCost { complete: true, ..SessionCost::default() };
839+ for log in entries {
840+ total.add(log);
841+ }
842+ total
843+ }
844+
845+ #[test]
846+ fn a_run_is_settled_at_the_gateways_figure_when_it_priced_every_request() {
847+ let gateway = logs(&[
848+ json!({ "cost": 0.012, "tokens_in": 4000, "tokens_out": 300, "model": "claude-sonnet-5-5" }),
849+ json!({ "cost": "0.003", "tokens_in": 900, "tokens_out": 40, "model": "claude-haiku-4-5" }),
850+ // Served from the gateway's own cache: no cost, and none owed.
851+ json!({ "cost": 0, "tokens_in": 900, "tokens_out": 40, "cached": true }),
852+ // An error with no tokens costs nothing either.
853+ json!({ "cost": null, "tokens_in": 0, "tokens_out": 0 }),
854+ ]);
855+ assert!(gateway.whole());
856+ assert_eq!(gateway.requests, 4);
857+ // Down from what the sandbox said, or up: the gateway's figure.
858+ assert_eq!(settled_cost(20_000, &gateway), (15_000, None));
859+ assert_eq!(settled_cost(9_000, &gateway), (15_000, None));
860+ }
861+
862+ #[test]
863+ fn a_model_the_gateway_cannot_price_is_never_settled_down_to_nothing() {
864+ let gateway = logs(&[
865+ json!({ "cost": 0.002, "tokens_in": 100, "tokens_out": 10, "model": "claude-haiku-4-5" }),
866+ json!({ "cost": 0, "tokens_in": 50_000, "tokens_out": 2_000, "model": "claude-new-1" }),
867+ json!({ "tokens_in": 50_000, "tokens_out": 2_000, "model": "claude-new-1" }),
868+ ]);
869+ assert!(!gateway.whole());
870+ assert_eq!((gateway.unpriced, gateway.unpriced_models.clone()), (2, vec!["claude-new-1".to_owned()]));
871+ // The sandbox said $0.90: kept, not cut to the gateway's $0.002.
872+ let (cost, why) = settled_cost(900_000, &gateway);
873+ assert_eq!(cost, 900_000);
874+ assert!(why.unwrap().contains("no price for 2 of its 3 requests (claude-new-1)"));
875+ // A sandbox that reported less than the gateway priced: the gateway's.
876+ assert_eq!(settled_cost(1_000, &gateway).0, 2_000);
877+ }
878+
879+ #[test]
880+ fn more_logs_than_were_read_never_settle_a_run_down() {
881+ let mut gateway = logs(&[json!({ "cost": 1.0, "tokens_in": 1, "tokens_out": 1 })]);
882+ gateway.complete = false;
883+ let (cost, why) = settled_cost(3_000_000, &gateway);
884+ assert_eq!(cost, 3_000_000);
885+ assert!(why.unwrap().contains("more than 1 of its requests"));
886+ }
887+
888+ #[test]
889+ fn a_gateway_figure_over_the_report_cap_is_charged_in_full() {
890+ // A sandbox's report is believed up to $100; the gateway's is not capped.
891+ let gateway = logs(&[json!({ "cost": 140.0, "tokens_in": 1, "tokens_out": 1 })]);
892+ assert_eq!(settled_cost(100_000_000, &gateway).0, 140_000_000);
893+ assert_eq!(crate::charge_micros(140.0, 20), 120_000_000);
894+ assert_eq!(crate::margin_on(140_000_000, 20), 168_000_000);
895+ // Exactly cost plus the margin, rounded up, in whole micros (dollars
896+ // as floats can come out a micro high), never under it.
897+ for cost in [0_i64, 1, 7, 999, 123_457, 99_999_999] {
898+ let exact = (cost * 120 + 99) / 100;
899+ assert_eq!(crate::margin_on(cost, 20), exact, "{cost}");
900+ assert!(crate::margin_on(cost, 20) * 100 >= cost * 120, "{cost}");
901+ assert!(crate::charge_micros(cost as f64 / 1e6, 20) >= exact, "{cost}");
902+ }
903+ }
904+
730905 #[test]
731906 fn a_sandbox_second_is_its_memory_and_disk_and_the_cpu_it_uses() {
732907 // An hour of sandboxes that kept a fifth of a vCPU busy.
+39−6
7575 (cost_micros * f64::from(100 + margin_percent) / 100.0).ceil() as i64
7676 }
7777
78+/// What a cost g1t trusts (the price book's, or what AI Gateway priced a
79+/// run at) is charged at: plus the margin, rounded up to a whole millionth.
80+/// Unlike `charge_micros`, never capped: only a sandbox's own report is
81+/// held to `MAX_RUN_COST_USD`, so a run that really cost more is charged
82+/// for all of it once it is settled.
83+pub fn margin_on(cost_micros: i64, margin_percent: u32) -> i64 {
84+ let cost = i128::from(cost_micros.max(0));
85+ let charge = (cost * i128::from(100 + margin_percent) + 99) / 100;
86+ i64::try_from(charge).unwrap_or(i64::MAX)
87+}
88+
7889 fn hash(token: &str) -> String {
7990 hex::encode(Sha256::digest(token.as_bytes()))
8091 }
743754 // covers a free workspace's overrun (see `credits`). Agents are
744755 // never the open-source pool's.
745756 let base = charge_micros(a.cost_usd, self.margin_percent);
746− let (charge, terms_note) = self.charged(&run.workspace, base).await?;
757+ let (charge, terms_note, discount) = self.charged(&run.workspace, base).await?;
747758 let month = credits::month_of(&rfc3339(now_ms()));
748759 let eligible = credits::eligible_for(Some(ComputeKind::Agent), None);
749760 let drawn = self.draw(&run.workspace, charge, &month, &eligible).await?;
768779 )
769780 .await?;
770781 self.record_drawn(&a.run_id, &drawn).await?;
782+ self.record_discount(&a.run_id, discount).await?;
771783 self.count_spend(&run.workspace, charge_micros(a.cost_usd, 0), charge - drawn.total(), &drawn).await;
772784 Ok(Outcome::Ok(true))
773785 }
847859 None => None,
848860 };
849861 let cost = parts.map_or(seconds as f64 * cost_per_second * price_scale, |(cost, _)| cost).ceil() as i64;
850− let (charge, terms_note) = self.charged(&workspace, credits::with_margin(cost, self.margin_percent)).await?;
862+ let (charge, terms_note, discount) = self.charged(&workspace, credits::with_margin(cost, self.margin_percent)).await?;
851863 let eligible = credits::eligible_for(a.kind, a.repo.as_deref());
852864 let drawn = self.draw(&workspace, charge, &credits::month_of(&timestamp), &eligible).await?;
853865 let charge = charge - drawn.total();
897909 ])?,
898910 ])
899911 .await?;
912+ self.record_discount(&a.reference, discount).await?;
900913 self.count_spend(&workspace, cost, charge, &drawn).await;
901914 if let Some(reservation) = &a.reservation_id {
902915 self.settle_reservation(SettleArgs { reservation_id: reservation.clone(), actual_micros: cost }).await?;
955968 /// What a workspace is charged for something that would be `base`:
956969 /// nothing while g1t is free, or as its account's terms say. With a
957970 /// note for the statement when it differs.
958− pub(crate) async fn charged(&self, workspace: &str, base: i64) -> Result<(i64, String)> {
971+ /// A charge at cost plus the margin (`base`) on the account's terms:
972+ /// what is charged, the note for the statement, and what a discount
973+ /// gave away below `base`. That last is written on the entry
974+ /// (`record_discount`) so the reconciliation counts it as given, never
975+ /// as margin lost: a sold charge is worth at least its cost plus the
976+ /// margin.
977+ pub(crate) async fn charged(&self, workspace: &str, base: i64) -> Result<(i64, String, i64)> {
959978 if self.free {
960− return Ok((0, " (free while g1t is being built out)".to_owned()));
979+ return Ok((0, " (free while g1t is being built out)".to_owned(), 0));
961980 }
962981 let terms = self.terms_of(workspace).await?;
963− let charge = terms.apply(base);
982+ let (charge, discount) = terms.discounted(base);
964983 let note = match terms.kind {
965984 TermsKind::Comped => " (comped)".to_owned(),
966985 TermsKind::Custom if terms.discount_percent > 0 && base > 0 => format!(" ({}% off)", terms.discount_percent),
967986 _ => String::new(),
968987 };
969− Ok((charge, note))
988+ Ok((charge, note, discount))
989+ }
990+
991+ /// What a discount gave away on an entry, below cost plus the margin
992+ /// (less than nothing on a correction down).
993+ pub(crate) async fn record_discount(&self, reference: &str, micros: i64) -> Result<()> {
994+ if micros == 0 {
995+ return Ok(());
996+ }
997+ self.db
998+ .prepare("UPDATE ledger SET discount_micros = ? WHERE reference = ?")
999+ .bind(&[(micros as f64).into(), reference.into()])?
1000+ .run()
1001+ .await?;
1002+ Ok(())
9701003 }
9711004 }
9721005
+206−10
111111
112112 /// What g1t gave away, by why: its own comped workspaces, free use (a
113113 /// free period, free allowances, overruns g1t covered), the trial, and the
114−/// open-source pool. The Team plan's included usage is paid for by the
115−/// plan's price, so it is sold, not given.
114+/// open-source pool, and discounts on an account's terms (what they took
115+/// below cost plus the margin, `ledger.discount_micros`). The Team plan's
116+/// included usage is paid for by the plan's price, so it is sold, not given.
116117 #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
117118 pub(crate) struct Given {
118119 pub comped: i64,
119120 pub free: i64,
120121 pub trial: i64,
121122 pub pool: i64,
123+ pub discount: i64,
122124 }
123125
124126 impl Given {
125127 pub fn total(&self) -> i64 {
126− self.comped + self.free + self.trial + self.pool
128+ self.comped + self.free + self.trial + self.pool + self.discount
127129 }
128130
129131 fn add(&mut self, other: &Given) {
131133 self.free += other.free;
132134 self.trial += other.trial;
133135 self.pool += other.pool;
136+ self.discount += other.discount;
134137 }
135138
136139 /// The same shares of `cost` as these are of `value`, at most all of it.
141144 }
142145 let given = cost as i128 * total.min(value) as i128 / value as i128;
143146 let part = |x: i64| (given * x.max(0) as i128 / total as i128) as i64;
144− Given { comped: part(self.comped), free: part(self.free), trial: part(self.trial), pool: part(self.pool) }
147+ Given {
148+ comped: part(self.comped),
149+ free: part(self.free),
150+ trial: part(self.trial),
151+ pool: part(self.pool),
152+ discount: part(self.discount),
153+ }
145154 }
146155 }
147156
368377 Cost,
369378 /// Cloudflare charged for something nothing charges customers for.
370379 Leak,
380+ /// Model usage AI Gateway put no price on: its cost is not what the
381+ /// provider bills, so neither the ledger nor the gateway total has it.
382+ Unpriced,
371383 }
372384
373385 impl DriftKind {
376388 DriftKind::Count => "count",
377389 DriftKind::Cost => "cost",
378390 DriftKind::Leak => "leak",
391+ DriftKind::Unpriced => "unpriced",
379392 }
380393 }
381394 }
413426 }
414427 }
415428 let enough = cf_cost.max(own_cost) >= min_cost_micros as f64;
416− if enough && !overhead && cf_cost > 0.0 && own_cost > 0.0 && !NOT_CLOUDFLARE.contains(&bucket) {
429+ // Models: what AI Gateway priced g1t's own provider traffic at (its
430+ // lines, as "Cloudflare's" side) against the ledger's model cost. Only
431+ // once the gateway has been read; then the ledger having none of it is
432+ // drift too (traffic no run was charged for).
433+ let models = NOT_CLOUDFLARE.contains(&bucket) && cf_cost > 0.0;
434+ if enough && !overhead && cf_cost > 0.0 && (own_cost > 0.0 || models) {
417435 let delta = delta_percent(own_cost, cf_cost);
418436 if delta.is_some_and(|d| d.abs() > threshold) {
419437 out.push(Drift { bucket: bucket.into(), kind: DriftKind::Cost, ours: own_cost, cloudflare: cf_cost, delta_percent: delta });
425443 out
426444 }
427445
446+/// What can make AI Gateway's cost differ from what the providers bill,
447+/// said for staff: cache tokens (priced by the gateway at its own rates for
448+/// them, which may lag the provider's), requests Cloudflare billed itself,
449+/// models it has no price for, and runs settled short.
450+fn caveat_notes(c: &costs::GatewayCaveats) -> Vec<String> {
451+ let mut notes = Vec::new();
452+ if c.cache_read_tokens > 0.0 || c.cache_write_tokens > 0.0 {
453+ notes.push(format!(
454+ "{} prompt-cache read and {} cache write tokens went through it: check its cost against the provider's invoice, since cache reads are billed far below input and writes above it",
455+ crate::features::thousands(c.cache_read_tokens.round() as u64),
456+ crate::features::thousands(c.cache_write_tokens.round() as u64)
457+ ));
458+ }
459+ if c.wholesale_usd > 0.0 {
460+ notes.push(format!(
461+ "{} of it Cloudflare billed itself (unified billing): that part is on Cloudflare's bill, not a provider's",
462+ dollars(micros(c.wholesale_usd))
463+ ));
464+ }
465+ if !c.unpriced.is_empty() {
466+ notes.push(format!("it has no price for {} (tokens used, $0)", c.unpriced.join(", ")));
467+ }
468+ if c.short_runs > 0 {
469+ notes.push(format!("{} runs were settled at no less than the sandbox reported because the gateway could not price all of them", c.short_runs));
470+ }
471+ notes
472+}
473+
474+/// The models drift's detail: the gateway's total against the ledger's.
475+pub(crate) fn models_detail(drift: &Drift, caveats: &costs::GatewayCaveats) -> String {
476+ let lower = drift.ours < drift.cloudflare;
477+ let mut detail = format!(
478+ "Models: AI Gateway priced g1t's own provider traffic at {} over the last {DRIFT_DAYS} days; the ledger's model cost for the same days is {} ({:+.1}%). {}",
479+ dollars(drift.cloudflare as i64),
480+ dollars(drift.ours as i64),
481+ drift.delta_percent.unwrap_or(0.0),
482+ if lower {
483+ "Model calls g1t paid for were not charged: runs not yet settled, runs with no session, or calls with no run (the ledger catches up as runs settle; a gap that stays is a leak)."
484+ } else {
485+ "The ledger counts more than the gateway priced: runs that went to a provider without the gateway, or sandbox reports the gateway could not correct."
486+ }
487+ );
488+ let notes = caveat_notes(caveats);
489+ if !notes.is_empty() {
490+ detail.push_str(" The gateway's cost may be off: ");
491+ detail.push_str(&notes.join("; "));
492+ detail.push('.');
493+ }
494+ detail
495+}
496+
497+/// The unpriced drift's detail.
498+pub(crate) fn unpriced_detail(caveats: &costs::GatewayCaveats) -> String {
499+ format!(
500+ "Models: AI Gateway's cost is not all of what the providers bill over the last {DRIFT_DAYS} days: {}. Runs on a model with no gateway price are charged no less than the sandbox reported; add the model's price to the gateway (or route away from it) so it is charged at cost.",
501+ caveat_notes(&costs::GatewayCaveats { cache_read_tokens: 0.0, cache_write_tokens: 0.0, wholesale_usd: 0.0, ..caveats.clone() }).join("; ")
502+ )
503+}
504+
505+/// Model usage AI Gateway could not price over the window, as drift on
506+/// the models bucket: models with tokens and no cost, or runs settled
507+/// short. None when there is none.
508+pub(crate) fn unpriced_drift(caveats: &costs::GatewayCaveats) -> Option<(Drift, String)> {
509+ if caveats.unpriced.is_empty() && caveats.short_runs == 0 {
510+ return None;
511+ }
512+ let drift = Drift {
513+ bucket: NOT_CLOUDFLARE[0].into(),
514+ kind: DriftKind::Unpriced,
515+ ours: f64::from(caveats.short_runs),
516+ cloudflare: caveats.unpriced.len() as f64,
517+ delta_percent: None,
518+ };
519+ Some((drift, unpriced_detail(caveats)))
520+}
521+
428522 /// When the last `days` in a row (each with enough cost to say something)
429523 /// were all under the floor: the first of them and the worst margin.
430524 /// Each item is a day's (day, revenue, cost).
651745 given_trial_micros: Option<i64>,
652746 #[serde(default)]
653747 given_pool_micros: Option<i64>,
748+ #[serde(default)]
749+ given_discount_micros: Option<i64>,
654750 }
655751
656752 impl From<MarginRow> for ProductDay {
669765 free: r.given_free_micros.unwrap_or(0),
670766 trial: r.given_trial_micros.unwrap_or(0),
671767 pool: r.given_pool_micros.unwrap_or(0),
768+ discount: r.given_discount_micros.unwrap_or(0),
672769 },
673770 }
674771 }
755852 trial: Option<i64>,
756853 oss: Option<i64>,
757854 covered: Option<i64>,
855+ discount: Option<i64>,
758856 cost: Option<i64>,
759857 }
760858 let charged_here = crate::storage::CHARGED_HERE.iter().map(|s| format!("'{s}'")).collect::<Vec<_>>().join(", ");
771869 SUM(COALESCE(trial_micros, 0)) AS trial,
772870 SUM(COALESCE(oss_micros, 0)) AS oss,
773871 SUM(COALESCE(given_micros, 0)) AS covered,
872+ SUM(COALESCE(discount_micros, 0)) AS discount,
774873 SUM(COALESCE(cost_micros, 0)) AS cost
775874 FROM ledger
776875 WHERE kind = 'usage' AND created_at >= ?1 AND created_at <= ?2 AND COALESCE(task, '') NOT IN ({charged_here})
789888 // to g1t. g1t's own workspaces are valued at price.
790889 let cost = if r.own_provider == 1 { 0 } else { r.cost.unwrap_or(0) };
791890 let cash = r.cash.unwrap_or(0);
792− let paid = cash + r.drawn.unwrap_or(0);
891+ // A discount took its part below cost plus the margin: it is
892+ // valued at price and that part counted as given, so a
893+ // discounted sale never reads as margin lost.
894+ let discount = r.discount.unwrap_or(0).max(0);
895+ let paid = cash + r.drawn.unwrap_or(0) + discount;
793896 let value = usage_value(r.internal == 1, cost, paid, self.margin_percent);
794897 let given = if r.internal == 1 {
795898 Given { comped: value, ..Given::default() }
796899 } else if paid == 0 && cost > 0 {
797900 Given { free: value, ..Given::default() }
798901 } else {
799− Given { free: r.covered.unwrap_or(0), trial: r.trial.unwrap_or(0), pool: r.oss.unwrap_or(0), comped: 0 }
902+ Given { free: r.covered.unwrap_or(0), trial: r.trial.unwrap_or(0), pool: r.oss.unwrap_or(0), comped: 0, discount }
800903 };
801904 if r.internal == 1 {
802905 internal.insert(r.workspace.clone());
9171020 statements.push(
9181021 self.db
9191022 .prepare(
920− "INSERT OR REPLACE INTO margin_days (day, bucket, cf_cost_micros, own_cost_micros, value_micros, cash_micros, cf_quantity, own_quantity, given_micros, given_comped_micros, given_free_micros, given_trial_micros, given_pool_micros, computed_at)
921− VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
1023+ "INSERT OR REPLACE INTO margin_days (day, bucket, cf_cost_micros, own_cost_micros, value_micros, cash_micros, cf_quantity, own_quantity, given_micros, given_comped_micros, given_free_micros, given_trial_micros, given_pool_micros, given_discount_micros, computed_at)
1024+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
9221025 )
9231026 .bind(&[
9241027 d.day.as_str().into(),
9341037 (d.given.free as f64).into(),
9351038 (d.given.trial as f64).into(),
9361039 (d.given.pool as f64).into(),
1040+ (d.given.discount as f64).into(),
9371041 now.as_str().into(),
9381042 ])?,
9391043 );
9751079 .collect())
9761080 }
9771081
1082+ /// What AI Gateway's lines over the days, and the runs settled in them,
1083+ /// say about whether its cost is what the providers bill.
1084+ async fn gateway_caveats(&self, since: &str, until: &str) -> Result<costs::GatewayCaveats> {
1085+ #[derive(Deserialize)]
1086+ struct Line {
1087+ meter: String,
1088+ quantity: f64,
1089+ cost_usd: f64,
1090+ }
1091+ let lines: Vec<(String, f64, f64)> = self
1092+ .db
1093+ .prepare("SELECT meter, quantity, cost_usd FROM cost_lines WHERE source = ?1 AND day >= ?2 AND day <= ?3")
1094+ .bind(&[costs::SOURCE_GATEWAY.into(), since.into(), until.into()])?
1095+ .all()
1096+ .await?
1097+ .results::<Line>()?
1098+ .into_iter()
1099+ .map(|l| (l.meter, l.quantity, l.cost_usd))
1100+ .collect();
1101+ let mut caveats = costs::gateway_caveats(&lines);
1102+ #[derive(Deserialize)]
1103+ struct Short {
1104+ n: Option<f64>,
1105+ }
1106+ caveats.short_runs = self
1107+ .db
1108+ .prepare("SELECT COUNT(*) AS n FROM runs WHERE gateway_note IS NOT NULL AND settled_at >= ?1 AND settled_at <= ?2")
1109+ .bind(&[since.into(), format!("{until}T23:59:59.999Z").into()])?
1110+ .first::<Short>(None)
1111+ .await?
1112+ .and_then(|s| s.n)
1113+ .unwrap_or(0.0) as u32;
1114+ Ok(caveats)
1115+ }
1116+
9781117 /// Drift over the last week, written to `cost_drift` (replacing the
9791118 /// last run's), with unmapped Cloudflare meters as leaks.
9801119 async fn find_drift(&self, until: &str) -> Result<Vec<(Drift, String)>> {
9861125 for d in days {
9871126 by.entry(d.bucket.clone()).or_default().push(d);
9881127 }
1128+ let caveats = self.gateway_caveats(&since, until).await?;
9891129 let mut found = Vec::new();
1130+ if let Some(drift) = unpriced_drift(&caveats) {
1131+ found.push(drift);
1132+ }
9901133 for (bucket, days) in &by {
9911134 let bucket_rules: Vec<&Rule> = rules.iter().filter(|r| &r.bucket == bucket).collect();
9921135 let threshold = bucket_rules.iter().map(|r| r.drift_percent).fold(f64::INFINITY, f64::min);
9951138 for drift in drifts(bucket, days, threshold, counted, settings.min_daily_cost_micros) {
9961139 let title = costs::bucket_title(bucket);
9971140 let detail = match drift.kind {
1141+ DriftKind::Cost if NOT_CLOUDFLARE.contains(&bucket.as_str()) => models_detail(&drift, &caveats),
9981142 DriftKind::Count => format!(
9991143 "{title}: g1t counted {}, Cloudflare {} over the last {DRIFT_DAYS} days ({:+.1}%). Customers are charged for what g1t counts; check what Cloudflare counts as a unit and change the repos service's operation_mapping (set_operation_mapping).",
10001144 crate::features::thousands(drift.ours.max(0.0).round() as u64),
10101154 DriftKind::Leak if bucket == UNMAPPED => {
10111155 format!("Cloudflare charged {} for meters no mapping claims. Map them on Costs & margin.", dollars(drift.cloudflare as i64))
10121156 }
1157+ DriftKind::Leak if NOT_CLOUDFLARE.contains(&bucket.as_str()) => format!(
1158+ "{title}: AI Gateway priced g1t's own provider traffic at {} over the last {DRIFT_DAYS} days and the ledger has no model charge for it, not even a comped or free one: model calls with no billing run behind them (a run started without a ticket, or something else using g1t's gateway).",
1159+ dollars(drift.cloudflare as i64)
1160+ ),
10131161 DriftKind::Leak => format!(
10141162 "{title}: Cloudflare charged {} over the last {DRIFT_DAYS} days and customers were charged nothing for it.",
10151163 dollars(drift.cloudflare as i64)
10161164 ),
1165+ // Raised from the gateway's lines, not per bucket.
1166+ DriftKind::Unpriced => unpriced_detail(&caveats),
10171167 };
10181168 found.push((drift, detail));
10191169 }
14361586 overall.given_free_micros += d.given.free;
14371587 overall.given_trial_micros += d.given.trial;
14381588 overall.given_pool_micros += d.given.pool;
1589+ overall.given_discount_micros += d.given.discount;
14391590 let sold = (d.cost() - d.given.total()).max(0);
14401591 if OVERHEAD.contains(&d.bucket.as_str()) {
14411592 overall.plans_micros += d.cash_micros;
18652016 let (days, workspaces) = fold(&[], &map, &[], &[], &[comped, trial, paying, free], &internal);
18662017 let models = days.iter().find(|d| d.bucket == "models").unwrap();
18672018 assert_eq!(models.cost(), 4_000_000);
1868− assert_eq!(models.given, Given { comped: 1_000_000, free: 1_000_000, trial: 500_000, pool: 0 });
2019+ assert_eq!(models.given, Given { comped: 1_000_000, free: 1_000_000, trial: 500_000, pool: 0, discount: 0 });
18692020 let given = |w: &str| workspaces.iter().find(|x| x.workspace == w).unwrap().given.total();
18702021 assert_eq!((given("flagon"), given("acme"), given("beta"), given("gamma")), (1_000_000, 500_000, 0, 1_000_000));
18712022 }
18722023
18732024 #[test]
2025+ fn a_discounted_sale_keeps_its_margin_and_counts_the_discount_as_given() {
2026+ // $1 of model cost at 20%, sold to an account with 30% off: charged
2027+ // $0.84, and $0.36 below cost plus the margin given (as usage_rows
2028+ // reads the ledger: value at price, the discount part given).
2029+ let mut sale = usage("2026-10-15", "acme", "agent", 1_200_000, 840_000, 1_000_000);
2030+ sale.given = Given { discount: 360_000, ..Given::default() };
2031+ let (days, _) = fold(&[], &BTreeMap::new(), &[], &[], &[sale], &BTreeSet::new());
2032+ let models = days.iter().find(|d| d.bucket == "models").unwrap();
2033+ assert_eq!(models.value_micros, 1_200_000);
2034+ assert_eq!(models.given, Given { discount: 300_000, ..Given::default() });
2035+ // What was sold (cost less given) still makes the margin.
2036+ let sold = models.cost() - models.given.total();
2037+ assert_eq!(margin_percent(models.cash_micros, sold).map(|m| m.round()), Some(17.0));
2038+ }
2039+
2040+ #[test]
2041+ fn the_gateways_total_against_the_ledgers_model_cost_is_drift() {
2042+ // The gateway priced $5 of g1t's own traffic; the ledger has $3.
2043+ let short = drifts("models", &[day("models", 5_000_000, 3_000_000, 3_600_000, 0.0, 0.0)], 10.0, false, 100_000);
2044+ assert_eq!(short.iter().map(|d| d.kind).collect::<Vec<_>>(), vec![DriftKind::Cost]);
2045+ assert!((short[0].delta_percent.unwrap() + 40.0).abs() < 1e-9);
2046+ // Gateway traffic with nothing on the ledger at all: cost drift and a leak.
2047+ let none = drifts("models", &[day("models", 2_000_000, 0, 0, 0.0, 0.0)], 10.0, false, 100_000);
2048+ assert_eq!(none.iter().map(|d| d.kind).collect::<Vec<_>>(), vec![DriftKind::Cost, DriftKind::Leak]);
2049+ // Within the threshold, or before the gateway was ever read: nothing.
2050+ assert!(drifts("models", &[day("models", 1_050_000, 1_000_000, 1_200_000, 0.0, 0.0)], 10.0, false, 100_000).is_empty());
2051+ assert!(drifts("models", &[day("models", 0, 1_000_000, 1_200_000, 0.0, 0.0)], 10.0, false, 100_000).is_empty());
2052+ // The detail says which way and why it may be off.
2053+ let caveats = costs::GatewayCaveats { cache_read_tokens: 3_000_000.0, unpriced: vec!["anthropic_claude_new_1".into()], ..Default::default() };
2054+ let detail = models_detail(&short[0], &caveats);
2055+ assert!(detail.contains("$5.00") && detail.contains("$3.00") && detail.contains("were not charged"), "{detail}");
2056+ assert!(detail.contains("3,000,000 prompt-cache read") && detail.contains("no price for anthropic_claude_new_1"), "{detail}");
2057+ }
2058+
2059+ #[test]
2060+ fn model_usage_the_gateway_cannot_price_is_drift_even_when_the_totals_agree() {
2061+ assert!(unpriced_drift(&costs::GatewayCaveats::default()).is_none());
2062+ // Cache tokens alone are a note on the cost drift, not drift.
2063+ assert!(unpriced_drift(&costs::GatewayCaveats { cache_write_tokens: 10.0, ..Default::default() }).is_none());
2064+ let (drift, detail) = unpriced_drift(&costs::GatewayCaveats { unpriced: vec!["anthropic_claude_new_1".into()], short_runs: 2, ..Default::default() }).unwrap();
2065+ assert_eq!((drift.bucket.as_str(), drift.kind.as_str()), ("models", "unpriced"));
2066+ assert!(detail.contains("no price for anthropic_claude_new_1") && detail.contains("2 runs were settled"), "{detail}");
2067+ }
2068+
2069+ #[test]
18742070 fn a_workspace_that_costs_more_than_it_pays_is_flagged() {
18752071 let rows = vec![("acme".to_string(), 5_000_000, 1_000_000), ("beta".to_string(), 900_000, 0), ("gamma".to_string(), 2_000_000, 3_000_000)];
18762072 let found = anomalies(&rows, 1.0, 1_000_000);
+2−1
270270 continue;
271271 }
272272 let base = credits::with_margin(cost, self.margin_percent);
273− let (charge, terms_note) = self.charged(&row.workspace, base).await?;
273+ let (charge, terms_note, discount) = self.charged(&row.workspace, base).await?;
274274 let plan = self.has_plan(&row.workspace).await?;
275275 let drawn = self.draw(&row.workspace, charge, &row.month, &month_end_eligible(&row.source, plan)).await?;
276276 let detail = if row.source == "storage" {
294294 drawn,
295295 })
296296 .await?;
297+ self.record_discount(&reference, discount).await?;
297298 }
298299 Ok(())
299300 }
+7−2
6868 canReachModel,
6969 changeSize,
7070 chooseTier,
71+ gatewaySession,
7172 lastAttemptFailed,
7273 modelEnv,
7374 parseRouting,
13931394 session = opened.value;
13941395 }
13951396 const own = session?.billedTo === "workspace";
1397+ // Straight to the gateway, without the proxy: the run still gets a
1398+ // session there, so billing settles it to what the gateway priced it
1399+ // at instead of leaving the sandbox's own figure.
1400+ const direct = !session && this.env.AI_GATEWAY_ID ? gatewaySession() : undefined;
13961401 // A workspace's own provider is not routed by tier: it runs the model
13971402 // its route names, or for an Anthropic provider, the large tier's.
13981403 const routed = routing.tiers[own ? "large" : tier];
14051410 task,
14061411 model: own ? `${modelName} (${session?.providerName ?? "own provider"})` : modelName,
14071412 billedTo: own ? "workspace" : "g1t",
1408− session: own ? null : (session?.id ?? null),
1413+ session: own ? null : (session?.id ?? direct ?? null),
14091414 tier: own ? null : tier,
14101415 });
14111416 if (!ticket.ok) return ticket;
14231428 // the harness's small tasks too.
14241429 ...(session.model ? { ANTHROPIC_SMALL_FAST_MODEL: session.model } : {}),
14251430 }
1426− : modelEnv(this.env, routing, task, tier, tags);
1431+ : modelEnv(this.env, routing, task, tier, direct ? { ...tags, session: direct } : tags);
14271432 if (ticket.value) {
14281433 // How the sandbox says what the run cost. Kept from the agent.
14291434 vars.BILLING_RUN = ticket.value.runId;
+12−0
99 canReachModel,
1010 changeSize,
1111 chooseTier,
12+ gatewaySession,
1213 lastAttemptFailed,
1314 modelEnv,
1415 parseRouting,
127128 });
128129 });
129130
131+test("a run straight to the gateway carries its session, so billing can settle it", () => {
132+ const session = gatewaySession();
133+ assert.match(session, /^rs_[0-9a-f]{24}$/);
134+ assert.notEqual(gatewaySession(), session);
135+ const vars = modelEnv({ ...direct, AI_GATEWAY_ID: "g1t" }, routes, "implement", "small", { ...tags, session });
136+ const metadata = JSON.parse(customHeaders(vars)["cf-aig-metadata"]);
137+ assert.equal(metadata.session, session);
138+ // The gateway keeps at most five metadata entries.
139+ assert.ok(Object.keys(metadata).length <= 5);
140+});
141+
130142 test("an authenticated gateway is sent its token", () => {
131143 const vars = modelEnv({ ...direct, AI_GATEWAY_ID: "g1t", AI_GATEWAY_TOKEN: "tok" }, routes, "implement", "large", tags);
132144 assert.equal(customHeaders(vars)["cf-aig-authorization"], "Bearer tok");
+18−3
123123 AI_GATEWAY_TOKEN?: string;
124124 };
125125
126−/** What a run is for, attached to each of its requests at the gateway. */
127−export type RunTags = { repo: string; pull: number };
126+/**
127+ * What a run is for, attached to each of its requests at the gateway.
128+ * `session` is the run's id there: billing finds the run's requests by it
129+ * and settles the run to what the gateway priced them at.
130+ */
131+export type RunTags = { repo: string; pull: number; session?: string };
128132
133+/**
134+ * A session id for a run that goes straight to the gateway (no model
135+ * proxy): `rs_` and 24 hex characters, which billing's log filter needs
136+ * no escaping for.
137+ */
138+export function gatewaySession(): string {
139+ const bytes = crypto.getRandomValues(new Uint8Array(12));
140+ return `rs_${Array.from(bytes, (b) => b.toString(16).padStart(2, "0")).join("")}`;
141+}
142+
129143 /** Whether there is a way to reach a model at all. */
130144 export function canReachModel(env: ModelRouting): boolean {
131145 return Boolean(env.ANTHROPIC_API_KEY || (env.AI_GATEWAY_ID && env.AI_GATEWAY_TOKEN));
160174
161175 vars.ANTHROPIC_BASE_URL = `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${env.AI_GATEWAY_ID}/anthropic`;
162176 // The gateway logs these with every request, so spend and failures can
163− // be read per kind of work, tier, repository and pull request.
177+ // be read per kind of work, tier, repository and pull request; and by
178+ // the run's session, which billing settles the run's charge by.
164179 const headers = [`cf-aig-metadata: ${JSON.stringify({ task, tier, ...tags })}`];
165180 if (env.AI_GATEWAY_TOKEN) {
166181 vars.AI_GATEWAY_TOKEN = env.AI_GATEWAY_TOKEN;