Skip to content

A job refused for want of a machine is tried again within seconds, not at the next minute's sweep

Closedsyntaqx opened this #292

Follows #266. When the runner refuses a job because no machine is free (There is no container instance that can be provided to this Durable Object), the job goes back to the queue. It is tried again only at the actions service's next one-minute cron sweep, or when another job's request happens to call start_queued.

Over 280 CI and Deploy jobs on 2026-10-20/21, jobs whose queue wait overlapped such a refusal waited a median 13.5 s and p90 120 s to be assigned; other jobs waited 5.2 s / 22 s. This is the tail of time to first step (p90 queued to assigned 32 s, max 346 s). On 2026-10-21 Sandbox4Core logged bursts of refusals at 00:32 to 00:34, 13:04 and 14:07 to 14:16; the telemetry query stops at 100 events.

Options:

  • Retry a refused job sooner: for example, a Durable Object alarm or a queue message with a 10 to 15 s delay, backing off, rather than waiting for the minute sweep.
  • Start the next queued job when a sandbox stops (onStop already reports it), so a freed machine is used at once.

Work

On it

s

syntaqxClaude CodeReleased

since

Sessions

In Chat

Why the work is the way it is, and how sure that is

Activity

ssyntaqxvia Claude Codeclaimed #292·
syntaqxopened flagon-io/g1t#303 for this·
syntaqxclosed this by merging flagon-io/g1t#303·
ssyntaqxvia Claude Codeno longer holds #292: the work item closed·

Sign in to comment.