Skip to content
391 linesCodeBlameRaw
1# Mirroring: keeping remotes in sync
2
3> **2026-10-08:** "bidirectional synchronization from GitHub or whatever
4> providers … as well as setting up a different remote (think self-hosted g1t
5> and mirroring to the hosted g1t.sh platform) so that it will keep remotes in
6> sync with each other." And: "GitHub is down, I want to run GitHub's
7> workflows on g1t, but only until GitHub is back up. But what if I push on
8> g1t?" And: "It needs to be highly clear when a repository is mirrored, and
9> things are disabled … It should still work as a repository when it's in
10> mirror mode."
11
12This replaces the three loose modes shipped in Projects step 5 (import,
13mirror, push, see PLAN.md "Projects") with one model that has a clear answer
14for every push, wherever it lands.
15
16## What exists today
17
18- GitHub only, through the `g1t-sh` App. `github_repos.mode` is one of
19 `import | mirror | push` (integrations `0003_github.sql`).
20- **Mirror** pulls on GitHub's push webhook and calls `mirror::copy` with
21 `Prune::Yes`, which *overwrites* g1t's refs and deletes g1t-only branches.
22 Anything pushed to g1t is silently lost at the next sync.
23- **Push** ("Move to g1t") forwards each `git.push` to GitHub. A ref GitHub
24 refuses ends up in `last_error`, and nothing reconciles it.
25- Mirrored and imported refs publish `git.push` with no actor. They start
26 `.g1t/workflows` like any push. They are written straight to the store, so
27 they skip `workflow_gate`.
28- `.github/workflows` is never read, except as a reusable `uses:` target.
29- No provider port. `Endpoint::github` and `https://github.com/{full_name}.git`
30 are hard-coded. `ProjectSource { kind: "mirror", provider }` is declared but
31 unused.
32
33The first three behaviours are the bugs this plan fixes. Pushes are lost
34silently, divergence goes undetected, and anyone who can push to GitHub can
35change a workflow that runs with g1t's secrets.
36
37## The rule
38
39**Every linked repository has exactly one leader at any moment. A write
40reaches the leader first, or it doesn't land. The leader's word is final.**
41
42Everything else follows from that rule:
43
44- *Two-way sync* doesn't need its own mode. When g1t follows, a push to g1t is
45 forwarded to the leader *before* g1t's ref moves (write-through). When g1t
46 leads, a fast-forward push on the other side is adopted. Both sides accept
47 pushes, and there is always a tie-breaker.
48- *Conflicts* can only happen when the leader was unreachable and someone
49 wrote anyway (failover), or when a follower took a write it couldn't
50 forward. A conflict is never resolved by overwriting silently.
51- *Echo loops* are impossible. When an update's new sha equals the tip
52 already held, the update is a no-op. That covers a webhook announcing our
53 own push and a chain of g1t instances alike.
54
55## Words (UI and docs)
56
57| State | Badge on the repository | Meaning |
58| --- | --- | --- |
59| No link | (none) | An ordinary g1t repository. *Import* is a one-time copy that leaves no link. |
60| **Mirror** | `Mirror of github.com/acme/web` | The remote leads and g1t follows. Pushes to g1t are forwarded to the remote. |
61| **Mirrored** | `Mirrored to github.com/acme/web` (+ more) | g1t leads and the remotes follow. A repository can have any number of these. |
62| **Failover** | `Mirror of github.com/acme/web · Failover` (amber) | The leader is unreachable, so g1t is temporarily accepting writes. They are replayed when the leader returns. |
63| **Reconciling** | `… · 2 branches diverged` (red) | Some branches moved on both sides and need a decision. Other branches keep syncing. |
64
65A repository is a Mirror of **at most one** remote, and it can also be
66Mirrored to others. Self-hosted g1t can be a Mirror of GitHub and Mirrored
67to g1t.sh, for example. Creating a link that would make a cycle is refused.
68For g1t-to-g1t links we can ask the other side for its leader chain.
69
70The existing modes map as follows: `mirror` → Mirror, `push` → Mirrored,
71`import` → no link.
72
73## Pushes, case by case
74
75### When g1t is a Mirror (the remote leads)
76
77| Where the push happens | What happens |
78| --- | --- |
79| On the remote | The webhook arrives (or a poll runs when there is no webhook), g1t fetches, and the ref moves. A force-push by the leader is followed, because the leader may rewrite history. The old tip is kept at `refs/g1t/replaced/<branch>/<time>` for 30 days, and the change shows in the activity feed. Nothing is lost silently. |
80| On g1t (person, agent, merge button, web edit) | **Write-through.** g1t receives the pack, pushes it to the leader with a compare-and-swap (expected old = our tip), and moves its own ref only when the leader accepts. If the leader refuses (branch protection, non-fast-forward, a hook), the push fails with the leader's own message passed through as `remote:` lines. Branch protection on the remote is enforced for free. |
81| On g1t, branch only on g1t | There are no g1t-only branches on a Mirror. Every branch is forwarded. This removes the current `Prune::Yes` data loss: g1t never holds anything the leader doesn't. |
82| On g1t, leader unreachable | Depends on the failover setting (below). The default refuses with: `acme/web is a mirror of github.com/acme/web, which isn't answering. Pushes resume when it's back, or turn on failover in Settings → Mirroring.` |
83
84The cost is latency: a push to a Mirror takes as long as a push to the
85remote plus our own write. That trade buys g1t's commits never diverging
86while the leader is up. The 40 MB pack relay limit in `mirror.rs` applies to
87forwarded pushes as well, and the error message has to say so.
88
89### When g1t is Mirrored (g1t leads)
90
91| Where the push happens | What happens |
92| --- | --- |
93| On g1t | It's g1t's normal push with all of g1t's rules. After it lands, a `git.push` event queues a forward to each follower (this exists today). A follower that refuses (its own protection, for example) marks that ref **stuck** on that remote and shows it on the repository. Retries back off and never force. |
94| On the remote, fast-forward | **Default: adopt.** The webhook triggers a fetch, and the update goes through g1t's ref rules as if the pusher had pushed to g1t, with the pusher mapped to a g1t account (see Actors). If g1t's rules allow it, the ref moves and the update is forwarded to the other followers. If they refuse (a protected branch, say), the ref is marked **diverged**. |
95| On the remote, not a fast-forward | Marked **diverged**. g1t never force-pushes over it without a person's or agent's decision. |
96
97A setting covers pushes made directly on the remote:
98
99- **Adopt fast-forwards** (default)
100- **Overwrite them**: g1t force-pushes its tip and keeps the replaced
101 remote tip under `refs/g1t/replaced/…`. This suits teams that consider the
102 remote read-only.
103- **Lock the remote**: g1t creates a ruleset on the GitHub repository that
104 lets only the g1t App update refs. This needs `administration: write` on
105 the App, which we request only when someone picks this option.
106
107### Diverged branches
108
109A diverged branch stops syncing and every other branch keeps going. Both
110tips are kept: g1t's at the branch, and the other side's at
111`refs/remotes/<remote>/<branch>`, which is browsable and diffable. The
112repository shows `main diverged from github.com/acme/web: 3 commits here, 1
113there` with three actions:
114
1151. **Keep g1t's**, which force-pushes to the other side (the old tip is kept).
1162. **Keep theirs**, which moves g1t's ref (the old tip is kept).
1173. **Merge them**, which opens a pull request merging the other tip into the
118 branch. One click hands it to @g1t, and it lands through the normal rules.
119
120## Failover: GitHub is down, and I push on g1t
121
122Failover is a *state* of a Mirror, not a separate mode. The setting
123"When github.com/acme/web is unreachable" has three choices:
124
125- **Stop accepting pushes** (default for new links)
126- **Ask me**: a banner and an inbox item for repository admins offer
127 **Start failover**.
128- **Fail over automatically**
129
130**Entering failover.** The leader is unreachable when forwarded writes fail
131with a timeout, a connection error or a 5xx, *and* a health probe of
132`info/refs` fails 3 times over 2 minutes. A provider status page alone isn't
133enough, because they lag and over-report. While in failover:
134
135- g1t accepts pushes, merges and agent work on every branch. Branch rules
136 still apply: g1t imports a read-only copy of the remote's protection when
137 the link is made, refreshes it on every sync, and enforces it during
138 failover. "Main needs a reviewed pull request" still holds while GitHub is
139 down.
140- Each ref records its **base**, the last sha both sides agreed on.
141- The repository wears the amber Failover badge with the duration and the
142 count of commits waiting to go back.
143
144**Leaving failover.** Once the probe succeeds 3 times over 5 minutes, each
145branch changed during failover is replayed:
146
147| Remote since base | g1t since base | Result |
148| --- | --- | --- |
149| unchanged | moved | Fast-forward push to the remote. |
150| moved | unchanged | Fetch, as normal. |
151| moved | moved, one contains the other | Fast-forward whichever side is behind. |
152| moved | moved, split | **Diverged** (above). |
153| (remote refuses: protected branch) | moved | **Replayed as a pull request on GitHub** from `g1t/failover/<branch>`, titled *Changes made on g1t while GitHub was unreachable*. g1t can't push straight to a protected main, and shouldn't. |
154
155The repository goes back to a plain Mirror when nothing is diverged or
156stuck. Until then it shows Reconciling. An admin can end failover by hand at
157any time, and the replay runs the same way.
158
159## Workflows
160
161### Which files run on g1t
162
163| Files | Default | Setting |
164| --- | --- | --- |
165| `.g1t/workflows` | Run on every push, whatever its origin | None needed. These are g1t's files: having them means you want them run. Workflows can filter on `g1t.event.origin` (`local` / `remote`). |
166| `.github/workflows` | **Off** | **Run GitHub's workflows on g1t: Off · While GitHub Actions is down · Always.** |
167
168This deliberately narrows the rule in `crates/actions/src/workflow.rs` ("g1t
169never reads `.github`"). g1t reads `.github/workflows` only for repositories
170linked to GitHub, and only when this setting is on. When both folders define
171the same `name:`, `.g1t` wins.
172
173### "While GitHub Actions is down"
174
175This is the scenario from the request. The window opens when either of these
176happens:
177
1781. A pushed commit gets no check suite on GitHub within 10 minutes. This
179 needs `checks: read` on the App and is the most reliable signal, because it
180 means GitHub really didn't run it.
1812. Someone with admin on the repository clicks **GitHub Actions is down: run
182 here** on the repository or on a single commit. A per-commit **Run on g1t**
183 button is always available, in any setting.
184
185GitHub's status page is shown next to the banner for context. It never opens
186or closes the window by itself.
187
188The window closes once GitHub starts check suites again for new commits. Runs
189already started on g1t finish.
190
191While the window is open:
192
193- Pushes and pull requests on g1t, **including failover pushes**, run the
194 `.github` workflows that match.
195- Commits pushed during the window that GitHub never ran are **backfilled**:
196 g1t offers to run them in one click.
197- Results go back to GitHub as check runs named `g1t / <workflow> /
198 <job>` (needs `checks: write`). People on GitHub see them, and GitHub's
199 required-check rules can name them if the team chooses.
200
201**Deploys and other side effects.** A workflow running in two places can
202deploy twice. In "While down" mode, jobs that declare an `environment:` or
203use a secret named `*DEPLOY*`/`*TOKEN*` that only GitHub has are **held for
204approval** by default. The setting is "Side-effect jobs in GitHub's
205workflows: hold for approval (default) · run". In "Always" mode they run.
206That mode is for teams that have moved CI to g1t and keep GitHub for code
207review.
208
209**Secrets.** GitHub won't give out secret values. When the setting is turned
210on, g1t lists every `secrets.X` the `.github` workflows reference, ticks off
211the ones already set on the project, and blocks "Always" until each is set
212or marked "not needed". A run that hits a missing secret fails with
213`NPM_TOKEN is set on GitHub but not on g1t: add it in Project → Secrets`.
214It does not fail with an empty string.
215
216**After GitHub returns.** Failover commits replayed to GitHub start GitHub's
217own workflows there. That is expected, because GitHub's checks are what
218GitHub's rules ask for. g1t's check runs for the same sha are already posted,
219so reviewers see both. Deploy jobs held on g1t can be discarded once GitHub
220has deployed.
221
222### Workflow changes that arrive from a remote
223
224Today a commit fetched from GitHub that edits `.g1t/workflows` runs with
225g1t's secrets, and its author never needed `workflow_files: write` on g1t.
226That gets fixed in step 1, before any of the rest:
227
228- A fetched push that changes workflow files, or `.github/workflows` while
229 that setting is on, runs only if the pusher maps to a g1t account with
230 `workflow_files: write` on the repository.
231- Otherwise its runs wait for approval: `Workflow changed on GitHub by
232 @octo, who has no write access to workflows here. Approve run?` This is
233 the same mechanism as runs from forks.
234
235## Actors
236
237GitHub's push webhook names the pusher, and `github_accounts` already links
238GitHub users to g1t users. A fetched push is attributed to the linked g1t
239account, or to `github:<login>` (shown greyed out, not a g1t user) when there
240is none. Today such pushes have no actor. Webhooks, audit and the workflow
241`actor` all get the attributed value.
242
243## What works on a Mirror, and what's disabled
244
245This table is also the source for the UI's disabled states. Every disabled
246control uses the shadcn Tooltip with the reason and the setting that would
247enable it.
248
249| | Mirror (remote leads) | Mirror in Failover | Mirrored (g1t leads) |
250| --- | --- | --- | --- |
251| Browse, clone, fetch, blame, search | ✓ | ✓ | ✓ |
252| Push branches and tags | ✓ forwarded to the remote | ✓ held, replayed later | ✓ |
253| Branch protection | The remote's (read-only copy shown) | The remote's copy, enforced by g1t | g1t's |
254| Pull requests | **The remote's.** Listed on g1t; *New pull request* and agents open them on the remote; *Merge* merges there through the API under its rules | g1t pull requests allowed. Each is replayed as a remote pull request when the remote returns | g1t's |
255| Merge queue, required g1t checks | Disabled: "Merges happen on GitHub, which leads this repository." | ✓ on g1t's copy of the rules | ✓ |
256| Issues, agents, plans | ✓ g1t's own. Agents push branches that are forwarded, and open pull requests on the remote | ✓ | ✓ |
257| `.g1t/workflows` | ✓ | ✓ | ✓ |
258| `.github/workflows` | Per setting | Per setting | Per setting (only if a follower is GitHub) |
259| Projects, deployments, previews, secrets | ✓ (the on-ramp) | ✓ | ✓ |
260| Releases | Read from the remote (later) | Held | g1t's, copied to followers (later) |
261| Rename, transfer | g1t side only. The link stays | Same | Same |
262| Delete branch, set default branch | Forwarded / read from the remote | Held | g1t's, forwarded |
263| Archive | Stops syncing. The link is kept and paused | Not allowed | Stops forwarding |
264| Unlink | Becomes an ordinary repository (choice: keep g1t-side failover commits) | Must reconcile first | Followers stop receiving |
265
266Other places that must show the state, not only the badge:
267
268- **Clone box:** `This is a mirror of github.com/acme/web. Pushes here are
269 forwarded there.`
270- **Push output:** `remote: forwarded to github.com/acme/web (412 ms)`, so
271 people learn the model the first time they push.
272- **Repository header:** the sync dot (`in sync · 40s ago`, `syncing`,
273 `failover · 23m`, `2 diverged`, `stuck on gitlab.com/…`). It opens
274 **Settings → Mirroring**, which lists every remote, each branch's state,
275 the last 50 sync events and **Sync now**.
276- **Workspace and project lists:** a small mirror glyph with the leader's
277 host.
278
279## The provider port
280
281All of this lives behind one port, so GitLab, Bitbucket, plain git and other
282g1t instances are each an adapter. The Rust trait lives in
283`services/integrations`, because the connections live there:
284
285```rust
286trait Remote {
287 fn kind(&self) -> RemoteKind; // github | g1t | gitlab | bitbucket | git
288 fn capabilities(&self) -> Capabilities; // webhooks, checks, lock, pull_requests, protection
289 async fn endpoint(&self) -> Result<mirror::Endpoint>;// URL + fresh credential for mirror.rs
290 async fn parse_event(&self, req: &Request) -> Result<Vec<RemoteRefChange>>; // verify + normalise
291 async fn probe(&self) -> Health; // info/refs reachability
292 async fn protection(&self) -> Result<Vec<RuleCopy>>; // read the remote's branch rules
293 async fn lock(&self, on: bool) -> Result<()>; // optional
294 async fn post_check(&self, sha: &str, run: &CheckRun) -> Result<()>; // optional
295 async fn pull_requests(&self) -> Result<PullRequestPort>; // optional, step 7
296}
297```
298
299- Each `Capabilities` flag that's off greys out the matching UI and setting.
300 The UI never offers a lever the provider can't honour.
301- Providers without webhooks (plain `git`, or a self-hosted g1t behind a
302 firewall) get a **poll** every 1 to 10 minutes with backoff, using
303 `ls-remote` and comparing tips. The poll uses the same code path as the
304 webhook.
305- **`g1t` adapter (self-hosted ↔ g1t.sh).** It authenticates with a
306 fine-grained token on the other instance (`contents: write`,
307 `webhooks: write`) and registers its own webhook there through our API.
308 Because we control both ends, write-through works in either direction, and
309 the cycle check can ask the other side for its chain. Issues and pull
310 requests between g1t instances come later, over the public API.
311
312## Data
313
314**integrations, `remotes`** (replaces `github_repos`; existing rows are
315migrated):
316- `id`, `repo_id`, `workspace`, `kind`, `url`, `role` (`leader` |
317 `follower`), and `connection_id` (installation or token row).
318- `provider_ref`, provider-specific JSON (GitHub: `installation_id`,
319 `github_repo_id`).
320- `settings` JSON: on-unreachable, remote-push policy, `.github` workflows,
321 side-effect jobs.
322- `state` (`ok` | `failover` | `reconciling` | `paused` | `error`),
323 `state_since`, `last_error`, `synced_at`.
324- A unique index on `(repo_id) WHERE role = 'leader'` enforces at most one
325 leader.
326
327**repos, `ref_sync`** (per repository, inside the repository's store so it
328moves atomically with refs):
329- `remote_id`, `ref`, `base_sha` (last agreed), `remote_sha` (last seen
330 there).
331- `state` (`ok` | `ahead` | `behind` | `held` | `diverged` | `stuck`),
332 `updated_at`.
333
334**repos, repository row:** `mirror_of` (remote id or null), cached from
335integrations through a `remote.updated` event. The git front door, commit
336API and merge paths can then decide forward-or-refuse without an RPC.
337`lifecycle::archived_refusal` is the template for a matching
338`mirror::route(repo, ref)` used at the same call sites.
339
340**contracts, `GitPush`** gains `origin: { kind: "local" | "remote",
341remote_id?, provider?, pusher? }`. Actions, webhooks, audit and the inbox
342read it. `ProjectSource`'s unused `mirror` variant is dropped: a Mirror is
343still a hosted repository with a full copy, so its project stays `hosted`, as
344PLAN.md already decided.
345
346**Events:** `remote.linked`, `remote.updated`, `remote.unlinked`,
347`remote.failover_started`, `remote.failover_ended`, `ref.diverged`,
348`ref.reconciled`. These are in the public webhook catalogue, so people can
349build alerts on them.
350
351## Build order
352
3531. **Safety and the port.** The `Remote` trait with the GitHub adapter. Move
354 `github_repos` to `remotes`. Add `ref_sync` with base shas. Add `origin`
355 and the attributed actor on `GitPush`. Echo suppression by sha. Keep
356 replaced tips instead of overwriting silently. Approval for workflow
357 changes arriving from a remote. Badge, sync dot, and **Settings →
358 Mirroring** (read-only state).
3592. **Write-through Mirror.** Forward pushes from the git front door, the
360 commit API, merges and agents. Pass the leader's refusals through. Apply
361 the disabled-feature table in the UI. Read-only copy of the remote's
362 protection.
3633. **Mirrored with remote pushes handled.** Adopt / overwrite / lock, stuck
364 refs, diverged refs with the three resolutions.
3654. **Failover.** Probe, ask/automatic, enforcement of the protection copy,
366 replay with the table above, pull-request replay for protected branches.
3675. **GitHub's workflows on g1t.** The three-way setting, the
368 missing-check-suite signal, *Run on g1t*, backfill, check runs back to
369 GitHub, held side-effect jobs, the secrets checklist.
3706. **g1t ↔ g1t.** The `g1t` adapter, polling, the cycle check, docs for
371 self-hosted ↔ g1t.sh.
3727. **Remote pull requests on Mirrors.** List, open, merge through the API,
373 agents opening theirs there (already "Not yet" in PLAN.md step 5).
3748. **GitLab, Bitbucket, plain git** adapters.
375
376Each step updates `guides/github.md` and a new `guides/mirroring.md` in the
377same change (docs standard). `guides/github.md`'s "anything pushed to the
378g1t copy directly is overwritten" goes away with step 1.
379
380## Decisions to confirm
381
3821. **Failover defaults to "Stop accepting pushes"** for new links, with
383 "Ask me" one click away. Automatic failover is opt-in, because it creates
384 work that has to be replayed.
3852. **On a Mirror, pull requests live on the leader**, and g1t has no pull
386 requests of its own there (except during failover). One place to merge
387 avoids two review histories for one branch. Making g1t the leader is the
388 way to get g1t's review and queue.
3893. **App permissions are asked for when a feature needs them**:
390 `checks: read/write` for step 5 and `administration: write` only for
391 *Lock the remote*. They are not requested up front.