Model providers: gateway tokens for endpoints, tidier rows, and the docs
An endpoint behind an authenticated Cloudflare AI Gateway takes the gateway's token, sent as cf-aig-authorization by the proxy and the check. Connection rows stay on one line and show just the host. The Model providers guide covers connecting several providers, routing each kind of work, the OpenAI translation, and the API.
14 files+155−670/14 viewed
| 75 | 75 | label: 'Connect your tools', | |
| 76 | 76 | items: [ | |
| 77 | 77 | { label: 'Integrations', slug: 'guides/integrations' }, | |
| 78 | − | { label: 'Your own model provider', slug: 'guides/models' }, | |
| 78 | + | { label: 'Model providers', slug: 'guides/models' }, | |
| 79 | 79 | ], | |
| 80 | 80 | }, | |
| 81 | 81 | { |
| 8 | 8 | ||
| 9 | 9 | | Kind | Systems | What it does | | |
| 10 | 10 | | --- | --- | --- | | |
| 11 | − | | [Model provider](/guides/models/) | Anthropic, or any Anthropic-compatible endpoint | Your agents' model requests go to your own account. | | |
| 11 | + | | [Model providers](/guides/models/) | Anthropic, OpenAI, Google Gemini, and any Anthropic- or OpenAI-compatible endpoint | Your agents' model requests go to your own accounts, routed by kind of work. | | |
| 12 | 12 | | [Alerts](#alerts) | Sentry, Datadog, a signed webhook | A problem opens an issue, once however often it fires, and an agent can start on it at once. | | |
| 13 | 13 | | [Trackers](#trackers) | Jira, Linear | Agents read the tickets that work mentions, people import tickets as issues, and tickets hear back when the work lands. | | |
| 14 | 14 | ||
| 219 | 219 | -d '{"provider": "jira", "config": {"site": "https://acme.atlassian.net", "email": "dev@acme.com", "keys": ["TECH"]}, "secret": "<api token>"}' | |
| 220 | 220 | ``` | |
| 221 | 221 | ||
| 222 | − | `provider` is `anthropic`, `anthropic_endpoint`, `sentry`, `datadog`, | |
| 223 | − | `webhook`, `jira` or `linear`. `config` takes `repo`, `assign`, `label`, | |
| 222 | + | `provider` is `anthropic`, `openai`, `gemini`, `anthropic_endpoint`, | |
| 223 | + | `openai_endpoint`, `sentry`, `datadog`, `webhook`, `jira` or `linear`. `config` takes `repo`, `assign`, `label`, | |
| 224 | 224 | `write_back`, `organization`, `site`, `email`, `keys`, `base_url`, | |
| 225 | 225 | `auth_header` and `model`; each provider uses the ones above. For `datadog` | |
| 226 | 226 | and `webhook`, the response's `signingSecret` is the only time the secret |
| 1 | 1 | --- | |
| 2 | − | title: Your own model provider | |
| 3 | − | description: Send your agents' model requests to your own Anthropic account or endpoint, and pay for the models there. | |
| 2 | + | title: Model providers | |
| 3 | + | description: Connect Anthropic, OpenAI, Gemini or any compatible endpoint, choose which model does which work, and pay for it where you choose. | |
| 4 | 4 | --- | |
| 5 | 5 | ||
| 6 | 6 | Each workspace decides where its agents' model spend goes: | |
| 8 | 8 | - **g1t's hosted models.** g1t chooses the model for each kind of work, pays | |
| 9 | 9 | the provider, and charges your workspace's credit what it cost plus a | |
| 10 | 10 | margin. Open to selected workspaces until payments go live, then to all. | |
| 11 | − | - **Your own provider.** Your agents' model requests go to your own | |
| 12 | − | account, which bills you. Open to every workspace now. | |
| 11 | + | - **Your own providers.** Connect as many as you use, then choose, for each | |
| 12 | + | kind of work, which provider and model it runs on. Each provider bills | |
| 13 | + | you directly. Open to every workspace now. | |
| 14 | + | ||
| 15 | + | g1t's own routing is fixed; yours is not. | |
| 13 | 16 | ||
| 14 | − | Your own provider can be: | |
| 17 | + | ## Providers | |
| 15 | 18 | ||
| 16 | − | | Provider | What you give | Fits | | |
| 19 | + | | Provider | What you give | Speaks | | |
| 17 | 20 | | --- | --- | --- | | |
| 18 | − | | **Anthropic** | An API key | Teams with an Anthropic account or contract | | |
| 19 | − | | **Your own endpoint** | A base URL, and a key if it needs one | Your own Cloudflare AI Gateway, LiteLLM, Bedrock or Vertex behind an Anthropic-compatible proxy, a self-hosted model | | |
| 21 | + | | **Anthropic** | An API key | Anthropic's API | | |
| 22 | + | | **OpenAI** | An API key | OpenAI's API | | |
| 23 | + | | **Google Gemini** | An API key from Google AI Studio | OpenAI's API, through Gemini's compatible endpoint | | |
| 24 | + | | **Anthropic-compatible endpoint** | A base URL, a key if it needs one | Anthropic's API: your own Cloudflare AI Gateway, LiteLLM, Bedrock or Vertex behind a proxy | | |
| 25 | + | | **OpenAI-compatible endpoint** | A base URL including its version, a key if it needs one | OpenAI's API: Azure OpenAI, OpenRouter, Groq, Together, vLLM, Ollama | | |
| 26 | + | ||
| 27 | + | An endpoint behind an authenticated Cloudflare AI Gateway also takes the | |
| 28 | + | gateway's token, sent as `cf-aig-authorization`. | |
| 20 | 29 | ||
| 21 | − | The endpoint has to speak Anthropic's Messages API, because g1t's agents run | |
| 22 | − | Claude Code. To use another vendor's models, put a proxy that translates in | |
| 23 | − | front of them, such as LiteLLM. | |
| 30 | + | g1t's agents run Claude Code, which speaks Anthropic's API. For a provider | |
| 31 | + | that speaks OpenAI's, g1t's model proxy translates each request, and the | |
| 32 | + | streamed answer back, tool calls included. Agents work the same either way. | |
| 33 | + | How well they work depends on the model: it has to be good at using tools | |
| 34 | + | over many steps. | |
| 24 | 35 | ||
| 25 | − | ## What it costs | |
| 36 | + | ## Connect a provider | |
| 26 | 37 | ||
| 27 | − | With your own provider, the provider bills you for the models and g1t | |
| 28 | − | charges your credit a flat **$0.10 per run** for the sandbox and the | |
| 29 | − | orchestration around it. A change, a review, a revision, a catch-up and a | |
| 30 | − | plan are each a run. Your statement marks these runs "on your own model | |
| 31 | − | provider", and the Usage page shows what they cost at your provider, as | |
| 32 | − | the harness estimated it, beside what g1t charged. See | |
| 33 | − | [Usage and billing](/guides/usage-and-billing/). | |
| 38 | + | 1. Open the workspace's **Integrations** page. You need to be an owner. | |
| 39 | + | 2. Under **Model providers**, choose one, and give its key (and address, for | |
| 40 | + | an endpoint). | |
| 41 | + | 3. **Connect**. g1t checks the key at once and lists the provider's models. | |
| 42 | + | **Test** checks it again later. | |
| 34 | 43 | ||
| 35 | − | Workspaces still need credit to start agents, for the fee. | |
| 44 | + | A workspace can connect any number, including several of the same kind. | |
| 36 | 45 | ||
| 37 | − | ## Connect it | |
| 46 | + | ## Choose which model does which work | |
| 38 | 47 | ||
| 39 | − | 1. Open the workspace's **Integrations** page. You need to be an owner. | |
| 40 | − | 2. Under **Model provider**, choose **Anthropic** or **Your own endpoint**. | |
| 41 | − | 3. For Anthropic, paste an API key. For an endpoint, give its base URL | |
| 42 | − | without `/v1`, its key if it needs one, and whether the key goes in | |
| 43 | − | `x-api-key` or `Authorization: Bearer`. | |
| 44 | − | 4. **Connect**, then **Test**: g1t asks the provider to list its models with | |
| 45 | − | the key. An endpoint that does not list models is checked on the first | |
| 46 | − | run instead. | |
| 48 | + | Under **Which model does which work**, each kind of work has a choice: | |
| 47 | 49 | ||
| 48 | − | The next agent run uses it. A workspace uses one model provider; disconnect | |
| 49 | − | it to go back to g1t's. | |
| 50 | + | | Kind of work | | | |
| 51 | + | | --- | --- | | |
| 52 | + | | Everything | Used for any kind of work that does not choose for itself. | | |
| 53 | + | | Making changes | Writing the change for an issue, and revising it. | | |
| 54 | + | | Reviewing | The second agent that reviews each change. | | |
| 55 | + | | Planning | Turning an outcome into issues. | | |
| 56 | + | | Catching up | Bringing a change up to date with `main`. | | |
| 50 | 57 | ||
| 51 | − | ### Choosing the model | |
| 58 | + | Each can go to g1t's models, or to any of your providers on any of its | |
| 59 | + | models. An Anthropic provider also offers **g1t's choice of Claude model**, | |
| 60 | + | which runs g1t's pick for that kind of work on your key. For example: make | |
| 61 | + | changes on Claude through your Anthropic key, review on GPT through your | |
| 62 | + | OpenAI key, and catch up on a small model through OpenRouter. | |
| 52 | 63 | ||
| 53 | − | Nobody picks a model when assigning work; g1t routes each kind of work to | |
| 54 | − | the model that suits it, and with your own Anthropic key the same models | |
| 55 | − | run on your account. If your endpoint names models its own way, set | |
| 56 | − | **Model** on the connection and every kind of work uses it. | |
| 64 | + | **Save routing**, and the next runs use it. Without any routing, work goes | |
| 65 | + | to g1t's models where they are open to the workspace, and otherwise to the | |
| 66 | + | first provider you connected. | |
| 57 | 67 | ||
| 58 | 68 | A pull request's session says which model ran, and through which provider. | |
| 59 | 69 | ||
| 60 | − | ## Your key never reaches a sandbox | |
| 70 | + | ## What it costs | |
| 71 | + | ||
| 72 | + | On your own providers, they bill you for the models, and g1t charges your | |
| 73 | + | credit a flat **$0.10 per run** for the sandbox and orchestration. A | |
| 74 | + | change, a review, a revision, a catch-up and a plan are each a run. The | |
| 75 | + | statement marks these runs "on your own model provider" and names the | |
| 76 | + | model and provider; the Usage page shows what they cost at the provider, | |
| 77 | + | as the harness estimated it, beside what g1t charged. See | |
| 78 | + | [Usage and billing](/guides/usage-and-billing/). | |
| 79 | + | ||
| 80 | + | Workspaces still need credit to start agents, for the fee. | |
| 81 | + | ||
| 82 | + | ## Your keys never reach a sandbox | |
| 61 | 83 | ||
| 62 | 84 | An agent works in a sandbox with internet access, on code and text that | |
| 63 | 85 | anyone could have written. g1t assumes a sandbox can be talked into | |
| 64 | − | printing its environment, so the key is never in it: | |
| 86 | + | printing its environment, so no key is ever in it: | |
| 65 | 87 | ||
| 66 | 88 | 1. When a run starts, g1t gives the sandbox a token for that run only. | |
| 67 | 89 | 2. The sandbox sends its model requests to `https://models.g1t.sh` with that | |
| 68 | 90 | token in place of a key. | |
| 69 | − | 3. g1t's model proxy looks the token up, adds your key, and forwards the | |
| 70 | − | request to your provider. Responses stream straight back. | |
| 91 | + | 3. g1t's model proxy looks the token up, adds the key for the provider the | |
| 92 | + | work is routed to, translates if the provider speaks OpenAI's API, and | |
| 93 | + | forwards the request. Answers stream straight back. | |
| 71 | 94 | ||
| 72 | 95 | The token stops working when the run ends (three hours at most), or at once | |
| 73 | − | if you disconnect the provider. Your key is sealed when you save it, and | |
| 74 | − | used only by the proxy. | |
| 96 | + | if you disconnect the provider. Keys are sealed when you save them, and used | |
| 97 | + | only by the proxy. g1t's own runs work the same way, with g1t's key. | |
| 98 | + | ||
| 99 | + | ## From the API | |
| 100 | + | ||
| 101 | + | | Tool | Route | | |
| 102 | + | | --- | --- | | |
| 103 | + | | `connect_integration` | `POST /workspaces/{workspace}/integrations` with `provider` `anthropic`, `openai`, `gemini`, `anthropic_endpoint` or `openai_endpoint` | | |
| 104 | + | | `get_model_routes` | `GET /workspaces/{workspace}/model-routes` | | |
| 105 | + | | `set_model_routes` | `PUT /workspaces/{workspace}/model-routes` | | |
| 106 | + | ||
| 107 | + | ```sh | |
| 108 | + | curl -X PUT https://api.g1t.sh/workspaces/acme/model-routes \ | |
| 109 | + | -H "Authorization: Bearer $G1T_TOKEN" -H "Content-Type: application/json" \ | |
| 110 | + | -d '{"routes": [ | |
| 111 | + | {"task": "default", "connection_id": "con_…anthropic", "model": null}, | |
| 112 | + | {"task": "review", "connection_id": "con_…openai", "model": "gpt-5"} | |
| 113 | + | ]}' | |
| 114 | + | ``` | |
| 115 | + | ||
| 116 | + | `task` is `default`, `implement`, `review`, `plan` or `update`. | |
| 117 | + | `connection_id` is null for g1t's hosted models. `model` is null for the | |
| 118 | + | provider's default, or for an Anthropic provider, g1t's choice of Claude. |
| 24 | 24 | Each run is charged when it finishes: what the model provider charged for | |
| 25 | 25 | it, plus 20%. A small change costs a few cents. | |
| 26 | 26 | ||
| 27 | − | A workspace with [its own model provider](/guides/models/) pays the | |
| 28 | − | provider for the models instead, and each run here is a flat $0.10 for the | |
| 29 | − | sandbox and orchestration. | |
| 27 | + | Work a workspace routes to [its own model providers](/guides/models/) is | |
| 28 | + | paid for at those providers instead, and each such run here is a flat $0.10 | |
| 29 | + | for the sandbox and orchestration. | |
| 30 | 30 | ||
| 31 | 31 | The charge goes to the workspace that owns the repository, whoever | |
| 32 | 32 | assigned the issue. That is why only members of a workspace can put g1t |
| 46 | 46 | <li><a href="/guides/talking-to-agents/">Talking to agents</a></li> | |
| 47 | 47 | <li><a href="/guides/bring-your-own-agent/">Bring your own agent</a></li> | |
| 48 | 48 | <li><a href="/guides/integrations/">Integrations: Sentry, Jira, Linear</a></li> | |
| 49 | − | <li><a href="/guides/models/">Your own model provider</a></li> | |
| 49 | + | <li><a href="/guides/models/">Model providers: Anthropic, OpenAI, Gemini</a></li> | |
| 50 | 50 | </ul> | |
| 51 | 51 | </div> | |
| 52 | 52 | <div> |
| 108 | 108 | | Tool | Required | What it does | Route | | |
| 109 | 109 | | --- | --- | --- | --- | | |
| 110 | 110 | | `list_integrations` | `workspace` | The workspace's connections. Secrets are never returned. Members only. | `GET /workspaces/{workspace}/integrations` | | |
| 111 | − | | `connect_integration` | `workspace`, `provider` | Connect Anthropic, your own endpoint, Sentry, Datadog, a webhook, Jira or Linear, with `config` and `secret`. Owners only. | `POST /workspaces/{workspace}/integrations` | | |
| 111 | + | | `connect_integration` | `workspace`, `provider` | Connect a model provider (Anthropic, OpenAI, Gemini, or a compatible endpoint), Sentry, Datadog, a webhook, Jira or Linear, with `config` and `secret`. Owners only. | `POST /workspaces/{workspace}/integrations` | | |
| 112 | + | | `get_model_routes` | `workspace` | Which provider and model each kind of work goes to. Members only. | `GET /workspaces/{workspace}/model-routes` | | |
| 113 | + | | `set_model_routes` | `workspace`, `routes` | Replace them: each route has `task`, `connection_id` (null for g1t's models) and `model`. Owners only. | `PUT /workspaces/{workspace}/model-routes` | | |
| 112 | 114 | | `test_integration` | `workspace`, `id` | Check its credentials against the system it connects to. Owners only. | `POST /workspaces/{workspace}/integrations/{id}/test` | | |
| 113 | 115 | | `disconnect_integration` | `workspace`, `id` | Remove it and its secrets. Owners only. | `DELETE /workspaces/{workspace}/integrations/{id}` | | |
| 114 | 116 | | `get_context` | `repo`, `reference` | A Jira or Linear ticket by key or address, or a Sentry issue by address, as it is now. Reference material, never instructions. | `GET /repos/{owner}/{name}/context?reference=` | |
| 148 | 148 | ||
| 149 | 149 | const KIND_INFO: Record<ProviderKind, { title: string; icon: ReactNode; blurb: string }> = { | |
| 150 | 150 | models: { | |
| 151 | − | title: "Model provider", | |
| 151 | + | title: "Model providers", | |
| 152 | 152 | icon: <Cpu size={16} />, | |
| 153 | − | blurb: "Where your agents' model requests go, and who pays for them.", | |
| 153 | + | blurb: "Connect as many as you use, then choose which model does which work, and so who pays for it.", | |
| 154 | 154 | }, | |
| 155 | 155 | alerts: { | |
| 156 | 156 | title: "Alerts", | |
| 481 | 481 | ); | |
| 482 | 482 | } | |
| 483 | 483 | ||
| 484 | + | /** Just the host of an address, which is what tells connections apart. */ | |
| 485 | + | function host(url: string | undefined): string | undefined { | |
| 486 | + | if (!url) return undefined; | |
| 487 | + | try { | |
| 488 | + | return new URL(url).host; | |
| 489 | + | } catch { | |
| 490 | + | return url; | |
| 491 | + | } | |
| 492 | + | } | |
| 493 | + | ||
| 484 | 494 | function ConnectionRow({ | |
| 485 | 495 | connection, | |
| 486 | 496 | owner, | |
| 500 | 510 | config.repo && `issues in ${config.repo}`, | |
| 501 | 511 | config.assign && "agents start at once", | |
| 502 | 512 | config.organization, | |
| 503 | − | config.site?.replace(/^https:\/\//, ""), | |
| 504 | − | config.baseUrl?.replace(/^https:\/\//, ""), | |
| 513 | + | host(config.site), | |
| 514 | + | host(config.baseUrl), | |
| 505 | 515 | config.model && `default ${config.model}`, | |
| 506 | 516 | connection.models.length > 0 && `${connection.models.length} models`, | |
| 507 | 517 | config.keys?.length ? config.keys.join(", ") : null, | |
| 510 | 520 | const waitingForSecret = connection.provider === "sentry" && !deliveries.length && !connection.lastUsedAt; | |
| 511 | 521 | return ( | |
| 512 | 522 | <div> | |
| 513 | − | <div className="flex flex-wrap items-center gap-3"> | |
| 523 | + | <div className="flex items-center gap-3"> | |
| 514 | 524 | <ProviderMark provider={connection.provider} /> | |
| 515 | 525 | <div className="min-w-0 grow"> | |
| 516 | 526 | <p className="truncate text-sm font-medium">{connection.name}</p> | |
| 517 | 527 | <p className="truncate text-xs text-muted">{facts.join(" · ")}</p> | |
| 518 | 528 | </div> | |
| 519 | 529 | {connection.lastUsedAt && ( | |
| 520 | − | <span className="text-xs text-faint"> | |
| 530 | + | <span className="hidden shrink-0 text-xs text-faint sm:inline"> | |
| 521 | 531 | Used <TimeAgo at={connection.lastUsedAt} /> | |
| 522 | 532 | </span> | |
| 523 | 533 | )} | |
| 524 | 534 | {owner && ( | |
| 525 | − | <Form method="post" className="flex gap-2"> | |
| 535 | + | <Form method="post" className="flex shrink-0 gap-2"> | |
| 526 | 536 | <input type="hidden" name="id" value={connection.id} /> | |
| 527 | 537 | <Button variant="quiet" type="submit" name="intent" value="test" disabled={busy}> | |
| 528 | 538 | Test | |
| 711 | 721 | <option value="x-api-key">x-api-key</option> | |
| 712 | 722 | </select> | |
| 713 | 723 | </Field> | |
| 724 | + | <Field label="Cloudflare AI Gateway token" hint="Only for an authenticated AI Gateway: sent as cf-aig-authorization."> | |
| 725 | + | <Input name="signingSecret" type="password" /> | |
| 726 | + | </Field> | |
| 714 | 727 | <Field label="Default model" hint="The model to use. g1t also lists the endpoint's models if it offers a list."> | |
| 715 | 728 | <Input name="model" placeholder="anthropic/claude-sonnet-4.5" /> | |
| 716 | 729 | </Field> | |
| 730 | 743 | <option value="authorization">Authorization: Bearer</option> | |
| 731 | 744 | </select> | |
| 732 | 745 | </Field> | |
| 746 | + | <Field label="Cloudflare AI Gateway token" hint="Only for an authenticated AI Gateway: sent as cf-aig-authorization."> | |
| 747 | + | <Input name="signingSecret" type="password" /> | |
| 748 | + | </Field> | |
| 733 | 749 | <Field label="Model" hint="Optional. Leave empty to use g1t's choice for each kind of work; set it if your endpoint names models its own way."> | |
| 734 | 750 | <Input name="model" placeholder="claude-sonnet-5-5" /> | |
| 735 | 751 | </Field> |
| 189 | 189 | `get_pull_request_changes`, `review_pull_request`, `merge_pull_request`, | |
| 190 | 190 | `get_merge_queue`, `message_agent`, `answer_message`, `take_messages`, | |
| 191 | 191 | `list_integrations`, `connect_integration`, `test_integration`, | |
| 192 | − | `disconnect_integration`, `get_context`, `import_issue`, | |
| 192 | + | `disconnect_integration`, `get_model_routes`, `set_model_routes`, | |
| 193 | + | `get_context`, `import_issue`, | |
| 193 | 194 | `list_repos`, `get_repo`, `create_repo`, `update_repo`, | |
| 194 | 195 | `get_repo_settings`, `update_repo_settings`, `list_events`, | |
| 195 | 196 | `create_workspace`, and `whoami`. MCP tools take the repository as `repo`, | |
| 200 | 201 | A workspace's owners connect it to outside systems on its **Integrations** | |
| 201 | 202 | page, or with `POST /workspaces/{workspace}/integrations`: | |
| 202 | 203 | ||
| 203 | − | - **Its own model provider** (`anthropic`, or `anthropic_endpoint` for any | |
| 204 | − | Anthropic-compatible URL): agents' model costs are billed there, and g1t | |
| 205 | − | charges $0.10 a run. Sandboxes never hold the key. | |
| 204 | + | - **Its own model providers** (`anthropic`, `openai`, `gemini`, | |
| 205 | + | `anthropic_endpoint`, `openai_endpoint`), as many as it uses, with each | |
| 206 | + | kind of work routed to one of them or to g1t's hosted models | |
| 207 | + | (`PUT /workspaces/{workspace}/model-routes`). Those providers bill the | |
| 208 | + | workspace and g1t charges $0.10 a run. Sandboxes never hold a key. | |
| 206 | 209 | - **Alerts** (`sentry`, `datadog`, `webhook`): each problem opens one issue | |
| 207 | 210 | in a chosen repository, optionally with an agent put on it at once. | |
| 208 | 211 | Senders sign requests to `https://api.g1t.sh/hooks/{integration}`. | |
| 247 | 250 | - [Accounts and sign-in](https://docs.g1t.sh/guides/authentication/) | |
| 248 | 251 | - [Workspaces and tokens](https://docs.g1t.sh/guides/workspaces/) | |
| 249 | 252 | - [Integrations](https://docs.g1t.sh/guides/integrations/) | |
| 250 | − | - [Your own model provider](https://docs.g1t.sh/guides/models/) | |
| 253 | + | - [Model providers](https://docs.g1t.sh/guides/models/) | |
| 251 | 254 | - [Usage and billing](https://docs.g1t.sh/guides/usage-and-billing/) | |
| 252 | 255 | - [Git](https://docs.g1t.sh/guides/git/) | |
| 253 | 256 | - [MCP tools](https://docs.g1t.sh/reference/mcp/) |
| 334 | 334 | pub api_key: Option<String>, | |
| 335 | 335 | /// `x-api-key` or `authorization`. | |
| 336 | 336 | pub auth_header: Option<String>, | |
| 337 | + | /// For an endpoint behind an authenticated Cloudflare AI Gateway: the | |
| 338 | + | /// gateway's own token, sent as `cf-aig-authorization`. | |
| 339 | + | #[serde(default)] | |
| 340 | + | pub gateway_token: Option<String>, | |
| 337 | 341 | } | |
| 338 | 342 | ||
| 339 | 343 | // --- Methods ----------------------------------------------------------------- | |
| 360 | 364 | #[serde(default)] | |
| 361 | 365 | pub secret: Option<String>, | |
| 362 | 366 | /// What it signs its requests to g1t with: Sentry's client secret. | |
| 363 | − | /// Made by g1t for Datadog and webhooks, and shown once. | |
| 367 | + | /// Made by g1t for Datadog and webhooks, and shown once. For a model | |
| 368 | + | /// endpoint behind an authenticated Cloudflare AI Gateway, the gateway's | |
| 369 | + | /// token. | |
| 364 | 370 | #[serde(default)] | |
| 365 | 371 | pub signing_secret: Option<String>, | |
| 366 | 372 | } |
| 130 | 130 | baseUrl: string | null; | |
| 131 | 131 | apiKey: string | null; | |
| 132 | 132 | authHeader: string | null; | |
| 133 | + | /** For an endpoint behind an authenticated Cloudflare AI Gateway: its token. */ | |
| 134 | + | gatewayToken?: string | null; | |
| 133 | 135 | }; | |
| 134 | 136 | ||
| 135 | 137 | export type ConnectInput = { |
| 406 | 406 | .await?; | |
| 407 | 407 | // A model provider is checked at once, which also learns its models. | |
| 408 | 408 | if provider.kind() == ProviderKind::Models { | |
| 409 | − | let checked = models::test(provider, &config, secrets.secret.as_deref()).await?; | |
| 409 | + | let checked = models::test(provider, &config, secrets.secret.as_deref(), secrets.signing_secret.as_deref()).await?; | |
| 410 | 410 | self.after_check(&id, &checked).await?; | |
| 411 | 411 | } | |
| 412 | 412 | let Some(row) = self.row(&id).await? else { | |
| 495 | 495 | let secrets = self.secrets(&row); | |
| 496 | 496 | let key = secrets.secret.as_deref(); | |
| 497 | 497 | if provider.kind() == ProviderKind::Models { | |
| 498 | − | let checked = models::test(provider, &config, key).await?; | |
| 498 | + | let checked = models::test(provider, &config, key, secrets.signing_secret.as_deref()).await?; | |
| 499 | 499 | self.after_check(&row.id, &checked).await?; | |
| 500 | 500 | return Ok(Outcome::Ok(match checked { | |
| 501 | 501 | Ok((message, _)) => Tested { ok: true, message }, | |
| 1251 | 1251 | base_url: None, | |
| 1252 | 1252 | api_key: None, | |
| 1253 | 1253 | auth_header: None, | |
| 1254 | + | gateway_token: None, | |
| 1254 | 1255 | }; | |
| 1255 | 1256 | let Some(connection_id) = session.connection_id else { | |
| 1256 | 1257 | return Ok(Some(base)); | |
| 1269 | 1270 | base_url: Some(models::base_url(provider, &config)), | |
| 1270 | 1271 | api_key: self.secrets(&row).secret, | |
| 1271 | 1272 | auth_header: Some(models::auth_header(provider, &config)), | |
| 1273 | + | gateway_token: matches!(provider, Provider::AnthropicEndpoint | Provider::OpenaiEndpoint) | |
| 1274 | + | .then(|| self.secrets(&row).signing_secret) | |
| 1275 | + | .flatten(), | |
| 1272 | 1276 | ..base | |
| 1273 | 1277 | })) | |
| 1274 | 1278 | } |
| 40 | 40 | ||
| 41 | 41 | /// Asks the provider for its models with the key. `Ok` with what to say and | |
| 42 | 42 | /// the models it offers; `Err` with what went wrong. | |
| 43 | − | pub async fn test(provider: Provider, config: &ConnectionConfig, key: Option<&str>) -> Result<std::result::Result<(String, Vec<String>), String>> { | |
| 43 | + | pub async fn test( | |
| 44 | + | provider: Provider, | |
| 45 | + | config: &ConnectionConfig, | |
| 46 | + | key: Option<&str>, | |
| 47 | + | gateway_token: Option<&str>, | |
| 48 | + | ) -> Result<std::result::Result<(String, Vec<String>), String>> { | |
| 44 | 49 | let base = base_url(provider, config); | |
| 45 | 50 | let url = match provider.api() { | |
| 46 | 51 | "anthropic" => format!("{base}/v1/models?limit=100"), | |
| 56 | 61 | headers.push(("x-api-key", key)); | |
| 57 | 62 | } | |
| 58 | 63 | } | |
| 64 | + | let gateway = gateway_token.map(|token| format!("Bearer {token}")); | |
| 65 | + | if let Some(gateway) = gateway.as_deref() { | |
| 66 | + | headers.push(("cf-aig-authorization", gateway)); | |
| 67 | + | } | |
| 59 | 68 | let answer = http::send(Method::Get, &url, &headers, None).await?; | |
| 60 | 69 | let system = provider.label(); | |
| 61 | 70 | if answer.ok() { |
| 53 | 53 | if (!path.startsWith("/v1/messages")) return refuse(404, `${path} has no counterpart at this provider.`); | |
| 54 | 54 | ||
| 55 | 55 | const headers = new Headers({ "content-type": "application/json" }); | |
| 56 | + | if (upstream.gatewayToken) headers.set("cf-aig-authorization", `Bearer ${upstream.gatewayToken}`); | |
| 56 | 57 | if (upstream.apiKey) { | |
| 57 | 58 | if (upstream.authHeader === "x-api-key") headers.set("x-api-key", upstream.apiKey); | |
| 58 | 59 | else headers.set("authorization", `Bearer ${upstream.apiKey}`); |
| 69 | 69 | if (upstream.authHeader === "authorization") headers.set("authorization", `Bearer ${key}`); | |
| 70 | 70 | else headers.set("x-api-key", key); | |
| 71 | 71 | } | |
| 72 | + | if (upstream.gatewayToken) headers.set("cf-aig-authorization", `Bearer ${upstream.gatewayToken}`); | |
| 72 | 73 | const base = (upstream.baseUrl ?? "https://api.anthropic.com").replace(/\/+$/, ""); | |
| 73 | 74 | return { url: `${base}${path}`, headers }; | |
| 74 | 75 | } |