Skip to content
Sign up

flagon-io/g1t

public
0

Commit

Projects have Observability: their requests, logs, traces, metrics and errors, from apps on g1t.page with nothing to set up, and from software anywhere else over OpenTelemetry.

A new service, services/telemetry (g1t-telemetry), keeps each project's telemetry in a Durable Object with its own SQLite: requests, logs (full-text indexed), spans, metric points, errors grouped by fingerprint, per-minute rollups with latency and CPU histograms (kept 90 days), ingest tokens and exports. Two doors fill it. Every app on g1t.page and the dispatcher in front of them name it as their Tail Worker: the dispatcher gives each request an id and a traceparent and writes one line naming the request and the app's project, the app's own event brings its logs, exceptions and CPU time, and the store joins the halves by id, so even a request a site's files answered is counted. And https://otel.g1t.sh/v1/{traces,logs,metrics} takes OTLP over HTTP in JSON or protobuf (a decoder of its own, no dependency), gzipped or not, with a project's g1to_ ingest token; api.g1t.sh/otlp is the same door, and the one self-hosted installations use. Apps on g1t.page are deployed with OTEL_EXPORTER_OTLP_* set to the project's deployments token, so an OpenTelemetry SDK in them sends spans that sit under the request's trace. Cloudflare's own OpenTelemetry export is taken at /cloudflare/v1/* once CLOUDFLARE_EXPORT_KEY is set. Exports send everything on to any OTLP/HTTP endpoint. The project's Observability page replaces its roadmap entry: Overview (numbers against the period before, requests and latency charts, numbers by deployment newest first, routes, open errors, services, regions), Requests, a request's page, Logs with a live tail, Traces and a waterfall, Errors (resolve, ignore, or make an issue with g1t assigned), Metrics, and Settings. It is for people with a role on the project, public or not. The API has every read and change under /workspaces/:workspace/projects/:project/observability, with the observability:read and observability:write scopes, and MCP has an observability tool. The guide is docs.g1t.sh/guides/observability/. Web analytics moves to Insights on the roadmap.

syntaqxcommitted Parenteb249dbBrowse files
84 files+2066−868 files not counted0/84 viewed
+50−1
1515 mod checks;
1616 mod customize;
1717 mod deployments;
18+mod telemetry;
1819 mod deploy_keys;
1920 mod limits;
2021 mod logs;
879880 }
880881 // A deleted comment has nothing to say, as GitHub's says nothing.
881882 Outcome::Ok(_) if route.no_content() => Ok(Response::empty()?.with_status(204)),
882− Outcome::Ok(value) => reply(&value),
883+ Outcome::Ok(value) => Response::from_json(&wire::snake_case_keeping(value, route.op.as_given())),
883884 // A token without the scope a call needs is told which one.
884885 Outcome::Fail(refused) => match (refused.code, audit::missing_scope(route.op, &viewer, &input)) {
885886 (FailureCode::Forbidden, Some(scope)) => Ok(reply(&json!({
976977
977978 // The API is called from browsers too: the reference's explorer, and apps
978979 // built on g1t. It carries no cookies, so any origin may call it.
980+/// The signal an OTLP path sends (`/otlp/v1/traces`, `logs` or `metrics`),
981+/// if it is one.
982+fn otlp_signal(path: &str) -> Option<&'static str> {
983+ let path = path.strip_prefix("/v1").filter(|rest| rest.starts_with('/')).unwrap_or(path);
984+ match path.trim_end_matches('/') {
985+ "/otlp/v1/traces" => Some("traces"),
986+ "/otlp/v1/logs" => Some("logs"),
987+ "/otlp/v1/metrics" => Some("metrics"),
988+ _ => None,
989+ }
990+}
991+
992+/// OpenTelemetry sent to the API's own address: a self-hosted install's
993+/// only door, and a second one hosted. It goes to the telemetry service as
994+/// it came, which checks the project's ingest token itself (not a g1t
995+/// token), and its answer comes back as it is, CORS headers included.
996+async fn forward_otlp(mut request: Request, env: &Env, signal: &str) -> Result<Response> {
997+ let method = request.method();
998+ let headers = worker::Headers::new();
999+ for name in ["authorization", "x-g1t-token", "content-type", "content-encoding"] {
1000+ if let Some(value) = request.headers().get(name)? {
1001+ headers.set(name, &value)?;
1002+ }
1003+ }
1004+ let mut init = worker::RequestInit::new();
1005+ init.with_method(method.clone()).with_headers(headers);
1006+ if method == Method::Post {
1007+ let body = request.bytes().await?;
1008+ init.with_body(Some(worker::js_sys::Uint8Array::from(body.as_slice()).into()));
1009+ }
1010+ let target = format!("https://telemetry/v1/{signal}");
1011+ let mut answer = env.service("TELEMETRY")?.fetch_request(Request::new_with_init(&target, &init)?).await?;
1012+ // A fresh response, with the service's headers: a fetched one's cannot
1013+ // be changed, and nothing should be added to them.
1014+ let status = answer.status_code();
1015+ let headers = worker::Headers::new();
1016+ for (name, value) in answer.headers().entries() {
1017+ headers.set(&name, &value)?;
1018+ }
1019+ let bytes = answer.bytes().await?;
1020+ let response = if bytes.is_empty() { Response::empty()? } else { Response::from_bytes(bytes)? };
1021+ Ok(response.with_status(status).with_headers(headers))
1022+}
1023+
9791024 #[event(fetch)]
9801025 async fn fetch(request: Request, env: Env, _ctx: Context) -> Result<Response> {
1026+ // OTLP passes through untouched, before any g1t token is read.
1027+ if let Some(signal) = otlp_signal(&request.path()) {
1028+ return forward_otlp(request, &env, signal).await;
1029+ }
9811030 let mut response = respond(request, &env).await?;
9821031 let headers = response.headers_mut();
9831032 headers.set("access-control-allow-origin", "*")?;
+4−3
8484 }
8585 },
8686 // A tool per operation, as the server had before its
87− // resource tools. Still answered, no longer listed.
88− None => match Op::by_name(name) {
87+ // resource tools. Still answered, no longer listed; never
88+ // one offered over REST alone.
89+ None => match Op::by_name(name).filter(|op| !crate::tools::REST_ONLY.contains(op)) {
8990 Some(op) => op,
9091 None => return Ok(Some(error(id, -32602, "Unknown tool."))),
9192 },
9798 // In `snake_case`, as the REST API answers; the protocol's
9899 // own envelope keeps MCP's spelling.
99100 Outcome::Ok(value) => (
100− serde_json::to_string_pretty(&g1t_kit::wire::snake_case(value))?,
101+ serde_json::to_string_pretty(&g1t_kit::wire::snake_case_keeping(value, op.as_given()))?,
101102 false,
102103 ),
103104 Outcome::Fail(failure) => (failure.message, true),
+28−0
1414 use crate::deploy_keys::DeployKeysOp;
1515 use crate::mirrors::MirrorsOp;
1616 use crate::deployments::DeploymentsOp;
17+use crate::telemetry::TelemetryOp;
1718 use crate::packages::PackagesOp;
1819 use crate::folios::FoliosOp;
1920 use crate::protection::ProtectionOp;
484485 ],
485486 ),
486487 (
488+ "Observability",
489+ "What a project's running software did: its requests, logs, traces, grouped errors and metrics, from its apps on g1t.page with nothing to set up and from OpenTelemetry (OTLP over HTTP) sent from anywhere with an ingest token. Reading needs a role of your own on the project's repository; resolving errors, Triage; settings, ingest tokens, exports and usage, Admin.",
490+ &[
491+ Op::Telemetry(TelemetryOp::GetOverview),
492+ Op::Telemetry(TelemetryOp::ListRequests),
493+ Op::Telemetry(TelemetryOp::GetRequest),
494+ Op::Telemetry(TelemetryOp::ListLogs),
495+ Op::Telemetry(TelemetryOp::ListTraces),
496+ Op::Telemetry(TelemetryOp::GetTrace),
497+ Op::Telemetry(TelemetryOp::ListErrors),
498+ Op::Telemetry(TelemetryOp::UpdateError),
499+ Op::Telemetry(TelemetryOp::ListMetrics),
500+ Op::Telemetry(TelemetryOp::GetMetric),
501+ Op::Telemetry(TelemetryOp::GetSettings),
502+ Op::Telemetry(TelemetryOp::UpdateSettings),
503+ Op::Telemetry(TelemetryOp::ListTokens),
504+ Op::Telemetry(TelemetryOp::CreateToken),
505+ Op::Telemetry(TelemetryOp::DeleteToken),
506+ Op::Telemetry(TelemetryOp::ListExports),
507+ Op::Telemetry(TelemetryOp::CreateExport),
508+ Op::Telemetry(TelemetryOp::UpdateExport),
509+ Op::Telemetry(TelemetryOp::DeleteExport),
510+ Op::Telemetry(TelemetryOp::GetUsage),
511+ ],
512+ ),
513+ (
487514 "Run protection",
488515 "What keeps workflow runs safe: environments' protection rules (required reviewers, a wait timer, which branches may deploy) and the reviews of the jobs they hold, approving a pull request's run from outside, what a job's token gets when its workflow writes no `permissions:`, and repository_dispatch, which a job's own token may send.",
489516 &[
840867 Op::Checks(op) => op.title(),
841868 Op::About(op) => op.title(),
842869 Op::Deployments(op) => op.title(),
870+ Op::Telemetry(op) => op.title(),
843871 Op::Protection(op) => op.title(),
844872 Op::Tokens(op) => op.title(),
845873 Op::Artifacts(op) => op.title(),
+48−1
3434 use crate::deploy_keys::DeployKeysOp;
3535 use crate::mirrors::MirrorsOp;
3636 use crate::deployments::DeploymentsOp;
37+use crate::telemetry::TelemetryOp;
3738 use crate::packages::PackagesOp;
3839 use crate::folios::FoliosOp;
3940 use crate::protection::ProtectionOp;
6970 pub projects: Fetcher,
7071 /// Deployments wherever they run, and environments.
7172 pub deployments: Fetcher,
73+ /// Observability: each project's requests, logs, traces, errors and
74+ /// metrics (services/telemetry). It checks no access; telemetry.rs does.
75+ pub telemetry: Fetcher,
7276 /// Packages: their settings, versions, deleting and restoring them.
7377 pub packages: Fetcher,
7478 /// The artifacts service (services/artifacts): docs, slides, designs and
101105 security: env.service("SECURITY")?,
102106 projects: env.service("PROJECTS")?,
103107 deployments: env.service("DEPLOYMENTS")?,
108+ telemetry: env.service("TELEMETRY")?,
104109 packages: env.service("PACKAGES")?,
105110 artifacts: env.service("ARTIFACTS")?,
106111 agents: env.service("AGENTS")?,
323328 About(AboutOp),
324329 /// Deployments wherever they run, and environments: deployments.rs.
325330 Deployments(DeploymentsOp),
331+ /// A project's observability: requests, logs, traces, errors, metrics,
332+ /// and how they are collected: telemetry.rs.
333+ Telemetry(TelemetryOp),
326334 /// Environments' protection rules, approving runs, the token's default
327335 /// permissions and repository dispatch: protection.rs.
328336 Protection(ProtectionOp),
716724 }
717725
718726 impl Op {
719− pub const ALL: [Op; 388] = [
727+ pub const ALL: [Op; 408] = [
720728 Op::Whoami,
721729 Op::GetWorkspace,
722730 Op::CreateWorkspace,
10031011 Op::Deployments(DeploymentsOp::ListEnvironments),
10041012 Op::Deployments(DeploymentsOp::GetEnvironment),
10051013 Op::Deployments(DeploymentsOp::StartDeploymentBuild),
1014+ Op::Telemetry(TelemetryOp::GetOverview),
1015+ Op::Telemetry(TelemetryOp::ListRequests),
1016+ Op::Telemetry(TelemetryOp::GetRequest),
1017+ Op::Telemetry(TelemetryOp::ListLogs),
1018+ Op::Telemetry(TelemetryOp::ListTraces),
1019+ Op::Telemetry(TelemetryOp::GetTrace),
1020+ Op::Telemetry(TelemetryOp::ListErrors),
1021+ Op::Telemetry(TelemetryOp::UpdateError),
1022+ Op::Telemetry(TelemetryOp::ListMetrics),
1023+ Op::Telemetry(TelemetryOp::GetMetric),
1024+ Op::Telemetry(TelemetryOp::GetSettings),
1025+ Op::Telemetry(TelemetryOp::UpdateSettings),
1026+ Op::Telemetry(TelemetryOp::ListTokens),
1027+ Op::Telemetry(TelemetryOp::CreateToken),
1028+ Op::Telemetry(TelemetryOp::DeleteToken),
1029+ Op::Telemetry(TelemetryOp::ListExports),
1030+ Op::Telemetry(TelemetryOp::CreateExport),
1031+ Op::Telemetry(TelemetryOp::UpdateExport),
1032+ Op::Telemetry(TelemetryOp::DeleteExport),
1033+ Op::Telemetry(TelemetryOp::GetUsage),
10061034 Op::Artifacts(ArtifactsOp::ListArtifacts),
10071035 Op::Artifacts(ArtifactsOp::ListRunArtifacts),
10081036 Op::Artifacts(ArtifactsOp::GetArtifact),
13181346 Op::Checks(op) => op.name(),
13191347 Op::About(op) => op.name(),
13201348 Op::Deployments(op) => op.name(),
1349+ Op::Telemetry(op) => op.name(),
13211350 Op::Protection(op) => op.name(),
13221351 Op::Tokens(op) => op.name(),
13231352 Op::Artifacts(op) => op.name(),
18981927 Op::Checks(op) => op.description(),
18991928 Op::About(op) => op.description(),
19001929 Op::Deployments(op) => op.description(),
1930+ Op::Telemetry(op) => op.description(),
19011931 Op::Protection(op) => op.description(),
19021932 Op::Tokens(op) => op.description(),
19031933 Op::Artifacts(op) => op.description(),
34213451 Op::Checks(op) => op.input(),
34223452 Op::About(op) => op.input(),
34233453 Op::Deployments(op) => op.input(),
3454+ Op::Telemetry(op) => op.input(),
34243455 Op::Protection(op) => op.input(),
34253456 Op::Tokens(op) => op.input(),
34263457 Op::Artifacts(op) => op.input(),
34333464 }
34343465 }
34353466
3467+ /// Fields of its answer that are keyed by data and so are sent as they
3468+ /// are, beside those every answer keeps (`g1t_kit::wire::USER_KEYED`).
3469+ pub(crate) fn as_given(self) -> &'static [&'static str] {
3470+ match self {
3471+ Op::Telemetry(_) => crate::telemetry::AS_GIVEN,
3472+ _ => &[],
3473+ }
3474+ }
3475+
34363476 /// Whether REST answers with a redirect to the `url` it returns, as
34373477 /// GitHub's downloads do.
34383478 pub(crate) fn redirects(self) -> bool {
35223562 if let Op::Customize(_) = self {
35233563 return false;
35243564 }
3565+ // A project's observability names the workspace and the project.
3566+ if let Op::Telemetry(_) = self {
3567+ return false;
3568+ }
35253569 if let Op::About(op) = self {
35263570 return op.needs_repo();
35273571 }
58205864 Op::Checks(op) => crate::checks::run(op, services, viewer, input).await,
58215865 Op::About(op) => crate::about::run(op, services, viewer, input).await,
58225866 Op::Deployments(op) => crate::deployments::run(op, services, viewer, input).await,
5867+ // A project's observability: telemetry.rs checks who may see
5868+ // and change it, as the site does.
5869+ Op::Telemetry(op) => crate::telemetry::run(op, services, viewer, input).await,
58235870 Op::Protection(op) => crate::protection::run(op, services, viewer, input).await,
58245871 Op::Tokens(op) => crate::token_policy::run(op, services, viewer, input).await,
58255872 Op::Artifacts(op) => crate::run_artifacts::run(op, services, viewer, input).await,
+658−0
1492814928 "deleted": true
1492914929 },
1493014930 "notes": "Only an artifact in the trash can be deleted for good: `trash_workspace_artifact` first."
14931+ },
14932+ "get_observability_overview": {
14933+ "params": {
14934+ "workspace": "flagon-io",
14935+ "project": "g1t"
14936+ },
14937+ "query": {
14938+ "from": "2026-10-12T13:00:00Z",
14939+ "to": "2026-10-12T14:00:00Z",
14940+ "environment": "production"
14941+ },
14942+ "response": {
14943+ "from_us": 1791810000000000,
14944+ "to_us": 1791813600000000,
14945+ "step_s": 60,
14946+ "totals": {
14947+ "requests": 18204,
14948+ "errors": 41,
14949+ "error_rate": 0.0023,
14950+ "latency": {
14951+ "p50": 38.2,
14952+ "p75": 71.5,
14953+ "p95": 212.0,
14954+ "p99": 640.0
14955+ },
14956+ "cpu": {
14957+ "p50": 4.1,
14958+ "p75": 6.8,
14959+ "p95": 14.2,
14960+ "p99": 29.5
14961+ }
14962+ },
14963+ "previous": {
14964+ "requests": 17650,
14965+ "errors": 3,
14966+ "error_rate": 0.0002,
14967+ "latency": {
14968+ "p50": 35.9,
14969+ "p75": 66.0,
14970+ "p95": 188.0,
14971+ "p99": 402.0
14972+ },
14973+ "cpu": {
14974+ "p50": 3.9,
14975+ "p75": 6.5,
14976+ "p95": 13.0,
14977+ "p99": 22.1
14978+ }
14979+ },
14980+ "points": [
14981+ {
14982+ "time_us": 1791810000000000,
14983+ "requests": 302,
14984+ "errors": 0,
14985+ "status_2xx": 288,
14986+ "status_3xx": 9,
14987+ "status_4xx": 5,
14988+ "status_5xx": 0,
14989+ "latency": {
14990+ "p50": 36.0,
14991+ "p75": 68.0,
14992+ "p95": 190.0,
14993+ "p99": 410.0
14994+ },
14995+ "cpu": {
14996+ "p50": 4.0,
14997+ "p75": 6.6,
14998+ "p95": 13.1,
14999+ "p99": 22.0
15000+ }
15001+ }
15002+ ],
15003+ "routes": [
15004+ {
15005+ "route": "/:workspace/:repo/pulls/:number",
15006+ "method": "GET",
15007+ "requests": 2210,
15008+ "errors": 37,
15009+ "latency": {
15010+ "p50": 61.0,
15011+ "p75": 120.0,
15012+ "p95": 480.0,
15013+ "p99": 842.0
15014+ }
15015+ }
15016+ ],
15017+ "deployments": [
15018+ {
15019+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15020+ "environment": "production",
15021+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15022+ "first_us": 1791813041000000,
15023+ "requests": 2904,
15024+ "errors": 39,
15025+ "latency": {
15026+ "p50": 40.1,
15027+ "p75": 77.0,
15028+ "p95": 260.0,
15029+ "p99": 720.0
15030+ }
15031+ }
15032+ ],
15033+ "services": [
15034+ {
15035+ "service": "g1t-web",
15036+ "source": "cloudflare",
15037+ "requests": 18204,
15038+ "errors": 41,
15039+ "last_us": 1791813599000000
15040+ }
15041+ ],
15042+ "regions": [
15043+ {
15044+ "region": "IAD",
15045+ "requests": 9120
15046+ },
15047+ {
15048+ "region": "AMS",
15049+ "requests": 4410
15050+ }
15051+ ]
15052+ },
15053+ "notes": "`totals` is the period and `previous` the period of the same length before it, to compare. Latency and CPU are in milliseconds. `deployments` lists each deployment that answered in the period, newest first: when the error rate or latency rose, the deployment it began with is usually the first one with the higher numbers. Points are one per `step_s` seconds, chosen from the period unless `step` says. See [Observability](/guides/observability/)."
15054+ },
15055+ "list_requests": {
15056+ "params": {
15057+ "workspace": "flagon-io",
15058+ "project": "g1t"
15059+ },
15060+ "query": {
15061+ "errors_only": "true",
15062+ "limit": "1"
15063+ },
15064+ "response": {
15065+ "items": [
15066+ {
15067+ "id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15068+ "time_us": 1791813807418000,
15069+ "duration_us": 842000,
15070+ "trigger": "fetch",
15071+ "method": "GET",
15072+ "url": "g1t.sh/flagon-io/g1t/pulls/412",
15073+ "route": "/:workspace/:repo/pulls/:number",
15074+ "status": 500,
15075+ "outcome": "exception",
15076+ "cpu_ms": 31.4,
15077+ "region": "IAD",
15078+ "country": "US",
15079+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15080+ "log_count": 2,
15081+ "error": {
15082+ "name": "TypeError",
15083+ "message": "Cannot read properties of undefined (reading 'head')"
15084+ },
15085+ "service": "g1t-web",
15086+ "environment": "production",
15087+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15088+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15089+ "source": "cloudflare"
15090+ }
15091+ ],
15092+ "next": "eyJ0IjoxNzkxODEzODA3NDE4MDAwfQ"
15093+ },
15094+ "notes": "Apps on g1t.page send their requests with nothing to set up; software hosted elsewhere sends them as OpenTelemetry server spans. `route` is the path with ids taken out, which the overview groups by. Pass `next` back as `cursor` for the next page; it is null on the last."
15095+ },
15096+ "get_request": {
15097+ "params": {
15098+ "workspace": "flagon-io",
15099+ "project": "g1t",
15100+ "id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x"
15101+ },
15102+ "response": {
15103+ "request": {
15104+ "id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15105+ "time_us": 1791813807418000,
15106+ "duration_us": 842000,
15107+ "trigger": "fetch",
15108+ "method": "GET",
15109+ "url": "g1t.sh/flagon-io/g1t/pulls/412",
15110+ "route": "/:workspace/:repo/pulls/:number",
15111+ "status": 500,
15112+ "outcome": "exception",
15113+ "cpu_ms": 31.4,
15114+ "region": "IAD",
15115+ "country": "US",
15116+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15117+ "log_count": 2,
15118+ "error": {
15119+ "name": "TypeError",
15120+ "message": "Cannot read properties of undefined (reading 'head')"
15121+ },
15122+ "service": "g1t-web",
15123+ "environment": "production",
15124+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15125+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15126+ "source": "cloudflare"
15127+ },
15128+ "logs": [
15129+ {
15130+ "id": "log_01kq9b7d9f1h3k5m7p9r1t3v5y",
15131+ "time_us": 1791813808251000,
15132+ "severity": "error",
15133+ "severity_number": 17,
15134+ "body": "loading pull request 412 failed: Cannot read properties of undefined (reading 'head')",
15135+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15136+ "span_id": "00f067aa0ba902b7",
15137+ "request_id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15138+ "attributes": {
15139+ "code.function": "loadPull",
15140+ "http.route": "/:workspace/:repo/pulls/:number"
15141+ },
15142+ "service": "g1t-web",
15143+ "environment": "production",
15144+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15145+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15146+ "source": "cloudflare"
15147+ }
15148+ ]
15149+ },
15150+ "notes": "`404` when the project has no request with that id, or it is older than the project's retention."
15151+ },
15152+ "list_logs": {
15153+ "params": {
15154+ "workspace": "flagon-io",
15155+ "project": "g1t"
15156+ },
15157+ "query": {
15158+ "q": "\"failed\"",
15159+ "level": "error,fatal",
15160+ "limit": "1"
15161+ },
15162+ "response": {
15163+ "items": [
15164+ {
15165+ "id": "log_01kq9b7d9f1h3k5m7p9r1t3v5y",
15166+ "time_us": 1791813808251000,
15167+ "severity": "error",
15168+ "severity_number": 17,
15169+ "body": "loading pull request 412 failed: Cannot read properties of undefined (reading 'head')",
15170+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15171+ "span_id": "00f067aa0ba902b7",
15172+ "request_id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15173+ "attributes": {
15174+ "code.function": "loadPull",
15175+ "http.route": "/:workspace/:repo/pulls/:number"
15176+ },
15177+ "service": "g1t-web",
15178+ "environment": "production",
15179+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15180+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15181+ "source": "cloudflare"
15182+ }
15183+ ],
15184+ "next": null
15185+ },
15186+ "notes": "`q` finds lines holding all of its words, or a phrase in quotes. `level` takes one or more of `trace`, `debug`, `info`, `warn`, `error` and `fatal`. Follow `trace_id` to [the trace](/reference/api/observability/get-trace/) or `request_id` to [the request](/reference/api/observability/get-request/)."
15187+ },
15188+ "list_traces": {
15189+ "params": {
15190+ "workspace": "flagon-io",
15191+ "project": "g1t"
15192+ },
15193+ "query": {
15194+ "errors_only": "true",
15195+ "min_duration_ms": "500"
15196+ },
15197+ "response": {
15198+ "items": [
15199+ {
15200+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15201+ "name": "GET /:workspace/:repo/pulls/:number",
15202+ "start_us": 1791813807418000,
15203+ "duration_us": 842000,
15204+ "span_count": 6,
15205+ "error_count": 1,
15206+ "services": [
15207+ "g1t-web",
15208+ "g1t-work"
15209+ ],
15210+ "status": 500,
15211+ "service": "g1t-web",
15212+ "environment": "production",
15213+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15214+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15215+ "source": "otlp"
15216+ }
15217+ ],
15218+ "next": null
15219+ },
15220+ "notes": "Each trace is its root span (or its earliest, when the root has not arrived), with what the rest add up to."
15221+ },
15222+ "get_trace": {
15223+ "params": {
15224+ "workspace": "flagon-io",
15225+ "project": "g1t",
15226+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736"
15227+ },
15228+ "response": {
15229+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15230+ "spans": [
15231+ {
15232+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15233+ "span_id": "00f067aa0ba902b7",
15234+ "parent_span_id": null,
15235+ "name": "GET /:workspace/:repo/pulls/:number",
15236+ "kind": "server",
15237+ "start_us": 1791813807418000,
15238+ "duration_us": 842000,
15239+ "status": "error",
15240+ "status_message": "TypeError",
15241+ "attributes": {
15242+ "http.request.method": "GET",
15243+ "http.response.status_code": 500
15244+ },
15245+ "resource": {
15246+ "service.name": "g1t-web",
15247+ "deployment.environment.name": "production"
15248+ },
15249+ "events": [],
15250+ "links": [],
15251+ "service": "g1t-web",
15252+ "environment": "production",
15253+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15254+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15255+ "source": "otlp"
15256+ }
15257+ ],
15258+ "logs": [
15259+ {
15260+ "id": "log_01kq9b7d9f1h3k5m7p9r1t3v5y",
15261+ "time_us": 1791813808251000,
15262+ "severity": "error",
15263+ "severity_number": 17,
15264+ "body": "loading pull request 412 failed: Cannot read properties of undefined (reading 'head')",
15265+ "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15266+ "span_id": "00f067aa0ba902b7",
15267+ "request_id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15268+ "attributes": {
15269+ "code.function": "loadPull",
15270+ "http.route": "/:workspace/:repo/pulls/:number"
15271+ },
15272+ "service": "g1t-web",
15273+ "environment": "production",
15274+ "deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15275+ "commit_sha": "7c1e9a4b2d6f80135ac9e2b7d4f6a8c0e1b3d5f7",
15276+ "source": "cloudflare"
15277+ }
15278+ ]
15279+ },
15280+ "notes": "Spans are in the order they started. `404` when the project has no spans of that trace."
15281+ },
15282+ "list_errors": {
15283+ "params": {
15284+ "workspace": "flagon-io",
15285+ "project": "g1t"
15286+ },
15287+ "query": {
15288+ "state": "open"
15289+ },
15290+ "response": [
15291+ {
15292+ "fingerprint": "9f2c41d07ab3e815",
15293+ "name": "TypeError",
15294+ "message": "Cannot read properties of undefined (reading 'head')",
15295+ "stack": "TypeError: Cannot read properties of undefined (reading 'head')\n at loadPull (app/routes/pulls.ts:88:31)",
15296+ "service": "g1t-web",
15297+ "environment": "production",
15298+ "first_us": 1791813062000000,
15299+ "last_us": 1791813808000000,
15300+ "count": 37,
15301+ "first_deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15302+ "last_request_id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15303+ "last_trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15304+ "state": "open",
15305+ "issue_number": null
15306+ }
15307+ ],
15308+ "notes": "The same exception is one group: its name, its message with numbers and ids taken out, and the top frame of its stack. `first_deployment_id` is the deployment it was first seen on, what likely brought it in. At most 200 groups, most recently seen first."
15309+ },
15310+ "update_error": {
15311+ "params": {
15312+ "workspace": "flagon-io",
15313+ "project": "g1t",
15314+ "fingerprint": "9f2c41d07ab3e815"
15315+ },
15316+ "request": {
15317+ "state": "resolved",
15318+ "issue_number": 418
15319+ },
15320+ "response": {
15321+ "fingerprint": "9f2c41d07ab3e815",
15322+ "name": "TypeError",
15323+ "message": "Cannot read properties of undefined (reading 'head')",
15324+ "stack": "TypeError: Cannot read properties of undefined (reading 'head')\n at loadPull (app/routes/pulls.ts:88:31)",
15325+ "service": "g1t-web",
15326+ "environment": "production",
15327+ "first_us": 1791813062000000,
15328+ "last_us": 1791813808000000,
15329+ "count": 37,
15330+ "first_deployment_id": "dpl_01kq9a2c4e6g8j0m2p4r6t8v0x",
15331+ "last_request_id": "req_01kq9b7d9f1h3k5m7p9r1t3v5x",
15332+ "last_trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
15333+ "state": "resolved",
15334+ "issue_number": 418
15335+ },
15336+ "notes": "A resolved group that happens again opens again. `issue_number` links the issue made for it, so an agent can take it; null unlinks it. Needs the Triage role or higher."
15337+ },
15338+ "list_metrics": {
15339+ "params": {
15340+ "workspace": "flagon-io",
15341+ "project": "g1t"
15342+ },
15343+ "response": [
15344+ {
15345+ "name": "http.server.request.duration",
15346+ "kind": "histogram",
15347+ "unit": "s",
15348+ "description": "Duration of HTTP server requests.",
15349+ "services": [
15350+ "g1t-api"
15351+ ],
15352+ "last_us": 1791813800000000
15353+ },
15354+ {
15355+ "name": "queue.depth",
15356+ "kind": "gauge",
15357+ "unit": "{message}",
15358+ "description": "Messages waiting.",
15359+ "services": [
15360+ "g1t-work"
15361+ ],
15362+ "last_us": 1791813780000000
15363+ }
15364+ ]
15365+ },
15366+ "get_metric": {
15367+ "params": {
15368+ "workspace": "flagon-io",
15369+ "project": "g1t",
15370+ "metric": "http.server.request.duration"
15371+ },
15372+ "query": {
15373+ "aggregate": "p95",
15374+ "group_by": "http.route",
15375+ "step": "300"
15376+ },
15377+ "response": {
15378+ "name": "http.server.request.duration",
15379+ "unit": "s",
15380+ "step_s": 300,
15381+ "series": [
15382+ {
15383+ "group": "/repos/:owner/:name/pulls",
15384+ "points": [
15385+ {
15386+ "time_us": 1791813300000000,
15387+ "value": 0.212
15388+ },
15389+ {
15390+ "time_us": 1791813600000000,
15391+ "value": 0.236
15392+ }
15393+ ]
15394+ }
15395+ ]
15396+ },
15397+ "notes": "A metric name with characters outside letters, digits, `.`, `_` and `-` is URL-encoded in the path. Gauges average in each step unless `aggregate` says `min` or `max`; sums add, and a cumulative sum becomes its increase; histograms give their mean, or a percentile from their buckets."
15398+ },
15399+ "get_observability_settings": {
15400+ "params": {
15401+ "workspace": "flagon-io",
15402+ "project": "g1t"
15403+ },
15404+ "response": {
15405+ "retention_days": 7,
15406+ "deployments": true
15407+ },
15408+ "notes": "Requests, logs and spans are kept for `retention_days`; the overview's charts are kept longer, as per-minute counts. Needs the Admin role."
15409+ },
15410+ "update_observability_settings": {
15411+ "params": {
15412+ "workspace": "flagon-io",
15413+ "project": "g1t"
15414+ },
15415+ "request": {
15416+ "retention_days": 14
15417+ },
15418+ "response": {
15419+ "retention_days": 14,
15420+ "deployments": true
15421+ },
15422+ "notes": "`retention_days` is 1, 3, 7, 14 or 30. Turning `deployments` off stops the project's apps on g1t.page from sending and keeps its tokens and history. Needs the Admin role."
15423+ },
15424+ "list_ingest_tokens": {
15425+ "params": {
15426+ "workspace": "flagon-io",
15427+ "project": "g1t"
15428+ },
15429+ "response": [
15430+ {
15431+ "id": "a1b2c3d4",
15432+ "name": "Deployments on g1t.page",
15433+ "hint": "g1to_prj_…2c8e",
15434+ "created_by": null,
15435+ "created_us": 1790841600000000,
15436+ "last_used_us": 1791813809000000,
15437+ "managed": true
15438+ },
15439+ {
15440+ "id": "k3m9q2x7",
15441+ "name": "fly.io collector",
15442+ "hint": "g1to_prj_…9f3a",
15443+ "created_by": "syntaqx",
15444+ "created_us": 1791796364000000,
15445+ "last_used_us": 1791813790000000,
15446+ "managed": false
15447+ }
15448+ ],
15449+ "notes": "Secrets are never returned: `hint` tells tokens apart. The `managed` one is g1t's own, for the project's apps on g1t.page, and is replaced by g1t, not by people. Needs the Admin role."
15450+ },
15451+ "create_ingest_token": {
15452+ "params": {
15453+ "workspace": "flagon-io",
15454+ "project": "g1t"
15455+ },
15456+ "request": {
15457+ "name": "fly.io collector"
15458+ },
15459+ "response": {
15460+ "token": {
15461+ "id": "k3m9q2x7",
15462+ "name": "fly.io collector",
15463+ "hint": "g1to_prj_…9f3a",
15464+ "created_by": "syntaqx",
15465+ "created_us": 1791796364000000,
15466+ "last_used_us": null,
15467+ "managed": false
15468+ },
15469+ "secret": "g1to_prj_…"
15470+ },
15471+ "notes": "The secret is returned this once. Send it as `Authorization: Bearer <secret>` (or the `x-g1t-token` header) with OTLP over HTTP, in JSON or protobuf, to `https://otel.g1t.sh/v1/traces`, `/v1/logs` and `/v1/metrics`, or to `/otlp/v1/…` on the API's own address. Needs the Admin role."
15472+ },
15473+ "delete_ingest_token": {
15474+ "params": {
15475+ "workspace": "flagon-io",
15476+ "project": "g1t",
15477+ "id": "k3m9q2x7"
15478+ },
15479+ "response": true,
15480+ "notes": "What sends with it is refused from then on. `404` for the managed token, which people cannot delete. Needs the Admin role."
15481+ },
15482+ "list_telemetry_exports": {
15483+ "params": {
15484+ "workspace": "flagon-io",
15485+ "project": "g1t"
15486+ },
15487+ "response": [
15488+ {
15489+ "id": "x7d2k9m4",
15490+ "name": "Honeycomb",
15491+ "endpoint": "https://api.honeycomb.io",
15492+ "signals": [
15493+ "traces",
15494+ "logs"
15495+ ],
15496+ "header_names": [
15497+ "x-honeycomb-team"
15498+ ],
15499+ "enabled": true,
15500+ "last_error": null,
15501+ "last_sent_us": 1791813809000000
15502+ }
15503+ ],
15504+ "notes": "Header values are secrets: written, never read back, so only `header_names` are shown. `last_error` is what the endpoint answered the last time sending failed. Needs the Admin role."
15505+ },
15506+ "create_telemetry_export": {
15507+ "params": {
15508+ "workspace": "flagon-io",
15509+ "project": "g1t"
15510+ },
15511+ "request": {
15512+ "name": "Honeycomb",
15513+ "endpoint": "https://api.honeycomb.io",
15514+ "signals": [
15515+ "traces",
15516+ "logs"
15517+ ],
15518+ "headers": {
15519+ "x-honeycomb-team": "hcaik_…"
15520+ }
15521+ },
15522+ "response": {
15523+ "id": "x7d2k9m4",
15524+ "name": "Honeycomb",
15525+ "endpoint": "https://api.honeycomb.io",
15526+ "signals": [
15527+ "traces",
15528+ "logs"
15529+ ],
15530+ "header_names": [
15531+ "x-honeycomb-team"
15532+ ],
15533+ "enabled": true,
15534+ "last_error": null,
15535+ "last_sent_us": null
15536+ },
15537+ "notes": "`endpoint` is the OTLP/HTTP base address; `/v1/traces`, `/v1/logs` and `/v1/metrics` are added. Requests go as log records. Telemetry is sent on as it arrives, from then on. Needs the Admin role."
15538+ },
15539+ "update_telemetry_export": {
15540+ "params": {
15541+ "workspace": "flagon-io",
15542+ "project": "g1t",
15543+ "id": "x7d2k9m4"
15544+ },
15545+ "request": {
15546+ "enabled": false
15547+ },
15548+ "response": {
15549+ "id": "x7d2k9m4",
15550+ "name": "Honeycomb",
15551+ "endpoint": "https://api.honeycomb.io",
15552+ "signals": [
15553+ "traces",
15554+ "logs"
15555+ ],
15556+ "header_names": [
15557+ "x-honeycomb-team"
15558+ ],
15559+ "enabled": false,
15560+ "last_error": null,
15561+ "last_sent_us": 1791813809000000
15562+ },
15563+ "notes": "Only what is given changes. A header given is set, a header given as `\"\"` is removed, and the rest are kept. Needs the Admin role."
15564+ },
15565+ "delete_telemetry_export": {
15566+ "params": {
15567+ "workspace": "flagon-io",
15568+ "project": "g1t",
15569+ "id": "x7d2k9m4"
15570+ },
15571+ "response": true,
15572+ "notes": "Needs the Admin role."
15573+ },
15574+ "get_observability_usage": {
15575+ "params": {
15576+ "workspace": "flagon-io",
15577+ "project": "g1t"
15578+ },
15579+ "response": {
15580+ "month": "2026-10",
15581+ "ingested_bytes": 4831920112,
15582+ "requests": 1840221,
15583+ "logs": 6204118,
15584+ "spans": 9921540,
15585+ "metric_points": 3120400,
15586+ "stored_bytes": 1210443776
15587+ },
15588+ "notes": "This month so far. `ingested_bytes` is what arrived, after decoding: what ingest is metered by. Needs the Admin role."
1493115589 }
1493215590 }
+2−0
117117 Op::Rules(RulesOp::DeleteRepoRuleset | RulesOp::DeleteWorkspaceRuleset) => return as_is,
118118 // Deployments travel in `snake_case` between services too.
119119 Op::Deployments(_) => return as_is,
120+ // So does telemetry.
121+ Op::Telemetry(_) => return as_is,
120122 // Artifacts are shaped by the API itself, in `snake_case`.
121123 Op::Artifacts(_) => return as_is,
122124 // So are archive links.
+103−0
99 use crate::deploy_keys::DeployKeysOp;
1010 use crate::mirrors::MirrorsOp;
1111 use crate::deployments::DeploymentsOp;
12+use crate::telemetry::TelemetryOp;
1213 use crate::packages::PackagesOp;
1314 use crate::folios::FoliosOp;
1415 use crate::protection::ProtectionOp;
158159 route("GET", "/workspaces/:workspace/projects", Op::ListProjects, &[]),
159160 route("GET", "/workspaces/:workspace/projects/:project", Op::GetProject, &[]),
160161 route("PATCH", "/workspaces/:workspace/projects/:project", Op::UpdateProject, &[]),
162+ // A project's observability: what its software did, and how it is collected.
163+ route(
164+ "GET",
165+ "/workspaces/:workspace/projects/:project/observability/overview",
166+ Op::Telemetry(TelemetryOp::GetOverview),
167+ &[("from", "from"), ("to", "to"), ("environment", "environment"), ("service", "service"), ("deployment_id", "deployment_id"), ("step", "step")],
168+ ),
169+ route(
170+ "GET",
171+ "/workspaces/:workspace/projects/:project/observability/requests",
172+ Op::Telemetry(TelemetryOp::ListRequests),
173+ &[
174+ ("from", "from"),
175+ ("to", "to"),
176+ ("q", "q"),
177+ ("status", "status"),
178+ ("errors_only", "errors_only"),
179+ ("route", "route"),
180+ ("min_duration_ms", "min_duration_ms"),
181+ ("environment", "environment"),
182+ ("service", "service"),
183+ ("deployment_id", "deployment_id"),
184+ ("cursor", "cursor"),
185+ ("limit", "limit"),
186+ ],
187+ ),
188+ route("GET", "/workspaces/:workspace/projects/:project/observability/requests/:id", Op::Telemetry(TelemetryOp::GetRequest), &[]),
189+ route(
190+ "GET",
191+ "/workspaces/:workspace/projects/:project/observability/logs",
192+ Op::Telemetry(TelemetryOp::ListLogs),
193+ &[
194+ ("from", "from"),
195+ ("to", "to"),
196+ ("q", "q"),
197+ ("level", "level"),
198+ ("trace_id", "trace_id"),
199+ ("request_id", "request_id"),
200+ ("environment", "environment"),
201+ ("service", "service"),
202+ ("deployment_id", "deployment_id"),
203+ ("cursor", "cursor"),
204+ ("limit", "limit"),
205+ ],
206+ ),
207+ route(
208+ "GET",
209+ "/workspaces/:workspace/projects/:project/observability/traces",
210+ Op::Telemetry(TelemetryOp::ListTraces),
211+ &[
212+ ("from", "from"),
213+ ("to", "to"),
214+ ("q", "q"),
215+ ("errors_only", "errors_only"),
216+ ("min_duration_ms", "min_duration_ms"),
217+ ("environment", "environment"),
218+ ("service", "service"),
219+ ("cursor", "cursor"),
220+ ("limit", "limit"),
221+ ],
222+ ),
223+ route("GET", "/workspaces/:workspace/projects/:project/observability/traces/:trace_id", Op::Telemetry(TelemetryOp::GetTrace), &[]),
224+ route(
225+ "GET",
226+ "/workspaces/:workspace/projects/:project/observability/errors",
227+ Op::Telemetry(TelemetryOp::ListErrors),
228+ &[("state", "state"), ("environment", "environment"), ("service", "service"), ("from", "from"), ("to", "to")],
229+ ),
230+ route("PATCH", "/workspaces/:workspace/projects/:project/observability/errors/:fingerprint", Op::Telemetry(TelemetryOp::UpdateError), &[]),
231+ route("GET", "/workspaces/:workspace/projects/:project/observability/metrics", Op::Telemetry(TelemetryOp::ListMetrics), &[]),
232+ route(
233+ "GET",
234+ "/workspaces/:workspace/projects/:project/observability/metrics/:metric",
235+ Op::Telemetry(TelemetryOp::GetMetric),
236+ &[("from", "from"), ("to", "to"), ("aggregate", "aggregate"), ("group_by", "group_by"), ("step", "step"), ("service", "service"), ("environment", "environment")],
237+ ),
238+ route("GET", "/workspaces/:workspace/projects/:project/observability/settings", Op::Telemetry(TelemetryOp::GetSettings), &[]),
239+ route("PATCH", "/workspaces/:workspace/projects/:project/observability/settings", Op::Telemetry(TelemetryOp::UpdateSettings), &[]),
240+ route("GET", "/workspaces/:workspace/projects/:project/observability/tokens", Op::Telemetry(TelemetryOp::ListTokens), &[]),
241+ route("POST", "/workspaces/:workspace/projects/:project/observability/tokens", Op::Telemetry(TelemetryOp::CreateToken), &[]),
242+ route("DELETE", "/workspaces/:workspace/projects/:project/observability/tokens/:id", Op::Telemetry(TelemetryOp::DeleteToken), &[]),
243+ route("GET", "/workspaces/:workspace/projects/:project/observability/exports", Op::Telemetry(TelemetryOp::ListExports), &[]),
244+ route("POST", "/workspaces/:workspace/projects/:project/observability/exports", Op::Telemetry(TelemetryOp::CreateExport), &[]),
245+ route("PATCH", "/workspaces/:workspace/projects/:project/observability/exports/:id", Op::Telemetry(TelemetryOp::UpdateExport), &[]),
246+ route("DELETE", "/workspaces/:workspace/projects/:project/observability/exports/:id", Op::Telemetry(TelemetryOp::DeleteExport), &[]),
247+ route("GET", "/workspaces/:workspace/projects/:project/observability/usage", Op::Telemetry(TelemetryOp::GetUsage), &[]),
161248 route("PUT", "/workspaces/:workspace/base_permission", Op::SetBasePermission, &[]),
162249 route(
163250 "GET",
14261513 }
14271514
14281515 #[test]
1516+ fn observability_is_addressed_by_workspace_and_project() {
1517+ let query = [("level".to_owned(), "error,warn".to_owned()), ("q".to_owned(), "timeout".to_owned())];
1518+ let (route, input) = resolve("GET", "/workspaces/acme/projects/web/observability/logs", &query, Value::Null).unwrap();
1519+ assert_eq!(route.op, Op::Telemetry(TelemetryOp::ListLogs));
1520+ assert_eq!(input, json!({ "level": "error,warn", "q": "timeout", "workspace": "acme", "project": "web" }));
1521+ let (route, input) =
1522+ resolve("GET", "/workspaces/acme/projects/web/observability/metrics/http.server.request.duration", &[], Value::Null).unwrap();
1523+ assert_eq!(route.op, Op::Telemetry(TelemetryOp::GetMetric));
1524+ assert_eq!(input["metric"], json!("http.server.request.duration"));
1525+ let (route, input) =
1526+ resolve("PATCH", "/workspaces/acme/projects/web/observability/errors/f00d", &[], json!({ "state": "resolved" })).unwrap();
1527+ assert_eq!(route.op, Op::Telemetry(TelemetryOp::UpdateError));
1528+ assert_eq!(input, json!({ "state": "resolved", "workspace": "acme", "project": "web", "fingerprint": "f00d" }));
1529+ }
1530+
1531+ #[test]
14291532 fn billing_is_addressed_by_workspace() {
14301533 let query = [("from".to_owned(), "2026-10-01".to_owned()), ("products".to_owned(), "agent,sandboxes".to_owned())];
14311534 let (route, input) = resolve("GET", "/workspaces/acme/usage", &query, Value::Null).unwrap();
+836−0
1+//! Observability over REST and MCP: what a project's running software did,
2+//! as requests, logs, traces, errors and metrics, and how it is collected
3+//! (settings, ingest tokens and exports).
4+//!
5+//! The telemetry service keeps each project's store and checks nothing:
6+//! this resolves the project through the projects service and decides
7+//! first, as the site does. Reading takes a role of the viewer's own on
8+//! the project's repository (logs and request addresses can hold private
9+//! data, so a public repository is not enough); resolving and ignoring
10+//! errors takes Triage; settings, ingest tokens, exports and usage take
11+//! Admin (`manage_integrations`). Anyone who may not read is told there is
12+//! no such project.
13+
14+use g1t_contracts::access::{self, Capability, RepoRef};
15+use g1t_contracts::{FailureCode, Outcome, Viewer};
16+use serde_json::{Map, Value, json};
17+use worker::{Headers, Method, Request, RequestInit, Result};
18+
19+use crate::operations::Services;
20+
21+/// One operation on a project's observability.
22+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
23+pub enum TelemetryOp {
24+ GetOverview,
25+ ListRequests,
26+ GetRequest,
27+ ListLogs,
28+ ListTraces,
29+ GetTrace,
30+ ListErrors,
31+ UpdateError,
32+ ListMetrics,
33+ GetMetric,
34+ GetSettings,
35+ UpdateSettings,
36+ ListTokens,
37+ CreateToken,
38+ DeleteToken,
39+ ListExports,
40+ CreateExport,
41+ UpdateExport,
42+ DeleteExport,
43+ GetUsage,
44+}
45+
46+/// Log levels, as the telemetry service groups them.
47+const SEVERITIES: [&str; 6] = ["trace", "debug", "info", "warn", "error", "fatal"];
48+/// An error group's states.
49+const ERROR_STATES: [&str; 3] = ["open", "resolved", "ignored"];
50+/// What a metric's values in a step may be reduced to.
51+const AGGREGATES: [&str; 7] = ["avg", "sum", "min", "max", "p50", "p95", "p99"];
52+/// The signals an export may send.
53+const SIGNALS: [&str; 3] = ["traces", "logs", "metrics"];
54+/// Retention a project can choose, in days (`RETENTION_CHOICES` in @g1t/contracts).
55+const RETENTION_DAYS: [u64; 5] = [1, 3, 7, 14, 30];
56+
57+/// Fields keyed by what the software sent (OpenTelemetry attributes and
58+/// resources), passed through as they are rather than put in `snake_case`.
59+pub const AS_GIVEN: &[&str] = &["attributes", "resource"];
60+
61+/// What an operation needs of the viewer on the project's repository.
62+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
63+enum Need {
64+ /// A role of their own: a member, a collaborator, or the workspace's token.
65+ Insider,
66+ Can(Capability),
67+}
68+
69+impl TelemetryOp {
70+ /// Every one: `Op::ALL` lists each as `Op::Telemetry(…)`, which a test
71+ /// checks against this.
72+ #[cfg(test)]
73+ pub const ALL: [TelemetryOp; 20] = [
74+ TelemetryOp::GetOverview,
75+ TelemetryOp::ListRequests,
76+ TelemetryOp::GetRequest,
77+ TelemetryOp::ListLogs,
78+ TelemetryOp::ListTraces,
79+ TelemetryOp::GetTrace,
80+ TelemetryOp::ListErrors,
81+ TelemetryOp::UpdateError,
82+ TelemetryOp::ListMetrics,
83+ TelemetryOp::GetMetric,
84+ TelemetryOp::GetSettings,
85+ TelemetryOp::UpdateSettings,
86+ TelemetryOp::ListTokens,
87+ TelemetryOp::CreateToken,
88+ TelemetryOp::DeleteToken,
89+ TelemetryOp::ListExports,
90+ TelemetryOp::CreateExport,
91+ TelemetryOp::UpdateExport,
92+ TelemetryOp::DeleteExport,
93+ TelemetryOp::GetUsage,
94+ ];
95+
96+ pub fn name(self) -> &'static str {
97+ match self {
98+ TelemetryOp::GetOverview => "get_observability_overview",
99+ TelemetryOp::ListRequests => "list_requests",
100+ TelemetryOp::GetRequest => "get_request",
101+ TelemetryOp::ListLogs => "list_logs",
102+ TelemetryOp::ListTraces => "list_traces",
103+ TelemetryOp::GetTrace => "get_trace",
104+ TelemetryOp::ListErrors => "list_errors",
105+ TelemetryOp::UpdateError => "update_error",
106+ TelemetryOp::ListMetrics => "list_metrics",
107+ TelemetryOp::GetMetric => "get_metric",
108+ TelemetryOp::GetSettings => "get_observability_settings",
109+ TelemetryOp::UpdateSettings => "update_observability_settings",
110+ TelemetryOp::ListTokens => "list_ingest_tokens",
111+ TelemetryOp::CreateToken => "create_ingest_token",
112+ TelemetryOp::DeleteToken => "delete_ingest_token",
113+ TelemetryOp::ListExports => "list_telemetry_exports",
114+ TelemetryOp::CreateExport => "create_telemetry_export",
115+ TelemetryOp::UpdateExport => "update_telemetry_export",
116+ TelemetryOp::DeleteExport => "delete_telemetry_export",
117+ TelemetryOp::GetUsage => "get_observability_usage",
118+ }
119+ }
120+
121+ /// For the API reference: "List logs".
122+ pub fn title(self) -> &'static str {
123+ match self {
124+ TelemetryOp::GetOverview => "Get a project's observability overview",
125+ TelemetryOp::ListRequests => "List requests",
126+ TelemetryOp::GetRequest => "Get a request",
127+ TelemetryOp::ListLogs => "List logs",
128+ TelemetryOp::ListTraces => "List traces",
129+ TelemetryOp::GetTrace => "Get a trace",
130+ TelemetryOp::ListErrors => "List errors",
131+ TelemetryOp::UpdateError => "Update an error",
132+ TelemetryOp::ListMetrics => "List metrics",
133+ TelemetryOp::GetMetric => "Get a metric",
134+ TelemetryOp::GetSettings => "Get observability settings",
135+ TelemetryOp::UpdateSettings => "Update observability settings",
136+ TelemetryOp::ListTokens => "List ingest tokens",
137+ TelemetryOp::CreateToken => "Create an ingest token",
138+ TelemetryOp::DeleteToken => "Delete an ingest token",
139+ TelemetryOp::ListExports => "List telemetry exports",
140+ TelemetryOp::CreateExport => "Create a telemetry export",
141+ TelemetryOp::UpdateExport => "Update a telemetry export",
142+ TelemetryOp::DeleteExport => "Delete a telemetry export",
143+ TelemetryOp::GetUsage => "Get observability usage",
144+ }
145+ }
146+
147+ pub fn description(self) -> &'static str {
148+ match self {
149+ TelemetryOp::GetOverview => "A project's numbers over a period (the last 24 hours unless you say): totals (requests, errors, error_rate, and latency and CPU percentiles p50, p75, p95 and p99 in milliseconds) and the same for the period before it, to compare; points, one per step (step_s seconds), with requests, errors, counts by status class and percentiles; routes, by requests; deployments, each one's numbers newest first, so a regression points at the deployment it began with; services and regions. from and to are RFC 3339 times or Unix milliseconds. Narrow it by environment, service or deployment_id. Needs a role on the project's repository.",
150+ TelemetryOp::ListRequests => "The requests (and other invocations) a project's software answered, newest first: each with its method, url, route (ids taken out), status, outcome, duration_us, cpu_ms, region, trace_id, log_count, error, and where it came from (service, environment, deployment_id, commit_sha, source). Apps on g1t.page send them with nothing to set up; software elsewhere, as OpenTelemetry server spans. Filter by words in q, status (2xx, 3xx, 4xx, 5xx or a status, comma-separated), errors_only (a 5xx or an outcome other than ok), route, min_duration_ms, environment, service and deployment_id, between from and to. A page holds limit; pass next back as cursor for the next. Needs a role on the project's repository.",
151+ TelemetryOp::GetRequest => "One request by id, with every log line it wrote, oldest first. Needs a role on the project's repository.",
152+ TelemetryOp::ListLogs => "Search a project's logs, newest first. q finds lines holding all of its words, or a phrase in quotes. level narrows to levels (trace, debug, info, warn, error, fatal, comma-separated); trace_id and request_id to one trace's or one request's lines; environment, service and deployment_id to where they ran; from and to to a period. Each line has its time_us, severity, body, attributes, trace_id, span_id and request_id. A page holds limit; pass next back as cursor for the next. Needs a role on the project's repository.",
153+ TelemetryOp::ListTraces => "A project's traces, newest first: each its root span's name, start_us, duration_us, span_count, error_count, the services it crossed and its HTTP status. Filter by words in q, errors_only, min_duration_ms, environment and service, between from and to. A page holds limit; pass next back as cursor for the next. Needs a role on the project's repository.",
154+ TelemetryOp::GetTrace => "One trace by its id (32 hex characters): every span, with its parent, kind, timing, status, attributes, resource, events and links, and the log lines written under it. Needs a role on the project's repository.",
155+ TelemetryOp::ListErrors => "A project's errors, grouped: the same exception (its name, its message with numbers and ids taken out, and the top frame of its stack) is one group, with its count, first and last time seen, the deployment it was first seen on (what likely brought it in), its last request and trace, its state (open, resolved or ignored) and the issue made for it. Most recently seen first, at most 200. Filter by state, environment and service, between from and to. Needs a role on the project's repository.",
156+ TelemetryOp::UpdateError => "Resolve, ignore or reopen an error group (state), or link it to the issue made for it (issue_number; null unlinks it). A resolved group that happens again opens again. Needs the Triage role or higher.",
157+ TelemetryOp::ListMetrics => "The metrics a project's software has sent: each with its name, kind (gauge, sum, histogram or summary), unit, description, the services that send it and when it last did. Needs a role on the project's repository.",
158+ TelemetryOp::GetMetric => "One metric over time, one value per step: gauges averaged in each step (or min or max), sums added (a cumulative sum becomes its increase), histograms their mean or a percentile (p50, p95, p99) from their buckets. group_by splits it into a series per value of an attribute (or service). The last 24 hours unless from and to say. Needs a role on the project's repository.",
159+ TelemetryOp::GetSettings => "How long a project's requests, logs and spans are kept (retention_days; charts are kept longer, as per-minute counts) and whether its apps on g1t.page send telemetry (deployments). Needs the Admin role.",
160+ TelemetryOp::UpdateSettings => "Change how long a project's telemetry is kept (retention_days: 1, 3, 7, 14 or 30) and whether its apps on g1t.page send it (deployments; off keeps the tokens and history). Only what you give changes. Needs the Admin role.",
161+ TelemetryOp::ListTokens => "A project's ingest tokens, which OpenTelemetry SDKs and collectors send with: each its name, hint (its first characters), who made it, when, and when it was last used. managed marks the one g1t made for its own deployments, which people cannot delete. Secrets are never returned. Needs the Admin role.",
162+ TelemetryOp::CreateToken => "Make an ingest token for a project. Returns token and secret: the secret is shown this once, so keep it. Send it as `Authorization: Bearer <secret>` (or `x-g1t-token`) with OTLP over HTTP to the project's endpoint. Needs the Admin role.",
163+ TelemetryOp::DeleteToken => "Delete an ingest token: what sends with it is refused from then on. The token g1t made for its own deployments cannot be deleted. Needs the Admin role.",
164+ TelemetryOp::ListExports => "Where a project's telemetry is also sent as it arrives: each export's name, OTLP/HTTP endpoint, signals (traces, logs, metrics), header_names (values are never returned), whether it is enabled, and its last error and send. Needs the Admin role.",
165+ TelemetryOp::CreateExport => "Send a project's telemetry on to any OTLP/HTTP endpoint as it arrives: a collector, or a vendor that takes OTLP. signals are traces, logs (requests go as log records) and metrics, all three unless you say; headers are sent with each request, such as an API key, and are written, never read back. Needs the Admin role.",
166+ TelemetryOp::UpdateExport => "Change an export: its name, endpoint, signals and whether it is enabled. Only what you give changes. Headers given are set; a header given as \"\" is removed; those not given are kept. Needs the Admin role.",
167+ TelemetryOp::DeleteExport => "Stop sending a project's telemetry to an export. Needs the Admin role.",
168+ TelemetryOp::GetUsage => "What a project has sent and kept this month: ingested_bytes (what ingest is metered by), requests, logs, spans, metric_points and stored_bytes. Needs the Admin role.",
169+ }
170+ }
171+
172+ /// Whether it changes anything.
173+ pub fn writes(self) -> bool {
174+ matches!(
175+ self,
176+ TelemetryOp::UpdateError
177+ | TelemetryOp::UpdateSettings
178+ | TelemetryOp::CreateToken
179+ | TelemetryOp::DeleteToken
180+ | TelemetryOp::CreateExport
181+ | TelemetryOp::UpdateExport
182+ | TelemetryOp::DeleteExport
183+ )
184+ }
185+
186+ fn need(self) -> Need {
187+ match self {
188+ TelemetryOp::UpdateError => Need::Can(Capability::Triage),
189+ TelemetryOp::GetSettings
190+ | TelemetryOp::UpdateSettings
191+ | TelemetryOp::ListTokens
192+ | TelemetryOp::CreateToken
193+ | TelemetryOp::DeleteToken
194+ | TelemetryOp::ListExports
195+ | TelemetryOp::CreateExport
196+ | TelemetryOp::UpdateExport
197+ | TelemetryOp::DeleteExport
198+ | TelemetryOp::GetUsage => Need::Can(Capability::ManageIntegrations),
199+ _ => Need::Insider,
200+ }
201+ }
202+
203+ pub fn input(self) -> Value {
204+ let text = |what: &str| json!({ "type": "string", "description": what });
205+ let integer = |what: &str| json!({ "type": "integer", "description": what });
206+ let mut properties = Map::new();
207+ properties.insert("workspace".into(), text("The workspace's slug, e.g. \"flagon-io\"."));
208+ properties.insert("project".into(), text("The project's slug, as in g1t.sh/{workspace}/{project}."));
209+ let mut required = vec!["workspace", "project"];
210+ let mut add = |name: &str, schema: Value| {
211+ properties.insert(name.to_owned(), schema);
212+ };
213+ let period = |add: &mut dyn FnMut(&str, Value)| {
214+ add("from", text("The start of the period: an RFC 3339 time or Unix milliseconds. 24 hours before `to` unless you say."));
215+ add("to", text("The end of the period: an RFC 3339 time or Unix milliseconds. Now unless you say."));
216+ };
217+ let place = |add: &mut dyn FnMut(&str, Value)| {
218+ add("environment", text("Only what ran in this environment, such as production or preview; several comma-separated."));
219+ add("service", text("Only what this service sent (OpenTelemetry's service.name; an app on g1t.page is its project's name); several comma-separated."));
220+ };
221+ let page = |add: &mut dyn FnMut(&str, Value)| {
222+ add("cursor", text("Where to carry on from: the last page's `next`."));
223+ add("limit", integer("How many a page holds."));
224+ };
225+ let words = "Words to find, all of them, or a phrase in quotes.";
226+ match self {
227+ TelemetryOp::GetOverview => {
228+ period(&mut add);
229+ place(&mut add);
230+ add("deployment_id", text("Only this deployment's (dpl_…)."));
231+ add("step", integer("Seconds per point of the charts; chosen from the period unless you say."));
232+ }
233+ TelemetryOp::ListRequests => {
234+ period(&mut add);
235+ add("q", text(words));
236+ add("status", text("Only these statuses: 2xx, 3xx, 4xx, 5xx or a status such as 404, comma-separated."));
237+ add("errors_only", json!({ "type": "boolean", "description": "Only those that failed: a 5xx, or an outcome other than ok." }));
238+ add("route", text("Only this route, as the list shows it, such as /users/:id."));
239+ add("min_duration_ms", integer("Only those that took at least this many milliseconds."));
240+ place(&mut add);
241+ add("deployment_id", text("Only this deployment's (dpl_…)."));
242+ page(&mut add);
243+ }
244+ TelemetryOp::GetRequest => {
245+ add("id", text("The request's id."));
246+ required.push("id");
247+ }
248+ TelemetryOp::ListLogs => {
249+ period(&mut add);
250+ add("q", text(words));
251+ add("level", json!({ "type": "string", "description": "Only these levels: trace, debug, info, warn, error or fatal, comma-separated." }));
252+ add("trace_id", text("Only the lines written under this trace."));
253+ add("request_id", text("Only the lines this request wrote."));
254+ place(&mut add);
255+ add("deployment_id", text("Only this deployment's (dpl_…)."));
256+ page(&mut add);
257+ }
258+ TelemetryOp::ListTraces => {
259+ period(&mut add);
260+ add("q", text(words));
261+ add("errors_only", json!({ "type": "boolean", "description": "Only traces with a span that failed." }));
262+ add("min_duration_ms", integer("Only traces that took at least this many milliseconds."));
263+ place(&mut add);
264+ page(&mut add);
265+ }
266+ TelemetryOp::GetTrace => {
267+ add("trace_id", text("The trace's id, 32 hex characters."));
268+ required.push("trace_id");
269+ }
270+ TelemetryOp::ListErrors => {
271+ add("state", json!({ "type": "string", "enum": ERROR_STATES, "description": "Only groups in this state." }));
272+ place(&mut add);
273+ period(&mut add);
274+ }
275+ TelemetryOp::UpdateError => {
276+ add("fingerprint", text("The error group's fingerprint, as list_errors gives it."));
277+ add("state", json!({ "type": "string", "enum": ERROR_STATES, "description": "Its state now: resolved, ignored, or open again." }));
278+ add("issue_number", json!({ "type": ["integer", "null"], "description": "The issue made for it; null unlinks it." }));
279+ required.push("fingerprint");
280+ }
281+ TelemetryOp::ListMetrics
282+ | TelemetryOp::GetSettings
283+ | TelemetryOp::ListTokens
284+ | TelemetryOp::ListExports
285+ | TelemetryOp::GetUsage => {}
286+ TelemetryOp::GetMetric => {
287+ add("metric", text("The metric's name, such as http.server.request.duration."));
288+ required.push("metric");
289+ period(&mut add);
290+ add("aggregate", json!({ "type": "string", "enum": AGGREGATES, "description": "How each step's values are reduced: avg for gauges and sum for sums unless you say; a percentile for histograms." }));
291+ add("group_by", text("An attribute (or service) to split it into a series per value."));
292+ add("step", integer("Seconds per point; chosen from the period unless you say."));
293+ place(&mut add);
294+ }
295+ TelemetryOp::UpdateSettings => {
296+ add("retention_days", json!({ "type": "integer", "enum": RETENTION_DAYS, "description": "Days requests, logs and spans are kept." }));
297+ add("deployments", json!({ "type": "boolean", "description": "Whether the project's apps on g1t.page send telemetry." }));
298+ }
299+ TelemetryOp::CreateToken => {
300+ add("name", text("What it is for, such as the machine or collector that sends with it."));
301+ required.push("name");
302+ }
303+ TelemetryOp::DeleteToken => {
304+ add("id", text("The token's id."));
305+ required.push("id");
306+ }
307+ TelemetryOp::CreateExport | TelemetryOp::UpdateExport => {
308+ if self == TelemetryOp::UpdateExport {
309+ add("id", text("The export's id."));
310+ required.push("id");
311+ } else {
312+ required.extend(["name", "endpoint"]);
313+ }
314+ add("name", text("Its name, for people."));
315+ add("endpoint", text("The OTLP/HTTP base address, such as https://otlp.example.com; /v1/traces and the rest are added."));
316+ add("signals", json!({ "type": "array", "items": { "type": "string", "enum": SIGNALS }, "description": "What it sends: traces, logs and metrics. All three unless you say." }));
317+ add("headers", json!({ "type": "object", "additionalProperties": { "type": "string" }, "description": "Headers sent with each request, such as an API key. Written, never read back; \"\" removes one." }));
318+ add("enabled", json!({ "type": "boolean", "description": "Whether it sends. True unless you say." }));
319+ }
320+ TelemetryOp::DeleteExport => {
321+ add("id", text("The export's id."));
322+ required.push("id");
323+ }
324+ }
325+ json!({ "type": "object", "properties": properties, "required": required })
326+ }
327+}
328+
329+/// A non-empty string field, trimmed.
330+fn text(input: &Value, key: &str) -> Option<String> {
331+ input[key].as_str().map(str::trim).filter(|value| !value.is_empty()).map(str::to_owned)
332+}
333+
334+/// A number given as a number or as text.
335+fn number(input: &Value, key: &str) -> std::result::Result<Option<u64>, String> {
336+ match &input[key] {
337+ Value::Null => Ok(None),
338+ Value::Number(number) => number.as_u64().map(Some).ok_or_else(|| format!("{key} must be a whole number.")),
339+ Value::String(digits) if digits.trim().is_empty() => Ok(None),
340+ Value::String(digits) => digits.trim().parse().map(Some).map_err(|_| format!("{key} must be a whole number.")),
341+ _ => Err(format!("{key} must be a whole number.")),
342+ }
343+}
344+
345+/// A flag given as a boolean or as text.
346+fn flag(input: &Value, key: &str) -> std::result::Result<Option<bool>, String> {
347+ match &input[key] {
348+ Value::Null => Ok(None),
349+ Value::Bool(value) => Ok(Some(*value)),
350+ Value::String(text) => match text.trim() {
351+ "" => Ok(None),
352+ "true" | "1" => Ok(Some(true)),
353+ "false" | "0" => Ok(Some(false)),
354+ _ => Err(format!("{key} is true or false.")),
355+ },
356+ _ => Err(format!("{key} is true or false.")),
357+ }
358+}
359+
360+/// A list given as an array or as comma-separated text.
361+fn list(input: &Value, key: &str) -> Vec<String> {
362+ match &input[key] {
363+ Value::Array(items) => items.iter().filter_map(Value::as_str).map(str::trim).filter(|item| !item.is_empty()).map(str::to_owned).collect(),
364+ Value::String(text) => text.split(',').map(str::trim).filter(|item| !item.is_empty()).map(str::to_owned).collect(),
365+ _ => Vec::new(),
366+ }
367+}
368+
369+/// A time as microseconds since the epoch, from Unix milliseconds or an
370+/// RFC 3339 time, in UTC (`Z`) or with an offset (`+02:00`).
371+pub(crate) fn micros(given: &Value) -> Option<u64> {
372+ let text = match given {
373+ Value::Number(number) => return number.as_u64().map(|ms| ms * 1000),
374+ Value::String(text) => text.trim(),
375+ _ => return None,
376+ };
377+ if !text.is_empty() && text.bytes().all(|byte| byte.is_ascii_digit()) {
378+ return text.parse::<u64>().ok().map(|ms| ms * 1000);
379+ }
380+ if let Some(ms) = g1t_contracts::time::parse_rfc3339(text) {
381+ return Some(ms * 1000);
382+ }
383+ // An offset: the time there, less the offset, is UTC.
384+ let at = text.len().checked_sub(6)?;
385+ let (local, offset) = text.split_at(at);
386+ let sign: i64 = match offset.as_bytes().first() {
387+ Some(b'+') => 1,
388+ Some(b'-') => -1,
389+ _ => return None,
390+ };
391+ let (hours, minutes) = offset[1..].split_once(':')?;
392+ let shift = sign * (hours.parse::<i64>().ok()? * 60 + minutes.parse::<i64>().ok()?) * 60_000;
393+ let ms = g1t_contracts::time::parse_rfc3339(&format!("{local}Z"))? as i64 - shift;
394+ u64::try_from(ms).ok().map(|ms| ms * 1000)
395+}
396+
397+/// The period and the place every list narrows by, in the service's names.
398+fn filter(input: &Value, out: &mut Map<String, Value>) -> std::result::Result<(), String> {
399+ for (key, field) in [("from", "from_us"), ("to", "to_us")] {
400+ let given = &input[key];
401+ if given.is_null() || given.as_str().is_some_and(|text| text.trim().is_empty()) {
402+ continue;
403+ }
404+ let at = micros(given).ok_or_else(|| format!("{key} must be an RFC 3339 time or Unix milliseconds."))?;
405+ out.insert(field.into(), at.into());
406+ }
407+ for (key, field) in [("environment", "environments"), ("service", "services")] {
408+ let values = list(input, key);
409+ if !values.is_empty() {
410+ out.insert(field.into(), json!(values));
411+ }
412+ }
413+ Ok(())
414+}
415+
416+/// The arguments the telemetry service takes for `op`, from the
417+/// operation's input, without the project; `Err` says what is missing or
418+/// wrong. `actor` is who makes a token.
419+pub(crate) fn args(op: TelemetryOp, input: &Value, actor: Option<&str>) -> std::result::Result<(&'static str, Map<String, Value>), String> {
420+ let mut out = Map::new();
421+ let copy = |out: &mut Map<String, Value>, keys: &[(&str, &str)]| {
422+ for (key, field) in keys {
423+ if let Some(value) = text(input, key) {
424+ out.insert((*field).to_owned(), value.into());
425+ }
426+ }
427+ };
428+ let page = |out: &mut Map<String, Value>| -> std::result::Result<(), String> {
429+ copy(out, &[("cursor", "cursor")]);
430+ if let Some(limit) = number(input, "limit")? {
431+ out.insert("limit".into(), limit.into());
432+ }
433+ Ok(())
434+ };
435+ let required = |key: &str, what: &str| text(input, key).ok_or_else(|| format!("Give {what}."));
436+ let method = match op {
437+ TelemetryOp::GetOverview => {
438+ filter(input, &mut out)?;
439+ copy(&mut out, &[("deployment_id", "deployment_id")]);
440+ if let Some(step) = number(input, "step")? {
441+ out.insert("step_s".into(), step.into());
442+ }
443+ "overview"
444+ }
445+ TelemetryOp::ListRequests => {
446+ filter(input, &mut out)?;
447+ copy(&mut out, &[("q", "text"), ("route", "route"), ("deployment_id", "deployment_id")]);
448+ let status = list(input, "status");
449+ if !status.is_empty() {
450+ out.insert("status".into(), json!(status));
451+ }
452+ if let Some(errors) = flag(input, "errors_only")? {
453+ out.insert("errors_only".into(), errors.into());
454+ }
455+ if let Some(ms) = number(input, "min_duration_ms")? {
456+ out.insert("min_duration_ms".into(), ms.into());
457+ }
458+ page(&mut out)?;
459+ "requests"
460+ }
461+ TelemetryOp::GetRequest => {
462+ out.insert("id".into(), required("id", "the request's id")?.into());
463+ "request"
464+ }
465+ TelemetryOp::ListLogs => {
466+ filter(input, &mut out)?;
467+ copy(&mut out, &[("q", "text"), ("trace_id", "trace_id"), ("request_id", "request_id"), ("deployment_id", "deployment_id")]);
468+ let levels: Vec<String> = list(input, "level").into_iter().map(|level| level.to_lowercase()).collect();
469+ if let Some(unknown) = levels.iter().find(|level| !SEVERITIES.contains(&level.as_str())) {
470+ return Err(format!("{unknown} is not a level: use trace, debug, info, warn, error or fatal."));
471+ }
472+ if !levels.is_empty() {
473+ out.insert("severities".into(), json!(levels));
474+ }
475+ page(&mut out)?;
476+ "logs"
477+ }
478+ TelemetryOp::ListTraces => {
479+ filter(input, &mut out)?;
480+ copy(&mut out, &[("q", "text")]);
481+ if let Some(errors) = flag(input, "errors_only")? {
482+ out.insert("errors_only".into(), errors.into());
483+ }
484+ if let Some(ms) = number(input, "min_duration_ms")? {
485+ out.insert("min_duration_ms".into(), ms.into());
486+ }
487+ page(&mut out)?;
488+ "traces"
489+ }
490+ TelemetryOp::GetTrace => {
491+ out.insert("trace_id".into(), required("trace_id", "the trace's id")?.to_lowercase().into());
492+ "trace"
493+ }
494+ TelemetryOp::ListErrors => {
495+ filter(input, &mut out)?;
496+ if let Some(state) = text(input, "state") {
497+ if !ERROR_STATES.contains(&state.as_str()) {
498+ return Err(format!("{state} is not a state: use open, resolved or ignored."));
499+ }
500+ out.insert("state".into(), state.into());
501+ }
502+ "errors"
503+ }
504+ TelemetryOp::UpdateError => {
505+ out.insert("fingerprint".into(), required("fingerprint", "the error group's fingerprint")?.into());
506+ if let Some(state) = text(input, "state") {
507+ if !ERROR_STATES.contains(&state.as_str()) {
508+ return Err(format!("{state} is not a state: use open, resolved or ignored."));
509+ }
510+ out.insert("state".into(), state.into());
511+ }
512+ match input.get("issue_number") {
513+ None => {}
514+ Some(Value::Null) => {
515+ out.insert("issue_number".into(), Value::Null);
516+ }
517+ Some(_) => {
518+ let issue = number(input, "issue_number")?.ok_or("issue_number must be a whole number, or null.")?;
519+ out.insert("issue_number".into(), issue.into());
520+ }
521+ }
522+ if !out.contains_key("state") && !out.contains_key("issue_number") {
523+ return Err("Give a state, or an issue_number.".to_owned());
524+ }
525+ "set_error"
526+ }
527+ TelemetryOp::ListMetrics => "metrics",
528+ TelemetryOp::GetMetric => {
529+ out.insert("name".into(), required("metric", "the metric's name")?.into());
530+ filter(input, &mut out)?;
531+ if let Some(aggregate) = text(input, "aggregate") {
532+ if !AGGREGATES.contains(&aggregate.as_str()) {
533+ return Err(format!("{aggregate} is not an aggregate: use avg, sum, min, max, p50, p95 or p99."));
534+ }
535+ out.insert("aggregate".into(), aggregate.into());
536+ }
537+ copy(&mut out, &[("group_by", "group_by")]);
538+ if let Some(step) = number(input, "step")? {
539+ out.insert("step_s".into(), step.into());
540+ }
541+ "metric"
542+ }
543+ TelemetryOp::GetSettings => "settings",
544+ TelemetryOp::UpdateSettings => {
545+ if let Some(days) = number(input, "retention_days")? {
546+ if !RETENTION_DAYS.contains(&days) {
547+ return Err("retention_days is 1, 3, 7, 14 or 30.".to_owned());
548+ }
549+ out.insert("retention_days".into(), days.into());
550+ }
551+ if let Some(on) = flag(input, "deployments")? {
552+ out.insert("deployments".into(), on.into());
553+ }
554+ "set_settings"
555+ }
556+ TelemetryOp::ListTokens => "tokens",
557+ TelemetryOp::CreateToken => {
558+ out.insert("name".into(), required("name", "the token a name")?.into());
559+ out.insert("created_by".into(), actor.map_or(Value::Null, Value::from));
560+ "create_token"
561+ }
562+ TelemetryOp::DeleteToken => {
563+ out.insert("id".into(), required("id", "the token's id")?.into());
564+ "revoke_token"
565+ }
566+ TelemetryOp::ListExports => "exports",
567+ TelemetryOp::CreateExport | TelemetryOp::UpdateExport => {
568+ if op == TelemetryOp::UpdateExport {
569+ out.insert("id".into(), required("id", "the export's id")?.into());
570+ } else {
571+ required("name", "the export a name")?;
572+ required("endpoint", "the endpoint to send to")?;
573+ }
574+ copy(&mut out, &[("name", "name"), ("endpoint", "endpoint")]);
575+ if !input["signals"].is_null() {
576+ let signals = list(input, "signals");
577+ if let Some(unknown) = signals.iter().find(|signal| !SIGNALS.contains(&signal.as_str())) {
578+ return Err(format!("{unknown} is not a signal: use traces, logs or metrics."));
579+ }
580+ out.insert("signals".into(), json!(signals));
581+ }
582+ match &input["headers"] {
583+ Value::Null => {}
584+ Value::Object(headers) => {
585+ let mut sent = Map::new();
586+ for (name, value) in headers {
587+ let value = value.as_str().ok_or_else(|| format!("The header {name} must be text."))?;
588+ sent.insert(name.trim().to_owned(), value.into());
589+ }
590+ out.insert("headers".into(), Value::Object(sent));
591+ }
592+ _ => return Err("headers must be an object of names and values.".to_owned()),
593+ }
594+ if let Some(enabled) = flag(input, "enabled")? {
595+ out.insert("enabled".into(), enabled.into());
596+ }
597+ "put_export"
598+ }
599+ TelemetryOp::DeleteExport => {
600+ out.insert("id".into(), required("id", "the export's id")?.into());
601+ "delete_export"
602+ }
603+ TelemetryOp::GetUsage => "usage",
604+ };
605+ Ok((method, out))
606+}
607+
608+/// Calls the telemetry service: its answer, or its refusal of a bad
609+/// argument (`400`, in words) as invalid.
610+async fn ask(services: &Services, method: &str, args: &Value) -> Result<Outcome<Value>> {
611+ let headers = Headers::new();
612+ headers.set("content-type", "application/json")?;
613+ let mut init = RequestInit::new();
614+ init.with_method(Method::Post)
615+ .with_headers(headers)
616+ .with_body(Some(serde_json::to_string(args)?.into()));
617+ let request = Request::new_with_init(&format!("https://service/rpc/{method}"), &init)?;
618+ let mut response = services.telemetry.fetch_request(request).await?;
619+ let text = response.text().await.unwrap_or_default();
620+ match response.status_code() {
621+ 200 => Ok(Outcome::Ok(g1t_kit::read_answer(method, &text)?)),
622+ 400 => Ok(Outcome::fail(FailureCode::Invalid, text.trim())),
623+ status => Err(worker::Error::RustError(format!("telemetry {method} failed with status {status}: {text}"))),
624+ }
625+}
626+
627+/// The repository a project's permission comes from, as the deployments
628+/// service reads it: a mirrored project has none, and its workspace's base
629+/// permission decides.
630+fn repo_of(project: &Value) -> (String, String, bool) {
631+ let source = &project["source"];
632+ let private = project["private"].as_bool().unwrap_or(true);
633+ if source["kind"] == "hosted" {
634+ (
635+ source["repoId"].as_str().unwrap_or_default().to_owned(),
636+ source["repo"]["namespace"].as_str().unwrap_or_default().to_owned(),
637+ private,
638+ )
639+ } else {
640+ (String::new(), project["workspace"].as_str().unwrap_or_default().to_owned(), private)
641+ }
642+}
643+
644+/// Whether the viewer may do what `need` says on the project; otherwise
645+/// the refusal.
646+fn allowed(viewer: &Viewer, project: &Value, need: Need) -> std::result::Result<(), Outcome<Value>> {
647+ let (id, namespace, private) = repo_of(project);
648+ let repo = RepoRef { id: &id, namespace: &namespace, private };
649+ let user = viewer.as_ref();
650+ if !user.is_some_and(|user| access::granted(user, repo).is_some()) {
651+ return Err(Outcome::fail(FailureCode::NotFound, "There is no such project."));
652+ }
653+ if let Need::Can(capability) = need
654+ && !access::can(user, repo, capability)
655+ {
656+ let name = format!("{}/{}", project["workspace"].as_str().unwrap_or_default(), project["slug"].as_str().unwrap_or_default());
657+ return Err(Outcome::fail(FailureCode::Forbidden, access::needs(capability, &name)));
658+ }
659+ Ok(())
660+}
661+
662+pub async fn run(op: TelemetryOp, services: &Services, viewer: &Viewer, input: &Value) -> Result<Outcome<Value>> {
663+ let slug = |key: &str| text(input, key).map(|value| value.to_lowercase());
664+ let (Some(workspace), Some(slug)) = (slug("workspace"), slug("project")) else {
665+ return Ok(Outcome::fail(FailureCode::Invalid, "Give the workspace's slug and the project's."));
666+ };
667+ let Some(user) = viewer else {
668+ let what = if op.writes() { "Changing" } else { "Reading" };
669+ return Ok(Outcome::fail(FailureCode::Unauthenticated, format!("{what} a project's observability needs a g1t access token.")));
670+ };
671+ let (method, mut args) = match args(op, input, Some(&user.username)) {
672+ Ok(found) => found,
673+ Err(message) => return Ok(Outcome::fail(FailureCode::Invalid, message)),
674+ };
675+ let found: Outcome<Value> =
676+ g1t_kit::call(&services.projects, "get", &json!({ "workspace": workspace, "slug": slug, "viewer": viewer })).await?;
677+ let project = match found {
678+ Outcome::Ok(project) => project,
679+ Outcome::Fail(failure) => return Ok(Outcome::Fail(failure)),
680+ };
681+ if let Err(refused) = allowed(viewer, &project, op.need()) {
682+ return Ok(refused);
683+ }
684+ let Some(project_id) = project["id"].as_str() else {
685+ return Ok(Outcome::fail(FailureCode::NotFound, "There is no such project."));
686+ };
687+ args.insert("project_id".into(), project_id.into());
688+
689+ // A change to an export names only what changes: the rest is as it is.
690+ if op == TelemetryOp::UpdateExport {
691+ let listed = ask(services, "exports", &json!({ "project_id": project_id })).await?;
692+ let Outcome::Ok(Value::Array(exports)) = listed else {
693+ return Ok(listed);
694+ };
695+ let Some(current) = exports.iter().find(|export| export["id"] == args["id"]) else {
696+ return Ok(Outcome::fail(FailureCode::NotFound, "The project has no export with that id."));
697+ };
698+ for key in ["name", "endpoint", "signals", "enabled"] {
699+ if !args.contains_key(key) {
700+ args.insert(key.into(), current[key].clone());
701+ }
702+ }
703+ }
704+ if op == TelemetryOp::CreateExport {
705+ args.entry("signals").or_insert_with(|| json!(SIGNALS));
706+ args.entry("enabled").or_insert(Value::Bool(true));
707+ }
708+
709+ let answered = ask(services, method, &Value::Object(args)).await?;
710+ // Nothing found answers null, and deleting nothing false.
711+ let missing = match op {
712+ TelemetryOp::GetRequest => Some("The project has no request with that id."),
713+ TelemetryOp::GetTrace => Some("The project has no trace with that id."),
714+ TelemetryOp::UpdateError => Some("The project has no error with that fingerprint."),
715+ TelemetryOp::DeleteToken => Some("The project has no ingest token with that id that people can delete."),
716+ TelemetryOp::DeleteExport => Some("The project has no export with that id."),
717+ _ => None,
718+ };
719+ Ok(match (answered, missing) {
720+ (Outcome::Ok(Value::Null | Value::Bool(false)), Some(message)) => Outcome::fail(FailureCode::NotFound, message),
721+ (answered, _) => answered,
722+ })
723+}
724+
725+#[cfg(test)]
726+mod tests {
727+ use super::*;
728+
729+ #[test]
730+ fn a_query_becomes_the_services_arguments() {
731+ let (method, args) = super::args(
732+ TelemetryOp::ListLogs,
733+ &json!({ "workspace": "acme", "project": "web", "q": "timeout", "level": "Error,warn", "environment": "production", "from": "1790918179000", "limit": "50", "ignored": 1 }),
734+ None,
735+ )
736+ .unwrap();
737+ assert_eq!(method, "logs");
738+ assert_eq!(args["text"], json!("timeout"));
739+ assert_eq!(args["severities"], json!(["error", "warn"]));
740+ assert_eq!(args["environments"], json!(["production"]));
741+ assert_eq!(args["from_us"], json!(1_790_918_179_000_000u64));
742+ assert_eq!(args["limit"], json!(50));
743+ assert!(!args.contains_key("ignored") && !args.contains_key("workspace"));
744+ assert!(super::args(TelemetryOp::ListLogs, &json!({ "level": "loud" }), None).is_err());
745+ assert!(super::args(TelemetryOp::ListLogs, &json!({ "from": "yesterday" }), None).is_err());
746+
747+ let (method, args) = super::args(
748+ TelemetryOp::ListRequests,
749+ &json!({ "status": "5xx,404", "errors_only": "true", "min_duration_ms": 250, "service": "api" }),
750+ None,
751+ )
752+ .unwrap();
753+ assert_eq!(method, "requests");
754+ assert_eq!((args["status"].clone(), args["errors_only"].clone(), args["min_duration_ms"].clone()), (json!(["5xx", "404"]), json!(true), json!(250)));
755+ assert_eq!(args["services"], json!(["api"]));
756+
757+ let (method, args) = super::args(TelemetryOp::GetMetric, &json!({ "metric": "http.server.request.duration", "aggregate": "p95", "step": "60" }), None).unwrap();
758+ assert_eq!((method, args["name"].clone(), args["step_s"].clone()), ("metric", json!("http.server.request.duration"), json!(60)));
759+ assert!(super::args(TelemetryOp::GetMetric, &json!({ "metric": "x", "aggregate": "median" }), None).is_err());
760+ }
761+
762+ #[test]
763+ fn changes_say_only_what_they_change() {
764+ let (method, args) = super::args(TelemetryOp::UpdateError, &json!({ "fingerprint": "f1", "issue_number": null }), None).unwrap();
765+ assert_eq!(method, "set_error");
766+ assert_eq!(args.get("issue_number"), Some(&Value::Null), "null unlinks the issue");
767+ assert!(!args.contains_key("state"));
768+ assert!(super::args(TelemetryOp::UpdateError, &json!({ "fingerprint": "f1" }), None).is_err());
769+ assert!(super::args(TelemetryOp::UpdateError, &json!({ "fingerprint": "f1", "state": "fixed" }), None).is_err());
770+ assert!(super::args(TelemetryOp::UpdateSettings, &json!({ "retention_days": 5 }), None).is_err());
771+ let (method, args) = super::args(TelemetryOp::CreateToken, &json!({ "name": "laptop" }), Some("ada")).unwrap();
772+ assert_eq!((method, args["created_by"].clone()), ("create_token", json!("ada")));
773+ let (method, args) = super::args(TelemetryOp::UpdateExport, &json!({ "id": "e1", "headers": { "x-api-key": "" } }), None).unwrap();
774+ assert_eq!(method, "put_export");
775+ assert_eq!(args["headers"], json!({ "x-api-key": "" }));
776+ assert!(!args.contains_key("signals"));
777+ assert!(super::args(TelemetryOp::CreateExport, &json!({ "name": "vendor" }), None).is_err());
778+ assert!(super::args(TelemetryOp::CreateExport, &json!({ "name": "v", "endpoint": "https://x", "signals": ["spans"] }), None).is_err());
779+ }
780+
781+ #[test]
782+ fn times_are_milliseconds_or_rfc_3339() {
783+ assert_eq!(micros(&json!(1_790_918_179_000u64)), Some(1_790_918_179_000_000));
784+ assert_eq!(micros(&json!("2026-10-02T05:16:19Z")), Some(1_790_918_179_000_000));
785+ assert_eq!(micros(&json!("2026-10-02T07:16:19+02:00")), Some(1_790_918_179_000_000));
786+ assert_eq!(micros(&json!("2026-10-02T00:16:19.500-05:00")), Some(1_790_918_179_500_000));
787+ assert_eq!(micros(&json!("soon")), None);
788+ }
789+
790+ #[test]
791+ fn reading_takes_a_role_of_ones_own() {
792+ use g1t_contracts::User;
793+ let project = json!({
794+ "id": "prj_1", "workspace": "acme", "slug": "web", "private": false,
795+ "source": { "kind": "hosted", "repoId": "repo_1", "repo": { "namespace": "acme", "name": "web" } },
796+ });
797+ let person = |username: &str, membership: Value| -> Viewer {
798+ let mut user = User { username: username.into(), ..User::default() };
799+ if !membership.is_null() {
800+ user.workspaces = vec![serde_json::from_value(membership).unwrap()];
801+ }
802+ Some(user)
803+ };
804+ let refused = |viewer: &Viewer, need: Need| match allowed(viewer, &project, need) {
805+ Err(Outcome::Fail(failure)) => Some(failure.code),
806+ _ => None,
807+ };
808+ // A public project's telemetry is not public.
809+ assert_eq!(refused(&person("eve", Value::Null), Need::Insider), Some(FailureCode::NotFound));
810+ assert_eq!(refused(&None, Need::Insider), Some(FailureCode::NotFound));
811+ let reader = person("ada", json!({ "slug": "acme", "role": "member", "base_permission": "read" }));
812+ assert_eq!(refused(&reader, Need::Insider), None);
813+ assert_eq!(refused(&reader, TelemetryOp::UpdateError.need()), Some(FailureCode::Forbidden));
814+ let writer = person("bo", json!({ "slug": "acme", "role": "member" }));
815+ assert_eq!(refused(&writer, TelemetryOp::UpdateError.need()), None);
816+ assert_eq!(refused(&writer, TelemetryOp::ListTokens.need()), Some(FailureCode::Forbidden));
817+ let owner = person("ana", json!({ "slug": "acme", "role": "owner" }));
818+ for op in TelemetryOp::ALL {
819+ assert_eq!(refused(&owner, op.need()), None, "{}", op.name());
820+ }
821+ assert_eq!(TelemetryOp::GetSettings.need(), Need::Can(Capability::ManageIntegrations));
822+ }
823+
824+ #[test]
825+ fn each_operation_is_described_with_a_schema() {
826+ for op in TelemetryOp::ALL {
827+ assert!(crate::operations::Op::ALL.contains(&crate::operations::Op::Telemetry(op)), "{}", op.name());
828+ assert!(!op.title().is_empty() && op.description().len() > 40, "{}", op.name());
829+ let required = op.input()["required"].as_array().unwrap().clone();
830+ assert!(required.contains(&json!("workspace")) && required.contains(&json!("project")), "{}", op.name());
831+ for field in required {
832+ assert!(op.input()["properties"].get(field.as_str().unwrap()).is_some(), "{}: {field}", op.name());
833+ }
834+ }
835+ }
836+}
+38−2
2525 use crate::deploy_keys::DeployKeysOp;
2626 use crate::mirrors::MirrorsOp;
2727 use crate::deployments::DeploymentsOp;
28+use crate::telemetry::TelemetryOp;
2829 use crate::packages::PackagesOp;
2930 use crate::folios::FoliosOp;
3031 use crate::protection::ProtectionOp;
300301 ],
301302 },
302303 Tool {
304+ name: "observability",
305+ title: "Observability",
306+ description: "What a project's running software did in production, to debug it: its requests, logs, traces, grouped errors and metrics, from its apps on g1t.page and from OpenTelemetry sent from anywhere. Name the project by workspace and project slug. Start with overview (error rate, latency, the routes and the deployment a regression began with) or errors, then follow a request or trace_id to its logs and spans. Times are RFC 3339 or Unix milliseconds; the last 24 hours unless you say. Needs a role of your own on the project's repository; resolving errors, Triage.",
307+ default_action: None,
308+ actions: &[
309+ a("overview", Op::Telemetry(TelemetryOp::GetOverview), "Requests, errors, latency and CPU over a period, by route, deployment, service and region, against the period before"),
310+ a("requests", Op::Telemetry(TelemetryOp::ListRequests), "Requests answered, newest first: q, status (5xx, 404), errors_only, route, min_duration_ms"),
311+ a("request", Op::Telemetry(TelemetryOp::GetRequest), "One request with every log line it wrote"),
312+ a("logs", Op::Telemetry(TelemetryOp::ListLogs), "Search a project's logs: words or a phrase in q, level (error,warn), trace_id, request_id"),
313+ a("traces", Op::Telemetry(TelemetryOp::ListTraces), "Traces, newest first: q, errors_only, min_duration_ms"),
314+ a("trace", Op::Telemetry(TelemetryOp::GetTrace), "One trace: every span, with its timing, status and attributes, and its logs"),
315+ a("errors", Op::Telemetry(TelemetryOp::ListErrors), "Errors grouped by exception, most recent first, with the deployment that brought each in"),
316+ a("update_error", Op::Telemetry(TelemetryOp::UpdateError), "Resolve, ignore or reopen an error group, or link the issue made for it"),
317+ a("metrics", Op::Telemetry(TelemetryOp::ListMetrics), "The metrics the project's software sends"),
318+ a("metric", Op::Telemetry(TelemetryOp::GetMetric), "One metric over time: aggregate (avg, sum, p95), group_by an attribute"),
319+ ],
320+ },
321+ Tool {
303322 name: "package",
304323 title: "Packages",
305324 description: "A workspace's packages in every registry (container images, npm, Cargo, Maven, NuGet, RubyGems, Composer): their versions and downloads, deleting and restoring them within 30 days, their visibility and repository, who has a role on them, and which repositories' workflows may use them (Manage Actions access). Name one by workspace, package_type and package_name.",
606625 },
607626 ];
608627
628+/// Operations offered over REST alone, not as an action of any tool: how
629+/// a project's telemetry is collected (settings, ingest tokens, exports,
630+/// usage) is for people setting it up, not for agents debugging.
631+pub const REST_ONLY: &[Op] = &[
632+ Op::Telemetry(TelemetryOp::GetSettings),
633+ Op::Telemetry(TelemetryOp::UpdateSettings),
634+ Op::Telemetry(TelemetryOp::ListTokens),
635+ Op::Telemetry(TelemetryOp::CreateToken),
636+ Op::Telemetry(TelemetryOp::DeleteToken),
637+ Op::Telemetry(TelemetryOp::ListExports),
638+ Op::Telemetry(TelemetryOp::CreateExport),
639+ Op::Telemetry(TelemetryOp::UpdateExport),
640+ Op::Telemetry(TelemetryOp::DeleteExport),
641+ Op::Telemetry(TelemetryOp::GetUsage),
642+];
643+
609644 /// Operations that cannot be undone, or reach beyond g1t's own records:
610645 /// clients ask before running a tool that has any of them.
611646 fn destructive(op: Op) -> bool {
881916 .flat_map(|tool| tool.actions.iter())
882917 .filter(|action| action.op == op)
883918 .count();
884− assert_eq!(count, 1, "{} is {count} actions", op.name());
919+ let expected = usize::from(!REST_ONLY.contains(&op));
920+ assert_eq!(count, expected, "{} is {count} actions", op.name());
885921 }
886922 for tool in TOOLS {
887923 let mut names = std::collections::HashSet::new();
892928 assert!(tool.action(default).is_some(), "{}", tool.name);
893929 }
894930 }
895− assert!(TOOLS.len() <= 20, "{} tools", TOOLS.len());
931+ assert!(TOOLS.len() <= 21, "{} tools", TOOLS.len());
896932 }
897933
898934 #[test]
+3−0
2525 { "binding": "ACTIONS", "service": "g1t-actions" },
2626 // Builds of deployments report through here.
2727 { "binding": "DEPLOYMENTS", "service": "g1t-deployments" },
28+ // Observability: a project's requests, logs, traces, errors and
29+ // metrics, and OTLP sent to /otlp/v1/{traces,logs,metrics}.
30+ { "binding": "TELEMETRY", "service": "g1t-telemetry" },
2831 // The context hub: search_context and get_entity.
2932 { "binding": "CONTEXT", "service": "g1t-context" },
3033 { "binding": "SEARCH", "service": "g1t-search" },
+1−0
111111 { label: 'Projects', slug: 'guides/projects' },
112112 { label: 'Deployments', slug: 'guides/deployments' },
113113 { label: 'Deployments API', slug: 'guides/deployments-api' },
114+ { label: 'Observability', slug: 'guides/observability' },
114115 { label: 'Packages', slug: 'guides/packages' },
115116 { label: 'Container images', slug: 'guides/containers' },
116117 { label: 'npm', slug: 'guides/npm' },
+11−0
633633 | Idle days | 7 | 1 to 90. A preview no one visits this long comes down. |
634634 | Build with a workflow | Off | Build through the deploy workflow g1t manages for the project; see [build with a workflow](#build-with-a-workflow). |
635635
636+## Logs, traces and errors
637+
638+Every request an app answers shows under the project's
639+[Observability](/guides/observability/) within seconds: its status,
640+duration, CPU time and the lines it wrote, with its exceptions grouped into
641+errors and its numbers by deployment. Nothing to set up. Each request
642+reaches the app with an `x-g1t-request-id` header, its id there, and a
643+`traceparent`, and the app is deployed with OpenTelemetry's
644+`OTEL_EXPORTER_OTLP_*` variables set, so an OpenTelemetry SDK in it sends
645+its spans to the project without configuration.
646+
636647 ## Secrets and variables
637648
638649 Builds and running apps read the project's
+280−0
Loading the diff…
+1−0
Loading the diff…
+1−0
Loading the diff…
+2−1
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…
Loading the diff…