Skip to main content

Design: Prometheus Metrics

Two collection styles, chosen per metric​

Counters inline. Creation, expiry deletions and HTTP outcomes increment where they happen.

Gauges at scrape time. cairn_artifacts_total, cairn_storage_bytes and cairn_artifacts_past_ttl come from aggregate queries run by a custom collector:

SELECT share_type, count(*), coalesce(sum(size_bytes),0) FROM artifacts GROUP BY share_type;
SELECT count(*) FROM artifacts WHERE expires_at < now();

Maintaining these inline would require every delete path, TTL edit and failed upload to adjust a gauge correctly forever. One missed adjustment yields a permanently wrong number that still looks plausible — and for artifacts_past_ttl specifically, drifting toward zero would mean silently reporting health.

The past-TTL gauge is the point​

expired_total and past_ttl answer different questions and only the second is load-bearing:

observationsweeper healthy, nothing to dosweeper broken
expired_total rate00
past_ttl0climbing

A success counter cannot separate these. That is why the spec requires the gauge be omitted on collector failure rather than zeroed: a zero here is the healthy reading, so a blind collector reporting zero would assert health at the moment it stopped being able to see.

Route classification​

A hand-maintained map from handler to route_class, applied in middleware. Deliberately not derived from the path: artifact handles are in paths, so a derived label would both explode cardinality and publish handles to anyone who can read the endpoint.

New routes default to other. cairn_http_requests_total{route_class="other"} rising is the signal that the map needs updating — the classifier's own control, without which it decays silently into one bucket.

Storage bytes​

Summed from the size recorded on each artifact row, not by asking the object store. The database is authoritative for what Cairn believes it has stored; divergence from the bucket's own accounting is a real finding and the two should be measured independently rather than one derived from the other.

Auth​

Handled ahead of artifact-scoped authorization, against the operator credential only. An artifact token grants access to artifacts, not to aggregate counts of everyone's.

401 with an empty body: share types and counts describe usage patterns, and an unauthenticated prober should learn nothing, including whether the endpoint is populated.

Testing​

  • A broken-sweeper test: expiry fails, past_ttl climbs, expiry_run_timestamp stops advancing while expired_total stays flat — the three-signal shape from the spec's first scenario.
  • A test that past_ttl is omitted and the error counter increments when the collector fails — explicitly asserting it is not zero, since zero means healthy.
  • A test that expiry_run_timestamp does not advance on a failed sweep.
  • A test that it is omitted before any successful sweep, rather than 1970.
  • A cardinality test: an artifact whose handle appears in the request path produces no label containing it.
  • An auth test: no credential 401, artifact token 401, operator 200.