Repository navigation
feat(slo): multiwindow burn-rate alerts on a service's AWS bill - #17
Merged
Merged
Conversation
cost.libsonnet: three tiers (3x/24h+6h P2, 1.5x/72h+12h P3, 1x/7d+24h P3) against a monthly dollar budget, over an hourly-cost CloudWatch metric, with every window ending 14h back because Cost Explorer reports late. Plus a P3 guard when the metric stops arriving. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
chris13524
marked this pull request as ready for review
October 7, 2026 14:19
Hourly cost with a line at each tier's pace, plus month-to-date and trailing 30-day spend as gauges against the budget, from the same budget and tiers the rules use. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
geekbrother
approved these changes
Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
slo/cost.libsonnet: multiwindow, multi-burn-rate alerting on a service's whole AWS bill against a monthly budget in dollars. It's the SLO burn-rate shape applied to spend.It watches the account total rather than one usage line, because a regression shows up on whichever line it shows up on. In August–September that was AMP query samples on pay-core and blockchain-api, which ran unnoticed for two months (#16). Cost Explorer shows which line moved once an alert fires.
The SLO multiples (14.4×/6×) are for outages. pay-core's AMP regression reached 1.8×, then 3.3×, of a sensible budget, and would have tripped neither. Nothing here pages.
Input and constraints
terraform/cost-reporter, in the companion PR). The poller re-writes recent hours as Cost Explorer revises them, so every query reads each hour's Maximum;Sumwould count each revision.metricQueryType, as pay-core'salerts.jsonnetpatch now does.Dashboard panels
From the same budget and tiers as the rules:
hourlyPanel: hourly cost, with a line at each tier's hourly pace (1×, 1.5×, 3× of budget/720h).monthPanel: month-to-date spend as a gauge against the budget.trailingPanel: trailing 30-day spend as a gauge against the budget.There's deliberately no rolling burn-multiple chart. The input must be read at each hour's Maximum, CloudWatch metric math has no moving sum, and Grafana's window transforms aren't available on every instance these dashboards render on.
Tests
tests/cost_smoke.jsonnet(new CI step) pins the allowances, windows, query shape, priorities and the no-data split. promtool can't evaluate CloudWatch queries. Mutation-checked: changing a tier multiple, the settle offset, or the statistic each fails the smoke test.Consumers: WalletConnect/pay-core#1909, reown-com/blockchain-api#1528.
🤖 Generated with Claude Code