Repository navigation
Enforce wall-clock budgets per test layer #12877
Description
Part of #12876.
Problem
Nothing limits how long tests run. The lifecycle step grew to 684 s on linux_arm64 and 747 s on linux_amd64, and its slowest test, deployment::web_search_cli_export_reapply_and_destroy, takes 241 s. A slower test passes CI as long as it passes.
Scope
- Record each layer's budget and per-test limit in one checked-in file.
cargo cireads the nextest JUnit report of each profile, compares wall time with the layer budget and each test with its limit, and fails naming every offender with its time and limit.- Nextest profiles stop a test at its limit with
slow-timeoutandterminate-after. - Budgets apply on the Linux runners; macOS and Windows report their times without failing.
- Start with budgets that pass today, then lower them as Test in three layers with wall-clock budgets #12876's other sub-issues land.
Done when
A test that exceeds its limit, or a layer that exceeds its budget, fails CI with its time and budget, and the testing guide names the budgets.
Activity
test: split tests into layers with wall-clock budgets (#12885)
ci: fail a test step that exceeds its wall-clock budget (#12938)
ci: report each platform's whole lifecycle suite and budget the live …
cv commentedon Oct 10, 2026
Status on
v1:.config/test-budgets.yaml. On the Linux runners, a step over its budget, or with a test over its limit, fails and names each offender with its time and limit.docs/contributing/testing.mdlists the budgets.Still open: #12969 lowers the lifecycle budget to 300 s per partition and 90 s per test. It closes this issue.
ci: lower the lifecycle budget to its measured times (#12969)
cv commentedon Oct 11, 2026
Done on
v1:.config/test-budgets.yaml, enforced on the Linux runners.Stopping a test at its limit inside nextest moves to #12979.
v1is not the default branch, so the closing keywords did not apply.