Warm up container before parallel test dispatch - #10204
Merged
Alexander Holstrup (aholstrup1) merged 4 commits intoAug 19, 2026
Merged
Conversation
On a fresh container the first test run intermittently fails with a masked client 500 on page 130455. The real root cause is a platform GDI+/System.Drawing first-touch race: the image-encoder registry initializes lazily once per NST process and is not thread-safe, so concurrent per-tenant company-opens (media import during InitCompany) hit ArgumentNullException('encoder') and terminate the session.
A fixed 5s inter-dispatch sleep already exists and is insufficient (it staggers dispatch start, not the async GDI+ moment). Instead, run the first app alone and await its completion so GDI+ warms single-threaded before the parallel fan-out. Complementary to the existing transient-race one-retry path, which is unchanged. No-op for single-app/single-tenant runs.
Pipeline-level workaround; remove once the platform guards its first-touch GDI+ init.
Alexander Holstrup (aholstrup1)
temporarily deployed
to
triage
August 12, 2026 13:04 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
github-actions
Bot
added
Build: scripts & configs
… app Shorten the warmup function/description comments, rename Invoke-GdiPlusWarmupDispatch to Invoke-WarmupDispatch, and reorder each legacy bucket so the smallest app runs first. Since the first app now runs serially before the parallel fan-out, leading with the smallest keeps that serial step cheap while the remaining apps stay longest-first for LPT scheduling.
Alexander Holstrup (aholstrup1)
changed the title
Alexander Holstrup (aholstrup1)
marked this pull request as ready for review
August 13, 2026 11:21
Alexander Holstrup (aholstrup1)
temporarily deployed
to
triage
August 13, 2026 11:21 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Alexander Holstrup (aholstrup1)
enabled auto-merge
August 14, 2026 11:17
Timing analysis of the PR build showed Tests-Local runs ~17-19 min, so using it as the serial warmup app added that much to Bucket1's critical path. Lead with Tests-Resource (~3 min) instead and move Tests-Local back into its longest-first position. Bucket2's warmup (Tests-Upgrade, ~1.6 min) was already small.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Add unit tests for the remaining-apps return value, the unresolved-appId skip branch, and the transient warmup failure landing in State.transient for the caller to re-queue.
spetersenms
approved these changes
Aug 17, 2026
Alexander Holstrup (aholstrup1)
requested a review
from Maria Zhelezova (mazhelez)
August 18, 2026 09:32
ankosiorek
approved these changes
Aug 19, 2026
Alexander Holstrup (aholstrup1)
deleted the
aholstrup1-gdiplus-warmup-mitigation
branch
August 19, 2026 08:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The parallel test dispatcher fans out one background job per app/tenant. Concurrent per-tenant company-opens on a fresh container can race on the first use of the container's process-wide state, causing the first test run to fail intermittently (a masked 500 opening page 130455). This runs the first test app alone and awaits it before the parallel fan-out, so that first open happens single-threaded and the rest fan out safely. The change reuses the existing dispatch/wait machinery, is a no-op for single-app or single-tenant runs, and leaves the existing transient-retry path in place.
TestConfiguration.jsonnow lists the smallest app first in each bucket so the serial step is cheap, with the remainder still longest-first for scheduling.Linked work
AB#646706
How I validated this
What I tested and the outcome
Unit tests:
build/scripts/tests/ParallelTestExecution.Test.ps1passes 10/10, including new tests that the first app is dispatched and awaited alone before fan-out, and that the step is skipped for single-tenant/single-app runs.End-to-end (this PR's CI run): all 113 Unit Tests container jobs succeeded, the warmup ran in every one, and none showed the failure signature or triggered the transient-race retry.
Risk & compatibility
Low. Only the first dispatch is reordered (one app runs alone, then the rest fan out as before); retry classification, result merging, and tenant/company provisioning are unchanged. Trade-off: the serial app adds its duration to the critical path, which is why each bucket now leads with its smallest app.