test(e2e): upgrade puppeteer to v25 to fix broken e2e navigation#666
Merged
Conversation
The puppeteer-backed e2e cases intermittently timed out, with every puppeteer test in a run hanging for the full 30s mocha timeout (e.g. run 26625946119 took 29m as ~50 tests each burned 30s). Two root causes: - The tests opted into `headless: 'new'`, which in the bundled Chromium (pptr 19 / Chromium ~115) is markedly slower and flakier to start in CI than the legacy `headless: true` mode, for no functional gain here. - The launch retry only caught thrown errors, not hangs. puppeteer's default launch/navigation timeouts equal the mocha timeout, so a stuck launch or navigation killed the test before the retry could help. Switch to legacy headless, and retry the whole launch -> navigate -> read cycle with a fresh browser, bounding each step with an explicit timeout so a hang surfaces as a catchable rejection. Puppeteer test timeout is raised to fit the retry attempts plus backoff.
github-actions
Bot
added
t-tooling
Member
Author
|
@bliuchak I'd reconsider running a CI check that takes ~30 minutes, we usually run e2e tests on master on a schedule once a day, it makes little to no sense to block PRs with this. |
The puppeteer e2e tests started failing wholesale (all 52 cases timing out on navigation) after the GitHub-hosted runner image bumped from ubuntu24/20260518 to ubuntu24/20260525. puppeteer was pinned at 19.11.1, whose bundled Chromium (~115, mid-2023) launches but can no longer navigate on the newer OS, while curl through the same proxy still works. Upgrade puppeteer to v25, which ships a current Chromium built for the new image. Adjust the two renamed launch options: - `ignoreHTTPSErrors` -> `acceptInsecureCerts` - `headless: 'new'` -> `headless: true` (the `'new'` value was removed; the modern headless mode is now the default `true`). This also reverts the earlier launch/navigate retry + per-step timeout experiment: it was based on a wrong root cause (assumed flaky spawns), didn't help, and tripled the failing run's wall-clock time by retrying a browser whose navigation could never succeed. Verified locally with puppeteer 25 that the new launch options launch and navigate, both directly and through an HTTP proxy.
B4nan
changed the title
Member
Author
|
Huh, so this was all about a dated puppeteer, this PR bumps it to resolve the flakiness and it now runs in ~3 minutes, so let's leave it in PR checks, that's completely reasonable. |
bliuchak
approved these changes
May 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
All 52 puppeteer-backed e2e cases started failing wholesale — every one timing out on navigation, ballooning the
E2E tests (Node.js 24)job from ~2 min to ~57 min (run 26636921133).Root cause is dependency rot triggered by a runner-image bump, not flakiness:
ubuntu24/20260518.149; the failing runs useubuntu24/20260525.161.19.11.1, whose bundled Chromium (~115, mid-2023) launches but can no longer navigate on the newer OS.curlthrough the same proxy still passes, confirming the proxy itself is fine and the problem is the stale browser.Fix
Upgrade puppeteer
19.11→25.x, which ships a current Chromium built for the new image. Two renamed launch options:ignoreHTTPSErrors→acceptInsecureCertsheadless: 'new'→headless: true(the'new'value was removed; modern headless is the defaulttrue)This also drops an earlier launch/navigate-retry + per-step-timeout experiment that was based on a wrong root cause (assumed flaky spawns) — it didn't help and tripled the failing run's wall-clock time by retrying a browser whose navigation could never succeed.
Verified locally with puppeteer 25 that the new launch options launch and navigate, both directly and through an HTTP proxy.