Skip to content

test: run the test suite on real browsers with Vitest browser mode - #224

Merged
1natsu172 merged 5 commits into
masterfrom
test/vitest-browser-mode
Oct 3, 2026
Merged

1natsu172 merged 5 commits into
masterfrom
test/vitest-browser-mode

Conversation

@1natsu172

@1natsu172 1natsu172 commented Oct 3, 2026 •

Copy link
Copy Markdown
Owner

Closes #210

Summary

  • Run every test file on Chromium, Firefox and WebKit with Vitest browser mode and the Playwright provider, replacing jsdom. The library depends on how real engines deliver MutationObserver records and run microtask checkpoints, which jsdom only approximates.
  • Rewrite the customMatcher XPath test so it actually checks detection of an element added later, which jsdom could not exercise (the old FIXME).
  • Install the Playwright browsers in CI, and update the Testing section of AGENTS.md.

Background

Findings checked on all three engines with throwaway probe tests:

  • Document sharing: browser mode runs each test file in its own iframe, so DOM nodes and globals do not leak across files. Tests in the same file still share one document, so the per-test test-sandbox element is still needed (#late is used by several tests in src/index.test.ts). The comments now give this reason instead of the jsdom issue.
  • node:timers/promises: importing it fails in the browser (Module "node:timers/promises" has been externalized for browser compatibility), so the tests use a local delay helper.
  • Coverage: the v8 provider refuses to start with several browser instances, so pnpm coverage runs on Chromium only (--project chromium).

Notes

  • Type tests (*.test-d.ts) still run and pass, but they now run once per browser instance, so tsc runs three times.
  • jsdom is no longer a direct dependency, but pnpm-lock.yaml still resolves it as the optional jsdom peer of vitest, so pnpm install keeps installing it although no test uses it. pnpm install --fix-lockfile and pnpm update vitest @vitest/* keep the entry; only a lockfile generated from scratch drops it. Regenerating the whole lockfile would move unrelated versions, so it is left as is here; the next Renovate lock file maintenance should drop it.
  • The browsers are not cached in CI. In the first run, installing them took 64s: about 49s for the system packages, which a cache cannot hold, and about 15s for the download. A cache hit would save only the download while restoring a 494MB entry, so the workflow installs them on every run.

Test plan

  • pnpm test-ci passes on Chromium, Firefox and WebKit (99 tests, no type errors)
  • Stable under describe.shuffle: 5 runs in a row with different seeds, all passing
  • MutationObserver spy tests give the same observe/disconnect counts and order on all three engines
  • pnpm vitest run --coverage --project chromium produces a v8 report
  • biome ci, typecheck and build pass
  • The CI test job passes on all three engines (15 files, 99 tests, no type errors; same counts as locally)
  • The CI test job passes after removing the Playwright cache

🤖 Generated with Claude Code

1natsu172 and others added 5 commits October 3, 2026 22:38
jsdom only approximates how real engines deliver MutationObserver
records and run microtask checkpoints, which this library depends on.
Run every test file on Chromium, Firefox and WebKit through the
Playwright provider instead, and drop jsdom.

- Replace node:timers/promises with a local delay helper, since Vite
  externalizes node:* modules in the browser and the import fails.
- Keep the per-test sandbox element: browser mode runs each test file
  in its own iframe, but the tests in a file still share one document.
- Limit pnpm coverage to Chromium, because the v8 provider refuses to
  start with several browser instances.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The XPath test appended the element before calling waitElement, because
jsdom could not exercise dynamic detection, and the FIXME callback in
Promise.all was never invoked. Start waiting first, append the element
afterwards, and assert that the initial check finds nothing while a
later observer-driven check returns the element.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cache the browser binaries keyed by the Playwright version. The system
packages the browsers need are not part of the cache, so install them
on every run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first run spent about 15s downloading the browsers and about 49s
installing their system packages, which the cache cannot hold. A hit
would save only the download, at the cost of restoring a 494MB entry
that also counts against the repository cache quota.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@1natsu172
1natsu172 marked this pull request as ready for review October 3, 2026 14:54
@1natsu172
1natsu172 merged commit d5148ac into master Oct 3, 2026
4 checks passed
@1natsu172
1natsu172 deleted the test/vitest-browser-mode branch October 3, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: run the test suite on real browsers with Vitest browser mode

1 participant