CI add visual regression tests

[kroq-gar78]

Rendering changes in Pycortex are currently only caught by eye. This PR automatically compares renders from both renderers (quickflat and webgl) against stored reference images, so a change in output fails a test instead of needing to be spotted in a screenshot. It also checks for consistency between the two renderers.

Tests

Four groups of reference images, covering the six public dataview classes through both renderers. (webgl references for Vertex2D are not included because it's broken on main, per gh-714.)

group	num. images	what it covers
alpha_dataviews/	10	every way pycortex encodes alpha: Volume/Vertex as a no-alpha baseline, Volume2D via the 2D alpha colormap RdBu_r_alpha, the RGB classes via the alpha= keyword. All composite the curvature underlay.
nan_dataviews/	10	the same six (Vertex2D xfailed) with NaNs over roughly half the primary data channel. Both renderers should draw those elements fully transparent, falling through to curvature, rather than mapping NaN through the colormap.
nan_alpha_dataviews/	4	VolumeRGB/VertexRGB with the NaNs in the alpha map instead. A separate path: alpha is not colour-mapped but used directly as a blend weight, so the NaN reaches the compositing arithmetic.
nonflat_views/	4	Volume and Vertex on the inflated and fiducial surfaces. webgl only — quickshow renders flatmaps and nothing else — so these get the reference check and no cross-renderer leg.

The images are tracked with git LFS and should be regenerated whenever a rendering change is blessed.

The within-renderer checks run at tighter tolerances than the cross-renderer checks, since the two renderers differ by a fixed affine transform. (fit_cross_renderer_affine.py re-derives the ideal transform.)

WebGL errors now fail loudly. Previously, WebGL console messages were not being caught properly due to buffering. This PR correctly propagates console error into pycortex. (This change would have caught gh-714.)

A blank render now fails loudly. save_3d_views also now raises on a blank view, naming the view and the file.

What it catches

Three real bugs:

• FIX NaN values in Volume/Vertex rendering as black instead of transparent #612, NaN vertices rendered opaque instead of transparent — a merged fix, reproduced here by deleting the one shader line that implements it. Fails all four criteria.
• FIX unify NaN and alpha handling across quickflat, WebGL and RGB dataviews #695, the open PR that flips quickflat's nanmean default. The mean difference (0.21 against a limit of 2.0) and the fraction of pixels moving more than 16 (0.68% against 2%) both pass. Only the gross-difference fraction and SSIM catch it — which is why there are four criteria and not one.
• Vertex2D objects do not render in webviewer #714, the Vertex2D flatmap shader failing to link. Live on main: the canvas comes back blank, nothing raises, and the png is still written.

Notes for reviewers

• On failure the tests write the actual render and an amplified difference into the test's tmp_path. There is no side-by-side composite; the two files are easier to flip between than one wide image.
• git lfs pull if the reference images arrive as 130-byte text stubs. The tests skip rather than fail on that, and on the images being absent.
• The two Vertex2D cases are xfail(strict=True). When the fix (Vertex2D objects do not render in webviewer #714) merges they turn XPASS.
• matplotlib and playwright are now pinned in dev environments for consistency with the reference images.
• CI fetches LFS only where the images are read. install_from_wheel.yml deliberately does not, since wheels exclude them.
• cortex/tests/reference_images/README.md covers regeneration and provenance.

[mvdoc]

@claude review

[claude]

Claude finished @mvdoc's task in 3m 11s —— View job

---

Review of "CI add visual regression tests"

Todo list:

• Gather context (PR body, changed files list)
• Fetch branch / diff against origin/main (repo is a shallow clone; diffed against the PR's own base commit 3779f7c, which matches the stated +1526/-237)
• Read CLAUDE.md for repo conventions (none present in this repo)
• Review core new test infra (test_visual_regression.py, testing_utils.py)
• Review headless.py / save_views.py changes (WebGL error propagation, blank-render detection)
• Review CI workflow changes and .gitattributes/LFS setup
• Review test_export.py / test_webgl_headless.py changes
• Post review feedback

---

This is a very well-engineered PR — the calibration work behind the tolerances (test_visual_regression.py:114-233) and the affine-fit derivation (fit_cross_renderer_affine.py) is unusually rigorous, and the README covers regeneration/provenance clearly. A few small things worth a look:

1. Stale doc comment: handle is a JSMixer, not a WebApp
cortex/export/save_views.py:202:

pw_thread = handle._pw_thread # `handle` is a `WebApp`

Per headless_viewer's own docstring (cortex/export/headless.py:339-341), handle is the JSMixer; handle.server is the WebApp. Minor, but worth fixing so it doesn't mislead a future reader.

2. Typo: cortex/export/save_views.py:201 — "we don't capture console output in the interactivemode" → "interactive mode".

3. browser_errors accumulates for the lifetime of the session, never cleared per-view
In both save_views.py:205 (save_3d_views, one raise ends the loop, so this is harmless there) and test_webgl_headless.py's _assert_no_browser_failures, the check reads the entire history of console errors seen since the viewer was created. For TestAddData (test_webgl_headless.py:783-868), the handle is shared class-wide across several sub-tests that each call _assert_no_browser_failures. If an early sub-test in that sequence trips a spurious/transient WebGL warning, every later sub-test in the class will also fail on the same stale message, rather than the failure being isolated to where it happened. Not a false negative (nothing is missed), but it can produce a confusing cascade of failures pointing at the wrong test. Worth a # known limitation note, or clearing/snapshotting the error list at test boundaries, if this comes up in practice.

4. Minor style: cortex/tests/test_export.py:85-86 — only one blank line between test_plot_panels_headless and the new test_filter_webgl_failures_keeps_only_real_failures (PEP8 wants two between top-level defs). Cosmetic only.

Things that look solid:

• filter_webgl_failures's narrow pattern list (cortex/export/headless.py:117-120) is well justified and has a dedicated unit test (test_export.py) that pins down both the false-positive noise (Leap Motion websocket, driver warnings) and the true positives (link failure, context-creation failure).
• The four-criteria image comparison (MAX_MEAN_ABS_DIFF/MAX_FRACTION_DIFFERING/GROSS_DIFF_THRESHOLD/MAX_SSIM_LOSS) is justified with real worked examples (FIX NaN values in Volume/Vertex rendering as black instead of transparent #612, FIX unify NaN and alpha handling across quickflat, WebGL and RGB dataviews #695, NF: refactor webgl lighting #679) rather than arbitrary numbers — good practice for a visual-regression suite, which is otherwise prone to bit-rot tolerances nobody can justify later.
• The LFS/wheel/sdist plumbing (.gitattributes, MANIFEST.in reliance, setup.py's exclude_package_data, the three workflow diffs) is internally consistent — I traced through each one and they agree with each other and with the README's "Distribution" section.
• xfail(strict=True) on the two Vertex2D cases is the right call so the fix for Vertex2D objects do not render in webviewer #714 doesn't pass silently as a false negative.

No blocking issues found; the three code notes above (1-3) are polish, not correctness bugs.

[kroq-gar78]

I pushed new commits that address problem 1, 2, and 4. I think it's not worth fixing problem 3, but I added a note for it.