feat(rrweb-player): add captions and hover note annotations

[Juice10]

Summary

Add captions and timeline notes to rrweb-player using custom events tagged annotation.

record.addCustomEvent('annotation', { kind: 'caption', text: 'Choose a project name.' });
record.addCustomEvent('annotation', { kind: 'caption', action: 'clear' });
record.addCustomEvent('annotation', { kind: 'note', text: 'Saving creates a workspace.' });

A caption stays visible until another caption replaces or clears it. Omitting action means set. Seeking restores the caption for that point in the recording. Captions default to hidden; showCaptions: true enables them, and recordings containing captions show a CC toggle.

Hovering or focusing a note's timeline marker opens its text. Clicking the marker seeks to that timestamp. Clicking or scrolling inside the open text panel leaves playback where it is. Notes do not change captions, and caption events do not add timeline markers.

CustomEventAnnotation is exported from @rrweb/types and re-exported by rrweb-player. Other custom-event tags keep their existing markers.

Implementation

The player listens for custom-event to update captions during playback. It indexes caption actions and timeline markers to restore state on seeks without scanning the recording every frame. Appended events extend the index; out-of-order insertions rebuild it. The parent player updates when caption text changes.

Validation

• All 46 player tests pass, including seeks, packed recordings, same-timestamp appended captions, and focus restoration after dismissing notes.
• Build, typecheck, and changed TypeScript lint pass. Typecheck reports 12 existing warnings.
• Performance tests cover 50,000 existing events on append, 100,000 caption actions on lookup and a single lookup per seek, and unchanged playback ticks.
• Browser checks cover cursor movement, typing, DOM changes, backward seeks, captions, note markers, and scrolling inside open notes.

[changeset-bot]

🦋 Changeset detected

Latest commit: 6468f6c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 22 packages

Name	Type
rrweb-player	Minor
@rrweb/types	Minor
@rrweb/all	Minor
@rrweb/browser-client	Minor
@rrweb/packer	Minor
@rrweb/record	Minor
@rrweb/replay	Minor
rrdom-nodejs	Minor
rrdom	Minor
rrvideo	Minor
rrweb-snapshot	Minor
rrweb	Minor
@rrweb/utils	Minor
@rrweb/web-extension	Minor
@rrweb/rrweb-plugin-canvas-webrtc-record	Minor
@rrweb/rrweb-plugin-canvas-webrtc-replay	Minor
@rrweb/rrweb-plugin-console-record	Minor
@rrweb/rrweb-plugin-console-replay	Minor
@rrweb/rrweb-plugin-network-record	Minor
@rrweb/rrweb-plugin-network-replay	Minor
@rrweb/rrweb-plugin-sequential-id-record	Minor
@rrweb/rrweb-plugin-sequential-id-replay	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
🔒 Security Review	✅ Completed 2026-09-07T13:43:33.221721Z	dd83cfb	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Copilot]

🟡 Changes recommended

The new note marker adds per-marker global keydown listeners and overly broad keydown propagation blocking, which should be adjusted to avoid scalability and keyboard-interaction issues.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adds an “annotation” custom-event convention to rrweb-player, enabling timed captions (stateful set/clear) and point-in-time hover notes (seekable timeline markers), along with public typing and documentation so recording authors can emit these events safely.

Changes:

• Introduces annotation parsing and caption state reconstruction (parseAnnotation, getCaptions, getActiveCaption) plus new unit/integration tests.
• Updates player/controller UI to render captions, add a CC toggle (showCaptions), and render seekable note markers with tooltip behavior.
• Documents the new payload shapes and adds a minor changeset for release.

File summaries

File	Description
packages/rrweb-player/test/annotations.test.ts	Unit tests for annotation parsing and caption timeline state logic.
packages/rrweb-player/test/annotations-player.test.ts	Happy-dom integration tests covering captions, CC toggle, notes, seeking, packing, and dynamic events.
packages/rrweb-player/src/types.ts	Adds showCaptions option to the public player props typing.
packages/rrweb-player/src/Player.svelte	Wires caption state to controller time, renders caption overlay, and passes events/time/caption state into the controller.
packages/rrweb-player/src/main.ts	Re-exports CustomEventAnnotation type from the package entrypoint.
packages/rrweb-player/src/Controller.svelte	Filters annotation events so only notes create timeline markers; adds CC toggle UI and note marker component usage.
packages/rrweb-player/src/components/CustomEventMarker.svelte	New interactive marker component with tooltip + Escape dismissal behavior.
packages/rrweb-player/src/annotations.ts	New annotation parsing and caption state helpers with exported payload union type.
packages/rrweb-player/README.md	Documents showCaptions and the annotation note/caption custom-event usage.
docs/recipes/custom-event.md	Adds a detailed recipe section describing note/caption behavior, payload validation, and TypeScript usage.
.changeset/quiet-notes-play.md	Minor changeset describing the new captions/notes feature set.

Review details

• Files reviewed: 11/11 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Bundle Size Changes

Size change: +83.48 kB (+0.80%) | Total size: 10512.52 kB

rrweb-player 🔍 - 1654.47 kB -> 1737.96 kB (+83.48 kB (+5.05%))

File	Base	PR	Diff
rrweb-player.cjs	473.44 kB	497.25 kB	+23.81 kB (+5.03%) 🔍
rrweb-player.js	473.32 kB	497.13 kB	+23.81 kB (+5.03%) 🔍
rrweb-player.umd.cjs	476.24 kB	500.10 kB	+23.86 kB (+5.01%) 🔍
rrweb-player.umd.min.cjs	221.15 kB	229.02 kB	+7.86 kB (+3.56%)
style.css	5.44 kB	7.52 kB	+2.09 kB (+38.38%) 🔍
style.min.css	4.88 kB	6.93 kB	+2.05 kB (+42.09%) 🔍

[Copilot]

🟡 Changes recommended

The new bind:currentTime coupling makes Player update every animation frame even when captions are disabled, creating a likely performance regression that should be addressed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

packages/rrweb-player/src/Controller.svelte:24

• events is required for customEvents/inactivePeriods derivations (events.length, indexing), but there is no runtime default. If Controller is ever instantiated without passing events (or during a refactor), it will throw at runtime.

Providing a safe default keeps the component more robust without changing current behavior when Player passes the prop.

• Files reviewed: 12/12 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟢 Approval recommended

The implementation is coherent, documented, and thoroughly covered for parsing, playback behavior, accessibility, dynamic events, and performance.

Review details

• Files reviewed: 15/15 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Same-timestamp appended captions can diverge during playback, and dismissing a focused note loses keyboard focus.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 2
• Review effort level: Balanced

[pkg-pr-new]

@rrweb/all

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/all@6468f6c

@rrweb/browser-client

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/browser-client@6468f6c

@rrweb/packer

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/packer@6468f6c

@rrweb/record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/record@6468f6c

@rrweb/replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/replay@6468f6c

rrdom

npm i https://pkg.pr.new/rrweb-io/rrweb/rrdom@6468f6c

rrdom-nodejs

npm i https://pkg.pr.new/rrweb-io/rrweb/rrdom-nodejs@6468f6c

rrvideo

npm i https://pkg.pr.new/rrweb-io/rrweb/rrvideo@6468f6c

rrweb

npm i https://pkg.pr.new/rrweb-io/rrweb@6468f6c

rrweb-player

npm i https://pkg.pr.new/rrweb-io/rrweb/rrweb-player@6468f6c

rrweb-snapshot

npm i https://pkg.pr.new/rrweb-io/rrweb/rrweb-snapshot@6468f6c

@rrweb/types

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/types@6468f6c

@rrweb/utils

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/utils@6468f6c

@rrweb/rrweb-plugin-canvas-webrtc-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-canvas-webrtc-record@6468f6c

@rrweb/rrweb-plugin-canvas-webrtc-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-canvas-webrtc-replay@6468f6c

@rrweb/rrweb-plugin-console-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-console-record@6468f6c

@rrweb/rrweb-plugin-console-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-console-replay@6468f6c

@rrweb/rrweb-plugin-network-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-network-record@6468f6c

@rrweb/rrweb-plugin-network-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-network-replay@6468f6c

@rrweb/rrweb-plugin-sequential-id-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-sequential-id-record@6468f6c

@rrweb/rrweb-plugin-sequential-id-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-sequential-id-replay@6468f6c

commit: 6468f6c

[Copilot]

🟡 Changes recommended

Caption restoration performs redundant logarithmic lookups for every caption crossed during a seek.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟢 Approval recommended

The implementation, public types, documentation, and focused coverage are consistent with the stated behavior.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 0 new
• Review effort level: Balanced