[i18n] Polish eval set: frozen intents.pl.jsonl + speaker-holdout gap (issue #79 remainder)

[grzanka]

Background

Spun out of #79 ("Define sentence/speaker counts for the Polish eval set"), which proposed three
independently-scoped tracks. Two of the three are done — Track 1 (human recordings,
eval/RECORDING.pl.md, issue #84) and Track 3 (TTS-generated batch, PR #97, later exceeded by
#106/PR #105's Chatterbox batches). This issue tracks the two genuinely open items left over.

1. Track 2 — the frozen Polish text eval set (eval/intents.pl.jsonl)

Not started — the file doesn't exist yet. Per #79's original proposal:

• Create eval/intents.pl.jsonl, seeded from the ~50 sentences in eval/RECORDING.pl.md
  (already hand-labeled as part of Track 1) rather than pre-writing a large set up front.
• Extend scripts/validate-intents.ts / pnpm run validate:eval to take a language argument
  and validate both files.
• Extend scripts/coverage-intents.ts the same way.
• Start with a much lower MIN_EXAMPLES floor than English's 100 (e.g. 20–30) and raise it as
  the set grows organically — don't copy the English floor verbatim on day one.
• src/lib/intent/lang/pl.ts (landed via [nlu][asr] Redesign the Whisper-correction and NLU-matcher layers for maintainability + multi-language (Polish) #87) is the matcher this validates against, so the
  main prerequisite [i18n] Define sentence/speaker counts for the Polish eval set (recorded vs. generated) #79 flagged for this track is already satisfied.

This is the actual missing deliverable from #79 — the PL equivalent of eval/intents.jsonl,
CI-gated the way the English set already is.

2. Track 1's unmet speaker-count gap

eval/RECORDING.pl.md's 50 sentences exist as audio in eval/audio/lg/, but — confirmed via
docs/tts-chatterbox-pl-clone.md:26 — all 50 clips are one speaker (the project owner). #84
asked for 2–3 volunteers; only the bootstrap speaker actually delivered.

This is the same shape of gap #20 already tracks for the English set (a tuning-set vs. holdout-set
distinction — rules can't be validated as generalizing past one speaker). Not blocking Track 2
above, and not urgent on its own — recording is human-dependent, not code-dependent, same as #20's
current status. Listed here so it isn't lost, not as a call to action:

• 1–2 additional Polish speakers record the same 50 sentences (eval/RECORDING.pl.md),
  recordings never used for rule tuning — a genuine holdout, mirroring [local] Multi-speaker ASR recordings — separate systematic from speaker-specific Whisper errors #20's ask for English.

Related

• [i18n] Define sentence/speaker counts for the Polish eval set (recorded vs. generated) #79 — parent issue (closed; this is its two remaining open items)
• [i18n] Record the 50-sentence Polish ASR eval set (eval/RECORDING.pl.md) #84 — Track 1 delivery (single-speaker bootstrap)
• [eval] Polish 1000-sentence TTS batch via Piper + experimental Qwen3-TTS (issue #79 Track 3) #97 / [eval] Explore Chatterbox TTS for Polish voice cloning / native synthesis (issue #79 follow-up) #106 — Track 3 delivery (TTS batches)
• [nlu][asr] Redesign the Whisper-correction and NLU-matcher layers for maintainability + multi-language (Polish) #87 — Polish matcher pack (lang/pl.ts), the prerequisite this issue's Track 2 validates against
• [local] Multi-speaker ASR recordings — separate systematic from speaker-specific Whisper errors #20 — the equivalent single-speaker-holdout gap for the English set