feat(voice): ordered TTS provider failover chain with per-rung liveness probe - #1866
Open
0bsolescence wants to merge 6 commits into
Open
feat(voice): ordered TTS provider failover chain with per-rung liveness probe#18660bsolescence wants to merge 6 commits into
0bsolescence wants to merge 6 commits into
Conversation
added 6 commits
August 15, 2026 11:43
Voice has exactly one path to speech. If ElevenLabs is unreachable, rate limited, or the account lapses, every spoken notification is simply lost, and an install with no ElevenLabs account cannot speak at all. [voice].providers, when present, is an ordered chain. The first provider that returns audio speaks; any failure (non-2xx, timeout, connection refused) falls through to the next; an exhausted chain logs and stays silent rather than throwing into the notify path. Absent or empty, nothing changes: the legacy ElevenLabs branch is untouched and still gated on the API key, which is what every existing install runs. - VoiceServer/providers.ts: new. Config normalization, both provider clients, and the chain runner. Zero external deps and no import of ../lib, so it is testable before 'bun install' has ever run. - voice.ts: pronunciation preprocessing split out of generateSpeech so a chain applies it once rather than once per attempt; playAudio takes the container format, because the Linux players pick a demuxer off the file extension and a wav body in a .mp3 file plays as noise. - The openai-compatible client sends stream: false. Kokoro-FastAPI defaults it to true, and a streamed reply commits to 200 before generation finishes, so a model that dies mid-sentence would arrive as truncated audio that the chain reads as success. Verified against schemas.py in v0.7.2. - ElevenLabs keeps no request timeout on the legacy path, where it never had one. Inside a chain every hop is bounded, or the fallback never gets a turn. Tests at PULSE/test/ per the doctrine's subsystem-package clause, driving both the chain library and the real /notify contract against stub servers on ephemeral ports. 44 pass, no network.
Refines the chain against the Kokoro-FastAPI source rather than against the
OpenAI shape it resembles.
- Liveness per rung, ahead of generation. openai-compatible gets a cheap GET
of the server root's /health on a 2s budget, where ANY HTTP response counts
as reachable — non-Kokoro servers have no /health, and a 404 still proves
the socket answered. Connection refused and timeout are the hard failures.
Without this a dead first provider burns its whole generation budget on
every notification instead of being stepped over. ElevenLabs is not probed
over the network: its liveness is whether a key is configured, which is the
same test the pre-chain path already made, so no new failure mode.
/health at the app root and /v1/models on the openai router were both
confirmed in api/src/main.py before picking the default.
- Generation timeout drops 15s -> 10s and health gets its own 2s, both
per-provider configurable. Kokoro on CPU runs ~1.3-2x realtime with ~3.5s to
first audio on older x86_64, so a one-line notification genuinely takes
seconds; 10s covers that without letting a wedged box hold the chain.
- speed is always sent, defaulting to 1.0, rather than omitted when unset.
- The container format now comes from the response Content-Type when it is
recognised, falling back to the requested response_format. A server that
ignores an unsupported format and sends mp3 anyway would otherwise have its
bytes written to a .flac temp file and played as noise.
- A JSON body on a 200 is now a failure. These servers return audio bytes or
nothing, so JSON is an error payload wearing a success code, and the chain
should fail over rather than play it.
- voice is never charset-validated. Kokoro carries blends in that same field
("am_fenrir(2)+am_michael(1)"), so any validation would reject valid config.
Backwards compatibility is unchanged: no providers array configured is still
the untouched ElevenLabs path, still gated on the API key, and PULSE.toml
still parses to exactly {enabled = true}.
61 pass, no network.
Review nits from the chain review: pcm synthesizes fine but raw headerless samples can't be demuxed from a temp-file extension, so stock players fail. The health string change (voice_system: "chain" when a chain is configured) is deliberate and gated — noted here so the PR body calls it out for dashboard consumers.
…box holds The homedir() refactor reads the OS home directly under Bun, defeating the test suite's HOME-redirection sandbox and letting the suite read the operator's real settings.json. $HOME wins when set; homedir() remains the fallback.
…oints Cross-vendor audit flagged that the unconditional stream:false field can be rejected as unknown by a strict OpenAI-compatible server, failing every request for that provider. New send_stream_flag config (default true, the Kokoro-safe behaviour that keeps failures in the status code) omits the field entirely when set false. Documented in PULSE.toml; regression test asserts both the default-sends and the opt-out-omits paths.
Cross-vendor re-audit: once the chain can return wav/flac/opus/aac, the cached single global player breaks non-mp3 playback — mpg123 (MPEG-only) would be chosen for a wav body even with aplay installed. Player resolution now filters by format: ffplay (all formats) stays first, mpg123→mp3, paplay→wav/flac, aplay→wav. selectPlayer() is a pure, unit-tested seam; the PATH probe is cached once and selection runs per playback. The no-player warning now names the format, the present players, and recommends installing ffplay for full coverage. ffplay cannot be bundled, only preferred.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An ordered provider failover chain for Pulse voice:
[[voice.providers]]entries in PULSE.toml are tried in order (e.g. a local OpenAI-compatible endpoint such as Kokoro first, ElevenLabs as fallback), with a per-rung liveness probe and a verified request contract so a dead local server degrades to the next provider instead of silence. Existing single-provider ElevenLabs configs work unchanged.Commits
feat(voice): ordered provider failover chain (providers.ts + chain wiring in voice.ts)feat(voice): per-rung liveness probe, verified request contractdocs(voice): pcm playback caveat in the format map and PULSE.tomltest(voice): honour $HOME override over homedir() so the suite's HOME sandbox holds under Bun (base-adaptation; without it the suite reads the operator's real settings.json)Testing
bun test test/VoiceServer/— 61 pass, 0 fail (providers.test.ts + voice.test.ts), run against currentmain.Developed with AI assistance (Claude); reviewed and tested by me.