iPhone dictation with an optional downloadable on-device speech model

[AlexChambers]

Problem

The current iPhone dictation quality falls short for my use. I want an alternative speech recognition model that runs entirely on the phone and works through T3 Code's existing microphone button.

Why run it on the phone?

T3 Code can connect to multiple environments. Dictation should work consistently when the user switches between them. Keeping the model and its settings on the phone avoids requiring a transcription installation on every environment or depending on a separate environment to process speech.

The phone would produce editable text locally, then send the reviewed prompt to whichever environment the user is working in. Model installation and selection would belong to that device.

Proposal

Allow users to download and select an optional speech model in the native iPhone app.

• Start with one supported model.
• Show its download size and supported languages before installation.
• Use the installed model when the user selects it for dictation.
• Insert the transcript into the editable draft for review before sending.
• Allow users to delete the model and switch back to Apple transcription.
• Run transcription locally after download, without uploading audio or requiring an API key.

Use an iPhone 16 Pro Max as the initial performance target. This would be a target for the optional model feature, not a new minimum requirement for the whole app. Final device support should follow the measurements.

First milestone: verify performance and accuracy on a real device

Before building model download and management, compare the current Apple transcriber with one candidate on an iPhone 16 Pro Max using the same recordings. Include normal dictated prompts, technical names, pauses, and longer dictation.

Evaluate:

• Transcription accuracy and how much manual correction the draft needs.
• Time from finishing speech to receiving the transcript.
• Cold startup time and peak memory use.
• Battery use and heat during repeated dictation.

WhisperKit and FluidAudio are possible engines to evaluate, not settled implementation choices. No candidate has been benchmarked for this proposal, and better accuracy is not yet established. The model choice should follow the results; a larger download alone does not establish better dictation quality.

Initial scope

One downloadable model, local transcription through the existing microphone flow, model deletion, and switching engines. Additional models and platforms can follow after the first implementation is evaluated.

The current code separates recording from transcription, which provides an integration point for another transcriber.

Related proposals

• #20725 proposes a local model on the connected environment. This proposal keeps the model and audio processing on the iPhone so dictation stays consistent across environments.
• #25629 proposes mobile transcription through the environment's Codex account.
• #25797 proposes local Whisper transcription on desktop.
• #12743 tracks Korean recognition problems on iPhone. This proposal does not assume replacing the engine fixes that language-selection issue.

Would an optional downloadable model on the iPhone be in scope, with device benchmarking as the first milestone before committing to an engine or the full download feature?

Drafted with GPT-6-Astra via Codex in T3 Code.