Speech to text and text to speech

What you’ll learn

The local models for turning speech into text and text into speech, the tools that run them efficiently, and how to choose between them, including the licensing points that matter for voices.

Speech to text

For transcription, Whisper large-v3 is the de facto standard: robust across many languages and accents, and MIT licensed. The model itself is one thing, and the runtime is another. Running it through whisper.cpp or a similar efficient runtime is what makes transcription practical on ordinary hardware, including CPU-only machines.

A couple of things to know:

• Whisper processes audio in 30-second windows, so very long recordings are chunked.
• It can hallucinate text during long silences or noisy passages, so transcripts of imperfect audio need a review pass.
• If you need real-time or faster-than-real-time transcription, use a smaller or distilled Whisper variant, which trades a little accuracy for speed.

Text to speech

For generating speech, the choice depends on what you need:

• Kokoro 82M is a tiny, Apache 2.0 model that produces natural narration and runs fast even on a CPU. It is the sensible default for clean narration, with a set of built-in voices.
• Piper is a fast, lightweight engine that runs offline on very modest hardware, and is the standard voice in Home Assistant setups. Note that its original repository was archived, with development moving to a newer one.
• Chatterbox is the pick when you need to clone a specific voice from a short sample. It is MIT licensed, unlike several cloning models that restrict commercial use, but it is heavier and wants a GPU.

A note on voice cloning

Cloning a voice raises consent and ethics questions that transcription and plain narration do not. Only clone voices you have permission to use, and be aware that some voice models licence their weights for non-commercial use only, so check before you build on one.

What can go wrong

• Trusting a transcript of poor audio. Whisper is strong, but noisy input produces errors and the occasional hallucination. Review important transcripts.
• Reaching for a heavy TTS model when a light one would do. For narration, Kokoro or Piper are fast and clean. Save the heavier cloning models for when you actually need a specific voice.
• Overlooking the voice licence. Some voices and cloning models are non-commercial. Check before shipping.

Next steps

The speech models and voice and audio tools in the catalogue list the full set. A common project is to combine transcription, a local language model, and speech output into a voice assistant that runs entirely on your own machine.