Whisper is one of those releases that quietly defined an entire category. Before September 2022, speech-to-text was a fragmented market — Google Cloud Speech, AWS Transcribe, Otter, Trint, Rev, Otter again, all with different accuracy profiles, different language support, and different pricing models. After Whisper, almost everything in the category became "Whisper, plus our wrapper" or "we trained from Whisper."
- Open weights — runs locally on a Mac mini or modest GPU
- 99-language support, with strong English and major-language coverage
- API is among the cheapest in the category
- Outputs verbatim or smoothed; supports word-level timestamps
- Massive ecosystem of fine-tunes, wrappers, and hosting providers
- Hallucinates on silent audio — invents content that wasn't there
- Large-v3 is the latest; OpenAI hasn't released a v4 yet
- Diarization (who-said-what) requires a second model on top
- Real-time streaming requires a third-party wrapper, not native
- Developers building transcription into their own apps
- Teams self-hosting for privacy or cost reasons
- Multilingual workflows where one model has to handle many languages
- Anyone needing word-level timestamps for caption rendering
- You need diarization out of the box (use Otter, Descript, AssemblyAI)
- You want a polished consumer interface (use Whisper-derivatives instead)
- You need real-time streaming with native API support
Pricing
Run locally on CPU or GPU. Apache 2.0 license.
Hosted Whisper-large-v3 via OpenAI's audio.transcriptions endpoint.
Replicate, Together, Groq all host Whisper at competitive prices.
What it is
Whisper is a transformer-based encoder-decoder model trained on 680,000 hours of multilingual, multitask supervised data scraped from the web. OpenAI released the weights under an Apache 2.0 license, which means anyone can run it, fine-tune it, or build a product on it without paying OpenAI anything.
The model comes in five sizes — tiny (39M parameters), base (74M), small (244M), medium (769M), large (1.5B). Larger models are slower but more accurate. The current release is large-v3, which dropped late 2023 and is still the public-weights state of the art. OpenAI's API serves a model branded "whisper-1" that's at minimum equivalent to large-v3, possibly better — they haven't said.
Why it dominated
Three reasons: it was free, it was good, and it shipped with code that ran on a laptop. The combination meant that every team in 2023 with a transcription problem tried Whisper before they tried anything paid. Most of them stopped looking after that. The category's commercial vendors — Otter, Descript, Rev — pivoted to building features on top of Whisper rather than competing with the underlying transcription quality.
The API followed in 2023 at $0.006 per minute, which is roughly an order of magnitude cheaper than the previous incumbents. That priced out the standalone-API category and forced consolidation around either (a) self-hosting, (b) OpenAI's hosted Whisper, or (c) a hosting provider like Replicate or Together that wraps Whisper at similar margins.
Where it falls short
Hallucinations on silent audio. This is the most-commented-on weakness — feed Whisper a 30-second segment of near-silence and it will sometimes generate a plausible sentence that isn't there. The fix is voice-activity-detection (VAD) preprocessing: skip the silent parts before sending audio to Whisper. Most production wrappers do this; the raw model doesn't.
No native diarization. Whisper transcribes; it doesn't separate speakers. The standard pattern is Whisper plus pyannote.audio for diarization, run as two passes and merged at the timestamp level. This is the difference between Whisper and a finished consumer product like Otter — Otter does the merge for you.
No native streaming. Whisper is a chunked-input model: feed it a 30-second window, get back a transcription. Real-time streaming wrappers exist (whisper.cpp, faster-whisper) but the streaming isn't part of the model architecture; latency on long-form live audio is bounded by the chunk window.
Verdict
Whisper is the default, the baseline, and the thing every other transcription tool has to differentiate against. Self-host it for privacy or scale, use the API for everything else. If you want a polished consumer experience with diarization and meeting summaries, look at Whisper-derivatives like Otter or TurboScribe — but know that the underlying transcription is still Whisper.
