Whisper Large V3 Turbo

This model is ideal for use cases like live note-taking applications, content captioning, customer support, and any real-time, voice-powered apps. Learn more about how Whisper is running faster and more accurately on Baseten in our launch blog.

Feature highlights

• Configurable update cadence for the partial transcriptions delivered

• Consistent real-time latency under high volume of concurrent audio streams

• Automatic language detection for multilingual transcription

Transcription accuracy

The Pareto plot below shows how this model compares with leading closed-source ASR solutions across accuracy and latency. Results are measured using Pipecat’s open-source STT Benchmark, which evaluates Semantic WER for transcription accuracy and TTFS (time to final segment) for latency.

✕

Accuracy vs latency — semantic word-error-rate vs median time-to-final-segment (single stream), plotted against closed-source providers.

Real-time performance

The graph below shows how TTFS (time to final segment) and unit economics (cost per audio hour) change as the number of concurrent WebSocket connections per GPU increases. Higher concurrency can significantly reduce cost per audio hour, with a corresponding tradeoff in latency.

Concurrency is configurable and can be tuned to achieve the latency and cost targets that best fit your workload and business requirements. See the concurrency guide for configuration details.

For a balance of latency and throughput, we recommend starting with RTX6000 using concurrency 24, and adjusting based on performance and cost needs based on the chart below if needed.

For workload-specific optimization, you can contact an engineer for further assistance.

✕

Latency vs concurrency — how median latency holds up as concurrent streams scale, by GPU.

Batch performance

The charts below show median latency and unit economics (cost per audio hour) change with audio length when transcribing prerecorded files at 1, 4 and 16 concurrent requests per GPU. Higher concurrency significantly reduces cost per audio hour, with a corresponding tradeoff in latency that grows with audio length.

Concurrency is configurable and can be tuned to achieve the latency and cost targets that best fit your workload and business requirements. See the concurrency guide for configuration details.

For batch workloads, we recommend starting with RTX6000 using concurrency 16, and lowering it to reduce latency on longer audio or raising it to increase throughput on latency tolerant workloads

✕

Batch transcription latency and cost across audio lengths at concurrency 1, 4 and 16. Running more requests in parallel lowers cost per audio hour, while P50 latency grows with audio length, most sharply at concurrency 16.

Contact us

For workload-specific optimization, you can contact an engineer for further assistance.