Hardware and model quality

Who this is for: hosters (sysadmins).

Hardware, model and capacity are separate choices. This page turns the project's measured profiles into plain guidance for either supported deployment method: Deploy with Docker Compose or Deploy with Co-op Cloud. It is a starting point, not a capacity promise — rehearse your own languages and audio before an event. If a word is new, see the glossary.

The models

The model setting chooses which Whisper weights the speech service loads. Larger is generally more accurate but slower and hungrier for memory — and not uniformly better.

Model	Speed	Accuracy	Memory / VRAM
base	Fastest	Lowest; the English throughput preset, which failed the Spanish quality checks	Not stated in the sources
small	Faster than medium/large-v3	Tested release default; multilingual	Not stated
medium	Slower than small	Stronger; 5.07% mean English WER on a single accuracy-oriented stream	Not stated
large-v3	Accurate but heavy	Accuracy-oriented; still lost to medium on a noisy French fixture	About 4.7 GiB VRAM and near 2.8 GiB host RSS during model start-up in one short probe; varies by host

What this means in practice:

• small is the safe default. It is the tested release default and passes the multilingual quality checks that base fails.
• medium is for when small loses meaning. It passed the quality gate as a single accuracy-oriented stream; sustained medium concurrency is untested.
• large-v3 is for accuracy with a GPU. Expect it to use several gigabytes of VRAM. Size alone is not a reliable guide — it lost to medium on noisy French.
• base is only for English throughput on a constrained CPU. Its Spanish quality failure makes it unfit for multilingual use.

Memory needs and available VRAM vary across hosts, and the numbers above come from a single probe on one machine.

CPU or GPU?

Situation	Guidance
Desktop CPU	Start with multilingual small and one stream. No GPU setup needed; larger models and more streams need more CPU time and memory.
NVIDIA GPU	Explicit CUDA; start with small, then compare medium/large-v3 as VRAM permits. Benchmark accuracy and throughput on your host.
AMD/Intel/Apple GPU	Not implemented in this backend. Use explicit CPU and validate the platform.
Shared or private server	Run the stack there and let producers reach it over HTTPS through the edge router; the producer's microphone stays with them.
Existing Docker host	Use the supplied Compose route; NVIDIA containers have extra prerequisites.

A GPU speeds up recognition but is optional. The relay never needs one.

Capacity settings

The speech service bounds how much work it admits: how many simultaneous streams it will accept, how many decoding workers and threads it uses, and how wide it searches while decoding. More is not always better. Extra workers serve more streams but consume more CPU, and worker and thread counts multiply together.

Your deployment method sets these when you configure the speech service. Rather than copying numbers from another host, start small — one stream — and benchmark on your own hardware before adding capacity.

Caption interval

The browser chooses how much speech to collect before processing: 3, 6 or 9 seconds, with six the default.

• Six seconds is the balanced default; keep it unless your hardware and languages pass both latency and accuracy checks.
• Three seconds shows captions earlier but sends requests more often and may lose recognition context. A three-second exact-transcript check also failed on large-v3/CUDA, with words repeated and dropped.
• Nine seconds delays captions and supplies more context.

Pause and Stop still flush buffered speech, and the interval does not enlarge the 30-second protective buffer or change retained-audio retry behaviour.

What the measurements mean

The sources report real tests, but short probes are not sustained-capacity promises. A few points in plain terms:

• Stream counts are ceilings to test, not promises. Some short probes admitted many streams, but longer runs and mixed transcription/translation fell behind.
• A GPU is not a free pass. On a TITAN RTX, large-v3/CUDA float16 passed an hour of twelve varied synthetic streams, but the workload covered six languages with limited bilingual output and does not qualify twelve simultaneous translations.
• "p95 inference" means 95% of requests finished faster than that figure; it is not the time until a caption appears on screen.
• Noisy speech is a separate limit. Repetitive French text and a German noise-probe failure remain even on good hardware.

Choose by language and recording conditions

Select a model using the languages and audio you will actually use, not its size. If audio buffers keep growing, the service is falling behind — reduce the workload rather than increasing buffers, which does not make recognition faster.