Skip to content

Translation and text-to-speech

Who this is for: admins (event producers) and users.

Two optional extras sit on top of ordinary captions: translation and text-to-speech. Both are off by default — nothing changes unless you ask for it. Unfamiliar words are defined in the glossary.

Two kinds of translation

Translation happens in two different places, and they are not the same thing:

  • The speech service can translate what it hears into English while it transcribes. That is the engine doing the work, on the machine running the speech service.
  • The overlay can translate the caption text into many other languages, inside each viewer's own browser, using Mozilla's local Bergamot engine. No cloud service and no API key, and the text never leaves the viewer's browser.

So "translate into English" describes the engine. The overlay can additionally show the captions in other languages.

The engine translates into English

The speech service can work in three modes: write down the words in the language spoken, translate them into English, or produce both at once. It is not a general two-way translator.

Translation is done locally. This fork has no cloud translation provider and no API key to pay for. It only works for the supported language pairs; if a pair is missing, only the original text appears.

To choose the mode, set it when you start capture and also decide what gets published:

  • Transcribe only — the original language appears on the overlay.
  • Translate only — English appears on the overlay.
  • Both — the original and the English translation are both produced.

The editor can publish either the original text or the English text to the audience room, so reviewers can decide what the audience sees.

The overlay translates into many languages

Each audience overlay can show the captions in a different language. The viewer's own browser does the translation locally, so it is private and free. Supported languages are:

Bulgarian (bg), Czech (cs), Dutch (nl), English (en), Estonian (et), French (fr), German (de), Icelandic (is), Italian (it), Norwegian Bokmål (nb), Norwegian Nynorsk (nn), Persian (fa), Polish (pl), Portuguese (pt), Russian (ru), Spanish (es) and Ukrainian (uk).

This overlay shows the captions in Spanish.

https://ui.example.org/overlay.html?room=output&translate=es

Because the work happens in the viewer's browser, you can hand out one link per language and let each person open the one they need. The models for this are prepared by whoever runs the site; see Backups, upgrades and recovery.

Extra options:

  • &dual=1 — show the translation and the original together (see below).
  • &fromlang=XX — state the source language, because the local engine cannot detect it automatically.
  • &context=1 and &contextsize=3 — use nearby lines as context for more natural translation in conversation.

Show the original and the translation together

If you want one overlay that carries both, enable the dual view. The translated (English) line is rendered first and the original transcript appears beneath it in a compact style, so the overlay stays roughly the same height.

This dual overlay shows the translation with the original underneath.

https://ui.example.org/overlay.html?room=output&dual=1

History settings such as maxlines and clear still apply to the combined block; you can still style the overlay in OBS or vMix. See Overlays for your audience for those options.

Text-to-speech (read aloud)

Text-to-speech, often shortened to TTS, makes a caption page speak the captions out loud. It is disabled unless you add a &tts parameter to the page. You can have the audience overlay read captions, the capture page read them back to you, or a separate manual page speak typed text.

Page Purpose
overlay.html Read captions to the audience
index.html Read captions back to the speaker
manual.html Read typed text aloud
TTS bridge A pop-out window that speaks for another page

This overlay reads the captions aloud in US English.

https://ui.example.org/overlay.html?room=output&tts=en-US

This capture page recognises and reads back in US English.

https://ui.example.org/index.html?room=output&lang=en-US&tts=en-US

Choosing a voice

By default TTS uses the browser or operating system's built-in voice. You can instead use the tts.rocks engine, which offers several providers such as Kokoro, Piper, ElevenLabs and OpenAI.

This overlay uses the tts.rocks engine with the Kokoro voice af_aoede.

https://ui.example.org/overlay.html?room=output&tts=en-US&ttslib=rocks&ttsprovider=kokoro&voicekokoro=af_aoede&korospeed=1.0

Optional switches:

  • &ttsstream=1 starts speaking interim (not-yet-final) text for faster output.
  • &ttspopout=1 opens the pop-out TTS bridge window automatically.
  • The tts.rocks site has a "Use with Caption.Ninja" panel and a voice picker that generate ready-made links, so you do not have to build them by hand.

When a dual overlay is in use, TTS speaks the translated text while the original is only displayed.

Keys in URLs are visible

A cloud provider key placed in a URL can be seen by anyone who has that link, and links are kept in OBS settings, the clipboard and browser history. Prefer a built-in or local voice, or only share overlays that contain no keys.

The capture page's Customize Overlay button and the translate page's overlay options can build TTS links for you. Keep TTS off when the event must stay entirely private, because some providers send text to an outside service.