Backups, upgrades and recovery

Who this is for: hosters (sysadmins).

This page covers two admin jobs: keeping the pieces you cannot easily recreate, and moving to a new version without losing them. It covers both supported ways to run the stack — Docker Compose and Co-op Cloud. New to a term such as volume or service access token? See the Glossary.

What to back up

Most of the stack is reproducible from Git, but a few files are secrets or generated state that exist only on the host. Back those up first.

Item	Where it lives	Why it matters
Relay private config	secrets/relay.private.json	Holds room tokens and allowed origins. Losing it means every link you handed out stops working.
Speech-service access file	secrets/caption-local.env	Holds the service access token (CAPTION_API_KEY) and allowed origins.
Capture auto-fill file	secrets/relay-tokens.env	Holds the relay address, room names and room tokens the capture page uses to build editor/audience links.
Basic Auth file	secrets/traefik.htpasswd	Passwords for the optional HTTP Basic Auth overlay.
Environment file	.env.production	Hostnames and the Let's Encrypt email. Tracked by Git, so treat it as non-secret.
Orchestration config	traefik.yml, docker-compose*.yml, web/	Also tracked by Git; back up with the repository.
Downloaded models	Docker volumes caption-local-cache and translation-models	Large and re-downloadable, so optional to back up. See below.
Transcripts	Wherever operators downloaded them	The stack persists nothing automatically; a transcript exists only as a file someone saved.

The secrets/ directory is git-ignored

Nothing in secrets/ is committed. If you lose the host without a copy, the relay config and tokens are gone. Store a copy somewhere safe and private — never in the repository.

The TLS certificates in the letsencrypt volume are not essential to back up: Let's Encrypt can reissue them, as long as DNS points at the host.

On Co-op Cloud the same secrets and state live inside the app's managed storage rather than a local secrets/ directory, so back them up through the recipe instead of copying files by hand.

How models are cached (and how to avoid re-downloading)

The speech service downloads its Whisper model on first use. Under Docker the model lives in the named volume caption-local-cache, mounted at /models; the UI's translation models live in the translation-models volume. Keeping these volumes means you never pay the download again.

Models are pinned so a rebuild does not silently change what you run:

• The default small model is pinned to a fixed Hugging Face revision.
• The throughput base model is pinned to its own revision; medium and large-v3 are pinned in server.py too.
• Other model names fall back to upstream resolution unless you supply a local directory.

To prepare for an offline event, run the stack once while still online so the volume is populated, then turn on offline mode and recreate the service. To save disk or bandwidth you can set TRANSLATION_MODELS=none to skip the translation model download.

Do not delete the volumes by accident

docker compose down -v also removes named volumes, including the cached models. Only use it when you actually intend to delete the model cache. A plain docker compose down keeps them.

Backups with Co-op Cloud

Co-op Cloud stores the app's state through its own backup mechanism. The generic command is abra app <domain> backup; consult the caption-ninja recipe for the exact backup and restore commands and what each one includes. Restoring is a recipe-specific operation, so follow the recipe rather than copying files.

Upgrading with Docker Compose

An upgrade has four steps: save your work, stop the service, bring in the new code or images, and rebuild/recreate. The cached model volumes are kept throughout.

1. Save anything in progress. Download pending transcripts; capture pages hold retry caches in memory only.

docker compose --env-file .env.production \
  -f docker-compose.yml -f docker-compose.prod.yml down

1. Fetch the new orchestration and submodule code. The application code lives in the caption-local and captionninja submodules.

git pull
git submodule update --init --recursive

1. Rebuild and start. This pulls or builds images and recreates containers.

docker compose --env-file .env.production \
  -f docker-compose.yml -f docker-compose.prod.yml up -d --build

1. Smoke-test. Every service should report healthy.

docker compose --env-file .env.production \
  -f docker-compose.yml -f docker-compose.prod.yml ps

After a config-only change, such as editing a label or an env file, recreate just the affected service instead of the whole stack:

docker compose --env-file .env.production \
  -f docker-compose.yml -f docker-compose.prod.yml up -d --force-recreate caption-local

Upgrading with Co-op Cloud

The generic command is abra app <domain> upgrade. Consult the caption-ninja recipe for the exact upgrade steps, which version or tag to choose, and whether a data migration runs first. Take a backup before upgrading, as above.

Recovery and rollback

Rollback follows the same shape in reverse. Keep the previous release archive or image before you upgrade, and restore it if the new version misbehaves.

• Docker Compose: restore the previous application files (or the old image tag) and rebuild with up -d --build.
• Co-op Cloud: restore a recent recipe backup, or deploy the previous version again; the recipe explains how.
• Keep the model volumes and caches in place unless you deliberately want a clean download.
• Verify with docker compose ps (or the recipe's status command) and one real caption before an event.
• Never run docker compose down -v as part of a rollback: it deletes the model cache.

For a broken relay config under Docker, regenerating secrets/relay.private.json with init-config.cjs creates fresh room tokens, which invalidates old links — so a known-good backup is faster. See Troubleshooting for the permission fix if the relay will not start.