An optional caption-text relay for people running their own caption.ninja setup. It is a new implementation of the browser message protocol, not the source of the public api.caption.ninja service. It needs no GPU, speech model, cloud API account or Caption Local Python environment. It runs separately from inference.

See the visual workflow and hosting options for how recognition, the relay, the editor and audience displays fit together.

Use Node.js 22 or newer and this directory's pinned ws dependency. Windows 11 and WSL/Linux results are recorded in Caption Local's relay evidence. Plan for 128 MiB RAM for a small private installation and measure your workload; this is an allowance, not a proven minimum. Text relay capacity does not establish speech recognition capacity. Internet hosting needs a domain and a TLS reverse proxy; local testing uses loopback without system changes.

The relay and browser helper use the repository's MPL-2.0 license. The only runtime npm dependency is ws; existing capture pages need no Node installation.

Install and start

In a captionninja checkout, run these commands from relay.

Windows PowerShell:

npm.cmd ci --ignore-scripts
node init-config.cjs
$env:CAPTION_RELAY_CONFIG = "$env:USERPROFILE\.caption-ninja-relay.private.json"
node server.cjs

Linux or WSL:

npm ci --ignore-scripts
node init-config.cjs
export CAPTION_RELAY_CONFIG="$HOME/.caption-ninja-relay.private.json"
node server.cjs

The configuration generator creates a new random master secret and refuses to overwrite an existing file. Rooms and their read/write tokens are not stored here: Caption Local derives them per room from the same master secret, so the relay needs no per-room configuration. Keep the file outside the website directory and restrict access to the service operator. Never commit it, upload it with the site, or paste the master secret into support logs. Share the same value as CAPTION_RELAY_MASTER with Caption Local; rotating it invalidates every derived room and token.

The relay listens on 127.0.0.1:8787. Visit http://127.0.0.1:8787/health for aggregate connection/message counters. It serves no files. Stop with Ctrl+C. Set process environment PORT to choose a different port; HOST is intended for container binding or an explicitly secured deployment. This program installs no background service and changes no firewall or system settings.

Connect capture, editor and overlay

Use the updated Caption Local and captionninja fork source with the captions-deploy Compose stack. It serves the room pages, capture page and relay from one origin. Do not use a bare python -m http.server for room mode: room sessions are HttpOnly same-origin cookies and the API/WebSocket must be proxied.

Create secrets/relay.json with this relay's master and the exact UI origin, then set the same CAPTION_RELAY_MASTER, RELAY_ADDRESS=wss://<ui>/ws and RELAY_SITE=https://<ui>/ in secrets/caption-local.env.
Start the stack and open the UI. Create a room; keep its private admin link.
Open the admin link. The room page fetches derived credentials from /api/rooms/<id>/config and links the capture page, editor and audience page.
Start captions. Review incoming text in the editor, then send it to the audience output. Open /watch/<id> for the audience or OBS.
Stop capture, let audio finish, and download your transcript before closing. Close the room when the audience no longer needs it.

Rooms are created on demand and retired when closed. There is no provisioning step, per-user key or manual room-token entry in room mode. A viewer cannot publish using an output read token, and credentials for one room cannot join another. The admin secret never reaches the relay; only derived tokens do. The legacy manual setup panels remain available for older cross-origin workflows.

For OBS, use the public audience link from the room or editor. The page fetches a short-lived output read token itself. The legacy Create a view-only OBS link helper still works for manual deployments; never put a write token in an audience link. Fragment and query credentials can be retained by OBS settings, the clipboard or browser history, so prefer the room audience link.

Optional cloud translation/TTS can still make external requests; leave those features off when you need all processing to stay private.

Internet hosting and Docker

The reference captions-deploy stack proxies /ws on the UI origin, so pages use wss://ui.example.org/ws and no separate relay hostname is needed. It also sets Host: 127.0.0.1:8765 for the speech service behind nginx.

If you run the relay separately, host the pages over HTTPS and put a TLS reverse proxy in front of the loopback relay. For example, a Caddy site can use:

relay.example.com {
    reverse_proxy 127.0.0.1:8787
}

See the Caddy reverse-proxy documentation for WebSocket proxy behavior and TLS deployment options.

Use the public relay URL or /ws in every participant's page. Configure the exact HTTPS website origins in the private JSON origins list. Origin checks supplement tokens; they are not authentication. Native clients without Origin still need valid derived credentials. Avoid proxy access logging of sensitive URLs and apply host-level connection/rate limits for public deployments. Forwarded IP headers are deliberately ignored: clients behind one proxy share the per-IP limit.

Docker alternative, from this directory after generating the config:

docker build -t caption-ninja-relay:local .
docker run --name caption-ninja-relay --rm --read-only --cap-drop ALL \
  --security-opt no-new-privileges --pids-limit 64 --memory 128m \
  -p 127.0.0.1:8787:8787 \
  --mount "type=bind,src=$HOME/.caption-ninja-relay.private.json,dst=/run/relay.json,readonly" \
  caption-ninja-relay:local

Ensure the container's unprivileged node user can read the mounted config without making it world-readable. On Linux, use an appropriate owner/group or run the container with your numeric UID/GID using --user "$(id -u):$(id -g)". Use docker stop caption-ninja-relay from another terminal to stop it. Linux CI has built the image and verified authenticated delivery and shutdown in an unprivileged, read-only container. A Windows test also verified real Caddy HTTPS/WSS delivery with an explicit temporary CA and rejection of invalid certificates/tokens. These checks do not validate a public domain, WAN performance or Docker Desktop; rehearse the deployment on your host. No Docker installation is required for the native setup.

Bounds, privacy and recovery

Defaults: 512 WebSockets total, 128 per direct peer IP, 256 per room; 40 messages per connection per second including join; 8 KiB messages; 4,000-character captions; 64 KiB pending transport output per connection; 10-second join deadline; 30-second heartbeat. Slow reliable viewers have a 10-second backpressure grace period before disconnection. Publishers retain at most 100 unacknowledged captions in the browser; the newest caption is rejected visibly if that queue fills. HTTP connections are also bounded. At most 1,000 derived rooms are tracked at once. Limits are implementation defaults, not guaranteed host capacity. Overload rejects clients or disconnects slow viewers instead of growing buffers.

An optional limits object in the private configuration can set these bounds. For example, "limits": {"perIp": 512} admits larger audiences behind one reverse proxy or on one test computer. The default is 128 per IP, so 32 producers plus 100 viewers sharing a proxy need an explicit increase. This raises admission capacity; it does not enlarge caption buffers or guarantee host throughput. Unknown limit names or nonpositive/noninteger values are rejected.

The server forwards captions only to authenticated readers in the same room. It now keeps a short replay history in RAM, along with delivery receipts: at most 120 seconds and 256 messages per room, bounded globally by 8,192 messages and a 16 MiB accounting budget. The oldest records expire first; the accounting budget is not a process RSS limit. Text never goes to disk or logs; no audio is stored. Startup logs and /health contain operational data only, including replay counts, duplicate receipts and gaps. A private relay failure never falls back to the public relay.

The private publisher keeps each caption until the relay acknowledges it, retrying the same identity if the acknowledgement is lost. Receipts suppress duplicate delivery within the retained history. Returning viewers resume from their last caption cursor, even when the producer reconnects first. New viewers start live. Each room's viewing token authorizes its retained history as well as live captions.

This is bounded recovery, not durable or exactly-once delivery. If the buffer expires or is evicted, the viewer shows a gap warning. Restarting the relay clears history and receipts; current-process captions can be replayed after restart, but earlier captions may be missing and an unacknowledged retry can repeat. Restart/gap warnings make this visible. A crashed or reloaded browser also loses its in-memory queue/cursor. The editor's existing 100-caption review queue remains bounded; if it overflows, private-mode users get an explicit dropped-caption warning.

Authentication denial stops publisher automatic retries. In room mode, reopen the current admin link so the client refreshes its derived credentials; rotate the admin link or close the room if a credential may have leaked. Restarting with a different master invalidates all derived credentials and disconnects existing clients. Rehearse outages before relying on a deployment at an event. Invalid format or oversized frames also stop retries and show an error. The 8 KiB limit counts UTF-8 bytes, so a long multilingual caption can exceed it before reaching 4,000 characters. Copy the text, shorten it and start a new publishing session before resending. A denied publisher retains its queue in memory until that session is closed; reconnecting the same invalid payload will not repair it.

Tests and protocol

Measured on Windows 11, a Core Ultra 7 265K and Node 22.19.0, using short captions in six languages and five captions/second per producer:

Text producers	Total viewers	Test duration	p95 relay delay	Result
32	100	1 hour	5.7 ms	1.8 million deliveries; no errors, duplicates or gaps
32	400	1 minute	10.3 ms	120,000 deliveries; no errors, duplicates or gaps

Both tests injected disconnects; the maximum observed delay was about two seconds, including an intentional two-second viewer outage. Server and simulated browser clients shared one Node process on loopback. The hour used the initial recovery client; the later short probe and targeted tests cover the invalid-frame retry fix. These are text delivery measurements, not speech recognition or public-network capacity. The 400-viewer probe is too short to establish sustained capacity. See the linked evidence for exact versions, memory/CPU, recovery limits and unsuccessful configurations. Admission was explicitly 512 per IP for loopback.

npm test
node benchmark.cjs 60 /path/to/new-results.json
node soak.cjs 3600 /path/to/new-hour-results.json 100

The benchmark uses 32 independent synthetic producers and two viewers per room, at five captions/second each. It measures loopback delivery, ordering, isolation, CPU and memory in one Node process containing both clients and server. It does not establish WAN latency or a sustained maximum capacity. The separate soak uses the actual reliable browser client with 32 producers, the requested total viewer count, periodic forced disconnects and fixed-size latency histograms. Its results include errors, gaps, duplicates, queue/retention bounds and memory samples. The integration probe is scripts/browser_private_relay.py in Caption Local; it uses a synthetic microphone, fake inference and this real relay with public traffic blocked.

Reliable clients join using {"protocol":2,"join":"ROOM","role":"read|write","token":"TOKEN"}. Returning readers include "cursor":{"epoch":"PREVIOUS_EPOCH","sequence":123}. The join response includes the process epoch, next sequence and any gap reason. Writers send {"msg":true,"final":"text","id":1,"delivery":{"client":"UNIQUE_PRODUCER_ID","sequence":1}}. The relay acknowledges that delivery identity and adds an epoch/sequence cursor to forwarded captions. interm can replace final; optional label and ln are preserved. Original caption IDs remain distinct from delivery identities. The legacy private join without protocol is still supported but has no reliable client acknowledgements or resume cursor. One socket has one immutable room/role; viewers cannot publish. Update both server and private pages to use recovery.