Advanced

Speaches

Speaches is an open-source server that speaks the OpenAI audio API: transcription with faster-whisper, speech with Kokoro and Piper. One container does both jobs for sidetone, and nothing leaves your network.

1. Run it

On a machine with a CPU only:

$ docker run -d --name speaches --restart unless-stopped \
    --publish 8000:8000 \
    --volume hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
    ghcr.io/speaches-ai/speaches:latest-cpu

With an NVIDIA GPU, which makes transcription many times faster, use the CUDA image and give it the GPU:

$ docker run -d --name speaches --restart unless-stopped --gpus=all \
    --publish 8000:8000 \
    --volume hf-hub-cache:/home/ubuntu/.cache/huggingface/hub \
    ghcr.io/speaches-ai/speaches:latest-cuda

The volume keeps the downloaded models across restarts. Speaches's own installation page has compose files for both, and the current image tags.

2. Download the models

Speaches downloads a model when it is first asked for it, but the first request then takes minutes and sidetone gives up after 50 seconds. Fetch them beforehand with its command-line tool, which needs nothing installed but uv:

# a voice for prompts: Kokoro, the better one, and Piper, the smaller one
$ uvx speaches-cli model download speaches-ai/Kokoro-82M-v1.0-ONNX
# transcription: large-v3-turbo is accurate and still quick; distil-small.en is for a small CPU
$ uvx speaches-cli model download deepdml/faster-whisper-large-v3-turbo-ct2
$ uvx speaches-cli model ls

uvx speaches-cli registry ls --task text-to-speech and --task automatic-speech-recognition list everything it can fetch. Point the tool at a server on another machine with SPEACHES_BASE_URL=http://host:8000.

3. Tell sidetone

Under Settings → Text to speech and Settings → Transcription, or in one request:

$ api -X PUT -d '{"tts_url":"http://speech.example.lan:8000/v1","tts_model":"speaches-ai/Kokoro-82M-v1.0-ONNX","tts_voice":"af_heart",
                "stt_url":"http://speech.example.lan:8000/v1","stt_model":"deepdml/faster-whisper-large-v3-turbo-ct2",
                "transcribe_voicemail":true}' localhost:8080/v1/settings

Three things people get wrong the first time:

The address ends at /v1. Not /v1/audio, not /v1/audio/speech; sidetone adds those and refuses an address that already has them.
The model is the full name Speaches shows, with the organisation in front: speaches-ai/Kokoro-82M-v1.0-ONNX, not kokoro. curl http://speech.example.lan:8000/v1/models lists what it has.
No API key. Leave the key fields empty; Speaches does not ask for one.

Then press Test text to speech and Test transcription. The second has Kokoro say a sentence and whisper write it down, and shows both.

Voices

Kokoro's voices are named by language and sex, then a name: af_heart and af_bella are American women, am_adam an American man, bf_emma and bm_george British. Piper models carry the voice in the model name, such as speaches-ai/piper-en_US-amy-medium, and take no voice setting. Try a few with Sounds → From text…: a prompt is made once and kept, so pick the one that suits the office.

What to expect

On a modern CPU Kokoro makes a ten-second prompt in a second or two, and large-v3-turbo transcribes a one-minute message in roughly its own length; on a GPU both are near instant. Transcription is where a weak machine shows: if voicemail emails go out without their text, the server took longer than the 90 seconds sidetone waits, and a smaller whisper model or a GPU is the fix.

← Speech servers Chatterbox →