Advanced

Speech servers

sidetone can say a prompt from a line of text and write a voicemail message down as text. It does neither itself: it asks a speech server, which can be OpenAI or an open-source one on a machine of your own. Off until you give it one.

One protocol, any server

sidetone speaks the OpenAI audio API and nothing else: /v1/audio/speech to turn text into sound and /v1/audio/transcriptions to turn sound into text. That is the one protocol the open-source servers have settled on too, so to sidetone they are all the same: OpenAI with an API key, or Speaches, Chatterbox, LocalAI, Kokoro-FastAPI or openedai-speech on your own hardware, which keeps everything in the house. There are no vendor plug-ins to install.

There are two places for a server, because they may be two servers, or one server twice:

• Settings → Text to speech: tts_url, tts_api_key, tts_model, tts_voice. Says prompts.
• Settings → Transcription: stt_url, stt_api_key, stt_model. Writes voicemail down.

An address is the base, up to and including /v1, such as http://localhost:8000/v1; the /audio/… part is added, and an address that already ends in one is refused. Models and voices are whatever that server calls them: GET /v1/models on the server lists its models, and an open-source server wants the full name it shows, such as speaches-ai/Kokoro-82M-v1.0-ONNX. The defaults, tts-1, alloy and whisper-1, are OpenAI's. A key is only needed where the server asks for one; a server of your own usually does not.

Try it where you set it up

Each section of the settings has a test button: Test text to speech speaks a sentence and reports how long it took; Test transcription has the text-to-speech server say a sentence, sends it to the transcription server, and shows what came back, so you can see the two agree. A failure says what the speech server said. Over the API it is POST /v1/settings/test-speech with {"what":"speak"} or {"what":"transcribe"}. The settings in the form are saved first, so that the test is of what you see. A model being loaded for the first time may need a second try: a request is given 50 seconds.

Prompts from text

Sounds → From text…, or:

$ api -d '{"text":"Thank you for calling. For sales press 1, for support press 2."}' localhost:8080/v1/audio/welcome/speech

The result is a sound like any uploaded one, named welcome here, to use as a greeting, a menu prompt or hold music. Whatever rate the server speaks at is brought down to telephone audio. Only the text goes to the server.

Voicemail as text

With transcribe_voicemail on, each new message's audio goes to the transcription server, and what it understood is kept with the message (transcript), shown in the console and put into the email, which waits up to 90 seconds for it and goes without it if the server is down. It is a machine's reading: names and numbers are where it goes wrong.

This is the one place something of a caller's leaves the server, and it goes to stt_url and nowhere else. With OpenAI's address, callers' messages leave your premises: say so where the law wants it said. With a server of your own, they do not.

Choosing

Prompts are made once and kept, so even a slow text-to-speech server is fine for them. Transcription runs on every message, so that is where speed matters.