Skip to main content

Class: ElevenLabsTextToSpeechProvider

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:223

Text-to-speech provider that uses the ElevenLabs TTS API.

API Contract​

  • Endpoint: POST {baseUrl}/text-to-speech/{voiceId}
  • Authentication: xi-api-key: <apiKey> header
  • Content-Type: application/json
  • Accept: audio/mpeg (MP3 response)
  • Request body: { text, model_id, voice_settings: { stability, similarity_boost, style, use_speaker_boost } }
  • Response: Raw MP3 audio bytes

A dialogue with line times​

synthesizeDialogue voices several speakers in one request through Text to Dialogue with timestamps:

Voice Settings​

ElevenLabs exposes fine-grained voice control via voice_settings:

  • stability (0.0–1.0) — Lower values = more expressive/variable, higher = more consistent
  • similarity_boost (0.0–1.0) — Higher values make output more similar to the original voice
  • style (0.0–1.0) — Style exaggeration (optional, only for v2+ models)
  • use_speaker_boost (boolean) — Enhances speaker similarity (default: true)

These can be passed via options.providerSpecificOptions.

Voice ID Resolution​

The voice ID is resolved with the following priority:

  1. options.voice (per-call override)
  2. config.voiceId (constructor default)
  3. options.providerSpecificOptions.voiceId (legacy override path)
  4. 'EXAVITQu4vr4xnSDxMaL' (hardcoded fallback — the "Sarah" voice)

Voice Listing​

listAvailableVoices fetches the user's voice library from the /voices endpoint and maps each entry to the normalized SpeechVoice shape. Returns an empty array on API errors (graceful degradation).

See​

ElevenLabsTextToSpeechProviderConfig for configuration options

Example​

const provider = new ElevenLabsTextToSpeechProvider({
apiKey: process.env.ELEVENLABS_API_KEY!,
voiceId: 'pNInz6obpgDQGcFmaJgB', // "Adam"
});
const result = await provider.synthesize('Hello world', {
providerSpecificOptions: { stability: 0.7, similarityBoost: 0.8 },
});

Implements​

Constructors​

Constructor​

new ElevenLabsTextToSpeechProvider(config): ElevenLabsTextToSpeechProvider

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:256

Creates a new ElevenLabsTextToSpeechProvider.

Parameters​

config​

ElevenLabsTextToSpeechProviderConfig

Provider configuration including API key and optional defaults.

Returns​

ElevenLabsTextToSpeechProvider

Example​

const provider = new ElevenLabsTextToSpeechProvider({
apiKey: 'xi-xxxx',
voiceId: 'pNInz6obpgDQGcFmaJgB',
model: 'eleven_multilingual_v2',
});

Properties​

displayName​

readonly displayName: "ElevenLabs" = 'ElevenLabs'

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:228

Human-readable display name for UI and logging.

Implementation of​

TextToSpeechProvider.displayName


id​

readonly id: "elevenlabs" = 'elevenlabs'

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:225

Unique provider identifier used for registration and resolution.

Implementation of​

TextToSpeechProvider.id


supportsStreaming​

readonly supportsStreaming: true = true

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:234

Streaming is supported — ElevenLabs offers a WebSocket streaming endpoint, and even the REST endpoint can be consumed as a stream.

Implementation of​

TextToSpeechProvider.supportsStreaming

Methods​

getProviderName()​

getProviderName(): string

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:271

Returns the human-readable provider name.

Returns​

string

The display name string 'ElevenLabs'.

Example​

provider.getProviderName(); // 'ElevenLabs'

Implementation of​

TextToSpeechProvider.getProviderName


listAvailableVoices()​

listAvailableVoices(): Promise<SpeechVoice[]>

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:544

Fetches the user's voice library from the ElevenLabs API.

Returns available voices mapped to the normalized SpeechVoice shape. Gracefully returns an empty array on API errors (e.g. network failure, invalid key) to avoid breaking voice selection UIs.

The voice library includes both ElevenLabs' pre-made voices and any custom/cloned voices in the user's account.

Returns​

Promise<SpeechVoice[]>

A promise resolving to an array of available voices, or an empty array if the API call fails.

Example​

const voices = await provider.listAvailableVoices();
const rachel = voices.find(v => v.name === 'Rachel');

Implementation of​

TextToSpeechProvider.listAvailableVoices


synthesize()​

synthesize(text, options?): Promise<SpeechSynthesisResult>

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:299

Synthesizes speech from text using the ElevenLabs TTS API.

Parameters​

text​

string

The text to convert to audio.

options?​

SpeechSynthesisOptions = {}

Optional synthesis settings. Use providerSpecificOptions to control ElevenLabs-specific voice settings (stability, similarityBoost, style, useSpeakerBoost).

Returns​

Promise<SpeechSynthesisResult>

A promise resolving to the MP3 audio buffer and metadata.

Throws​

When the ElevenLabs API returns a non-2xx status code, with the status and the body. Common causes: invalid API key (401), voice not found (404), character limit exceeded (400), or rate limit (429). A key that answers with a spent quota hands over to the next key of the pool, each key once.

Example​

const result = await provider.synthesize('Hello there!', {
voice: 'pNInz6obpgDQGcFmaJgB',
providerSpecificOptions: {
stability: 0.3, // More expressive
similarityBoost: 0.9, // Closer to original voice
style: 0.5, // Moderate style exaggeration
},
});

Implementation of​

TextToSpeechProvider.synthesize


synthesizeDialogue()​

synthesizeDialogue(lines, options?): Promise<ElevenLabsDialogueResult>

Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:405

Voices a dialogue of several speakers in one request, through ElevenLabs' Text to Dialogue with timestamps, and returns the audio with each line's place in it.

The audio is the bytes ElevenLabs sent, decoded from base64 with no re-encode. Each voice segment comes back with the index of its input line and its start and end seconds, in the order the API gave them; no segments are merged. The character alignments come back when the answer has them.

The key and the base URL are the ones synthesize uses: the xi-api-key header from the provider's key pool, where a key that answers with a spent quota hands over to the next key, each key once, and config.baseUrl.

ElevenLabs' reference gives two limits the method leaves to the API: at most 10 unique voice ids per request, and, for reliable generation, at most 2,000 characters of text across all lines; longer requests can end early or return a validation error. Split a longer dialogue into requests.

Parameters​

lines​

readonly ElevenLabsDialogueLine[]

The lines in speaking order, each with its text and voice id.

options?​

ElevenLabsDialogueOptions = {}

The model, the output format and an optional seed.

Returns​

Promise<ElevenLabsDialogueResult>

The audio, its format, the voice segments and the character alignments when present.

Throws​

Before any request, for an empty dialogue, a line without text or voice id, or a seed that is not a whole number from 0 to 4294967295.

Throws​

When ElevenLabs answers with an error status (401 for a bad key, 422 for a request it refuses, 429 for a spent quota), with the status and the body.

Throws​

When a successful answer has no audio_base64, one that decodes to no bytes, no well-formed voice_segments (a line index outside the lines sent included), or an alignment that is present but malformed.

Example​

const dialogue = await provider.synthesizeDialogue(
[
{ text: 'Can you hear me?', voiceId: 'voice-a' },
{ text: 'Loud and clear.', voiceId: 'voice-b' },
],
{ model: 'eleven_v4', outputFormat: 'mp3_44100_128', seed: 7 },
);
for (const segment of dialogue.segments) {
console.log(segment.lineIndex, segment.startSeconds, segment.endSeconds);
}