Class: ElevenLabsTextToSpeechProvider
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:223
Text-to-speech provider that uses the ElevenLabs TTS API.
API Contract
- Endpoint:
POST {baseUrl}/text-to-speech/{voiceId} - Authentication:
xi-api-key: <apiKey>header - Content-Type:
application/json - Accept:
audio/mpeg(MP3 response) - Request body:
{ text, model_id, voice_settings: { stability, similarity_boost, style, use_speaker_boost } } - Response: Raw MP3 audio bytes
A dialogue with line times
synthesizeDialogue voices several speakers in one request through Text to Dialogue with timestamps:
- Endpoint:
POST {baseUrl}/text-to-dialogue/with-timestamps?output_format=<format> - Request body:
{ inputs: [{ text, voice_id }], model_id, seed? } - Response: JSON with
audio_base64,voice_segments(each withdialogue_input_indexand its start and end seconds), andalignmentandnormalized_alignmentwhen present - Reference: https://elevenlabs.io/docs/api-reference/text-to-dialogue/convert-with-timestamps
Voice Settings
ElevenLabs exposes fine-grained voice control via voice_settings:
- stability (0.0–1.0) — Lower values = more expressive/variable, higher = more consistent
- similarity_boost (0.0–1.0) — Higher values make output more similar to the original voice
- style (0.0–1.0) — Style exaggeration (optional, only for v2+ models)
- use_speaker_boost (boolean) — Enhances speaker similarity (default: true)
These can be passed via options.providerSpecificOptions.
Voice ID Resolution
The voice ID is resolved with the following priority:
options.voice(per-call override)config.voiceId(constructor default)options.providerSpecificOptions.voiceId(legacy override path)'EXAVITQu4vr4xnSDxMaL'(hardcoded fallback — the "Sarah" voice)
Voice Listing
listAvailableVoices fetches the user's voice library from the
/voices endpoint and maps each entry to the normalized SpeechVoice
shape. Returns an empty array on API errors (graceful degradation).
See
ElevenLabsTextToSpeechProviderConfig for configuration options
Example
const provider = new ElevenLabsTextToSpeechProvider({
apiKey: process.env.ELEVENLABS_API_KEY!,
voiceId: 'pNInz6obpgDQGcFmaJgB', // "Adam"
});
const result = await provider.synthesize('Hello world', {
providerSpecificOptions: { stability: 0.7, similarityBoost: 0.8 },
});
Implements
Constructors
Constructor
new ElevenLabsTextToSpeechProvider(
config):ElevenLabsTextToSpeechProvider
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:256
Creates a new ElevenLabsTextToSpeechProvider.
Parameters
config
ElevenLabsTextToSpeechProviderConfig
Provider configuration including API key and optional defaults.
Returns
ElevenLabsTextToSpeechProvider
Example
const provider = new ElevenLabsTextToSpeechProvider({
apiKey: 'xi-xxxx',
voiceId: 'pNInz6obpgDQGcFmaJgB',
model: 'eleven_multilingual_v2',
});
Properties
displayName
readonlydisplayName:"ElevenLabs"='ElevenLabs'
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:228
Human-readable display name for UI and logging.
Implementation of
TextToSpeechProvider.displayName
id
readonlyid:"elevenlabs"='elevenlabs'
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:225
Unique provider identifier used for registration and resolution.
Implementation of
supportsStreaming
readonlysupportsStreaming:true=true
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:234
Streaming is supported — ElevenLabs offers a WebSocket streaming endpoint, and even the REST endpoint can be consumed as a stream.
Implementation of
TextToSpeechProvider.supportsStreaming
Methods
getProviderName()
getProviderName():
string
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:271
Returns the human-readable provider name.
Returns
string
The display name string 'ElevenLabs'.
Example
provider.getProviderName(); // 'ElevenLabs'
Implementation of
TextToSpeechProvider.getProviderName
listAvailableVoices()
listAvailableVoices():
Promise<SpeechVoice[]>
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:544
Fetches the user's voice library from the ElevenLabs API.
Returns available voices mapped to the normalized SpeechVoice shape. Gracefully returns an empty array on API errors (e.g. network failure, invalid key) to avoid breaking voice selection UIs.
The voice library includes both ElevenLabs' pre-made voices and any custom/cloned voices in the user's account.
Returns
Promise<SpeechVoice[]>
A promise resolving to an array of available voices, or an empty array if the API call fails.
Example
const voices = await provider.listAvailableVoices();
const rachel = voices.find(v => v.name === 'Rachel');
Implementation of
TextToSpeechProvider.listAvailableVoices
synthesize()
synthesize(
text,options?):Promise<SpeechSynthesisResult>
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:299
Synthesizes speech from text using the ElevenLabs TTS API.
Parameters
text
string
The text to convert to audio.
options?
Optional synthesis settings. Use providerSpecificOptions
to control ElevenLabs-specific voice settings (stability, similarityBoost,
style, useSpeakerBoost).
Returns
Promise<SpeechSynthesisResult>
A promise resolving to the MP3 audio buffer and metadata.
Throws
When the ElevenLabs API returns a non-2xx status code, with the status and the body. Common causes: invalid API key (401), voice not found (404), character limit exceeded (400), or rate limit (429). A key that answers with a spent quota hands over to the next key of the pool, each key once.
Example
const result = await provider.synthesize('Hello there!', {
voice: 'pNInz6obpgDQGcFmaJgB',
providerSpecificOptions: {
stability: 0.3, // More expressive
similarityBoost: 0.9, // Closer to original voice
style: 0.5, // Moderate style exaggeration
},
});
Implementation of
TextToSpeechProvider.synthesize
synthesizeDialogue()
synthesizeDialogue(
lines,options?):Promise<ElevenLabsDialogueResult>
Defined in: packages/agentos/src/io/speech/providers/ElevenLabsTextToSpeechProvider.ts:405
Voices a dialogue of several speakers in one request, through ElevenLabs' Text to Dialogue with timestamps, and returns the audio with each line's place in it.
The audio is the bytes ElevenLabs sent, decoded from base64 with no re-encode. Each voice segment comes back with the index of its input line and its start and end seconds, in the order the API gave them; no segments are merged. The character alignments come back when the answer has them.
The key and the base URL are the ones synthesize uses: the xi-api-key header from the provider's key
pool, where a key that answers with a spent quota hands over to the next key, each key once, and
config.baseUrl.
ElevenLabs' reference gives two limits the method leaves to the API: at most 10 unique voice ids per request, and, for reliable generation, at most 2,000 characters of text across all lines; longer requests can end early or return a validation error. Split a longer dialogue into requests.
Parameters
lines
readonly ElevenLabsDialogueLine[]
The lines in speaking order, each with its text and voice id.
options?
ElevenLabsDialogueOptions = {}
The model, the output format and an optional seed.
Returns
Promise<ElevenLabsDialogueResult>
The audio, its format, the voice segments and the character alignments when present.
Throws
Before any request, for an empty dialogue, a line without text or voice id, or a seed that is not a whole number from 0 to 4294967295.
Throws
When ElevenLabs answers with an error status (401 for a bad key, 422 for a request it refuses, 429 for a spent quota), with the status and the body.
Throws
When a successful answer has no audio_base64, one that decodes to no bytes, no well-formed
voice_segments (a line index outside the lines sent included), or an alignment that is present but malformed.
Example
const dialogue = await provider.synthesizeDialogue(
[
{ text: 'Can you hear me?', voiceId: 'voice-a' },
{ text: 'Loud and clear.', voiceId: 'voice-b' },
],
{ model: 'eleven_v4', outputFormat: 'mp3_44100_128', seed: 7 },
);
for (const segment of dialogue.segments) {
console.log(segment.lineIndex, segment.startSeconds, segment.endSeconds);
}