Skip to content
Projects

Jellypod-Inc/speech-sdk has 52 GitHub stars and 13 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-16 04:08 UTC. Last push 2026-10-29.

How these counts are measured

What you can do

  • Universal — one generateSpeech() call across every supported provider.
  • Streaming — streamSpeech() returns a standard ReadableStream<Uint8Array>.
  • Conversations — generateConversation() produces multi-speaker audio from one or many providers.
  • Word-level timestamps — native where available, forced alignment otherwise, plus SRT/WebVTT captions.

Measured star growth

in 24h
0
in 7 days
Update delayed
in 15 days
Update delayed
in 1 month
Update delayed

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Universal Text-To-Speech TypeScript SDK with Multi-Provider Support.

Creative & Media

README.md

Original
Speech SDK

Speech SDK

The Universal SDK for text-to-speech across all major providers

A lightweight, provider-agnostic TypeScript SDK. Zero lock-in. Runs in Node.js, Edge runtimes, and the browser.

Created and maintained by Jellypod.

npm version npm downloads license Discord Stars

Docs · Quick start · Providers · Streaming · Conversations · Timestamps


Speech SDK

Features

  • Universal — one generateSpeech() call across every supported provider.
  • Streaming — streamSpeech() returns a standard ReadableStream<Uint8Array>.
  • Conversations — generateConversation() produces multi-speaker audio from one or many providers.
  • Word-level timestamps — native where available, forced alignment otherwise, plus SRT/WebVTT captions.
  • Audio tags, pronunciations, voice cloning & voice design — handled per provider behind one API.
  • Output control — volume normalization and wav / mp3 / pcm output.

Install

npm install @speech-sdk/core

[!TIP] Using an AI coding assistant? Give it full knowledge of this library: npx skills add Jellypod-Inc/speech-sdk --skill speech-sdk.

Quick start

import { generateSpeech } from '@speech-sdk/core';

const result = await generateSpeech({
  model: 'openai/gpt-4o-mini-tts',
  text: 'Hello from speech-sdk!',
  voice: 'alloy',
});

result.audio.uint8Array;  // Uint8Array
result.audio.base64;      // string (lazy)
result.audio.mediaType;   // "audio/mpeg"

Set the provider's env var (e.g. OPENAI_API_KEY) or pass apiKey. A bare provider id ('openai') uses its default model.

Need a custom baseURL, fetch, or several configurations of one provider? Use a factory instead of a string:

import { createOpenAI } from '@speech-sdk/core/providers';

const openai = createOpenAI({ apiKey: 'sk-...', baseURL: 'https://my-proxy.com/v1' });

await generateSpeech({ model: openai('gpt-4o-mini-tts'), text: 'Hello!', voice: 'alloy' });

Use instructions for non-spoken delivery direction ('warm and measured') on models that support it. Provider-specific parameters pass through untouched via providerOptions.

Supported providers

Provider Prefix Env var
OpenAI openai OPENAI_API_KEY
ElevenLabs elevenlabs ELEVENLABS_API_KEY
Google Gemini TTS google GOOGLE_API_KEY
Cartesia cartesia CARTESIA_API_KEY
Deepgram deepgram DEEPGRAM_API_KEY
HeyGen heygen HEYGEN_API_KEY
Hume hume HUME_API_KEY
Inworld inworld INWORLD_API_KEY
Fish Audio fish-audio FISH_AUDIO_API_KEY
Gradium gradium GRADIUM_API_KEY
Murf murf MURF_API_KEY
Resemble resemble RESEMBLE_API_KEY
fal fal-ai FAL_API_KEY
Mistral mistral MISTRAL_API_KEY
xAI xai XAI_API_KEY
MiniMax minimax MINIMAX_API_KEY
Speechify speechify SPEECHIFY_API_KEY

Providers can also route through Vercel AI Gateway with createVercelAIGatewayClient(). See the docs for per-provider models, voices, and caveats.

Streaming

import { streamSpeech } from '@speech-sdk/core';

const { audio, mediaType } = await streamSpeech({
  model: 'cartesia/sonic-3.5',
  text: 'Streaming straight to the client.',
  voice: 'voice-id',
});

return new Response(audio, { headers: { 'Content-Type': mediaType } });

streamConversation() streams native two-speaker dialogue on models that support it (e.g. Gemini 3.8).

Conversations

generateConversation() turns an ordered list of turns into one clip. Turns on a single multi-speaker model use its native dialogue endpoint; anything else is generated in parallel, level-matched, and stitched.

import { generateConversation } from '@speech-sdk/core';

const result = await generateConversation({
  turns: [
    { model: 'openai/tts-1',                      voice: 'nova',                 text: "Hi, I'm hosted by OpenAI." },
    { model: 'elevenlabs/eleven_multilingual_v2', voice: 'JBFqnCBsd6RMkjVDRZzb', text: "And I'm hosted by ElevenLabs." },
  ],
});

result.metadata.path; // 'native' | 'native-split' | 'stitch'

Pass timestamps: true, splitTurns: true to also get one sample-accurate clip per turn in result.turns.

Timestamps

const result = await generateSpeech({
  model: 'elevenlabs/eleven_multilingual_v2',
  text: 'Hello from speech-sdk!',
  voice: 'JBFqnCBsd6RMkjVDRZzb',
  timestamps: true,
});

result.timestamps; // [{ text: 'Hello', start: 0.0, end: 0.32 }, ...] (seconds)

Models without native timestamps need a timestampProvider, such as createElevenLabs().forcedAlignment(). Timestamps never fail synthesis; metadata.timestampsSource reports 'native', 'aligned', or 'estimated'.

Convert them to captions with timestampsToCaptions(result.timestamps, { format: 'vtt' }) (SRT by default).

More features

await generateSpeech({
  model: 'elevenlabs/eleven_v3',
  voice: 'voice-id',
  text: '[laugh] What is LLM?',                                        // audio tags
  pronunciations: { rules: [{ word: 'LLM', replacement: 'el el em' }] }, // pronunciations
  volumeDbfs: -20,                                                     // loudness normalization
  output: { format: 'mp3', bitrate: 96 },                              // wav | mp3 | pcm
});
  • Audio tags — [tag] cues are passed natively, mapped, or stripped with a warning. Tags a voice reads aloud are detected and cut automatically.
  • Voice cloning — pass voice: { audio: base64 } or voice: { url } on providers that support it.
  • Voice design — designVoice({ provider, name, description }) returns a reusable voiceId.

Error handling

import { generateSpeech, SpeechSdkProviderError } from '@speech-sdk/core';

try {
  await generateSpeech({ /* ... */ });
} catch (error) {
  if (error instanceof SpeechSdkProviderError) {
    error.status;    // 401, 429, 500, ...
    error.code;      // stable provider error code
    error.retryable;
  }
}

5xx, 429, and network errors are retried with jittered exponential backoff (honoring Retry-After), 2 retries by default; override with maxRetries. All errors extend SpeechSDKError.

Development

pnpm install
pnpm test              # unit tests
pnpm run test:e2e      # e2e tests (requires provider API keys)
pnpm run typecheck
pnpm fix               # format + lint

Created and maintained by Jellypod.