Skip to content
Projects

vertesia/llumiverse has 30 GitHub stars and 6 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-16 18:47 UTC. Last push 2026-11-14.

How these counts are measured

Measured star growth

in 24h
0
in 7 days
Collecting history
in 15 days
Collecting history
in 1 month
Collecting history

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Universal connector to LLMs for Node.js & Bun

AI

README.md

Original

Llumiverse - The Universal, Lightweight LLM Driver

Build npm version License

Llumiverse is the unified connectivity layer for the Large Language Model ecosystem. It provides a robust, modular, and strongly-typed interface for interacting with almost any AI provider in Node.js, Bun, and TypeScript environments.

Think of it as the "JDBC" or "ODBC" for LLMs.

It solely focuses on abstracting the connection and execution protocol, letting you switch providers with zero code changes. It does not enforce prompt templating, rigid chain structures, or "magic" orchestration logic. You build the application; we handle the plumbing.

Why Llumiverse?

  • 🚫 No Vendor Lock-in: Switch from OpenAI to Vertex AI to Bedrock in minutes.
  • 🧩 Modular & Lightweight: Install only what you need. Zero bloat.
  • πŸ›‘οΈ Type-Safe: First-class TypeScript support with normalized interfaces for completion, streaming, and tool use.
  • ⚑ Universal Support: Works in Node.js, serverless functions, and Bun.

Supported Platforms

Provider Completion Chat Streaming Multimodal Tool Calling Embeddings
AWS Bedrock βœ… βœ… βœ… βœ… βœ… βœ…
Microsoft Foundry βœ… βœ… βœ… βœ… βœ… βœ…
Azure OpenAI βœ… βœ… βœ… βœ… βœ… βœ…
Google Vertex AI βœ… βœ… βœ… βœ… βœ… βœ…
Groq βœ… βœ… βœ… N/A βœ… N/A
HuggingFace βœ… βœ… βœ… N/A N/A N/A
IBM WatsonX βœ… βœ… βœ… N/A N/A βœ…
Mistral AI βœ… βœ… βœ… N/A βœ… βœ…
OpenAI βœ… βœ… βœ… βœ… βœ… βœ…
OpenAI Compatible βœ… βœ… βœ… βœ… βœ… βœ…
Replicate βœ… βœ… βœ… N/A N/A N/A
Together AI βœ… βœ… βœ… N/A N/A N/A
xAI (Grok) βœ… βœ… βœ… βœ… βœ… βœ…

Llumiverse vs. The Rest

Vs. LangChain / LlamaIndex: These are application frameworks that provide high-level abstractions for chains, memory, and RAG. Llumiverse is a driver library. It operates at a lower level.

  • Use Llumiverse if: You want full control over your prompt logic, you want to build your own custom framework, or you simply need a clean, unified way to call an API without the weight of a massive dependency tree.
  • Use LangChain if: You want pre-built "chains" and are okay with higher complexity and abstraction overhead.

Vs. Vercel AI SDK: Vercel's SDK is fantastic for frontend/full-stack React/Next.js applications.

  • Use Llumiverse if: You are building backend services, background workers, CLI tools, or generic Node.js applications where frontend-specific hooks aren't relevant. Llumiverse is environment-agnostic.

Requirements

  • Node.js 24.x
  • The OpenAI driver uses OpenAI SDK 7.25.0, whose minimum supported Node.js version is 22.
  • Bun 1.0+ (experimental support)

Microsoft Foundry and Azure OpenAI SDKs

Microsoft Foundry (the azure_foundry provider) uses @azure/ai-projects for project/deployment discovery, OpenAI SDK 7 for OpenAI-compatible inference, and @anthropic-ai/foundry-sdk for Anthropic deployments. Claude Messages requests use the resource /anthropic/v1/messages endpoint. Claude source model IDs select the shared Anthropic options and behavior; the deployment name is sent in the API request. Chat Completions, Responses, embeddings, and image requests use the OpenAI v1 endpoint resolved by Projects. Projects retains its own declared OpenAI dependency; no dependency override is required. The retired Azure AI Inference beta transport is no longer used. Deployments must support OpenAI v1 and the requested modality; image embedding availability remains deployment-specific. Azure Responses image tools use v1 requests and a blocking streaming fallback. Microsoft currently documents image streaming and multi-turn image editing as unsupported. These callable paths do not establish Azure parity with OpenAI iterative image editing.

Foundry authenticates with Microsoft Entra ID (azureADTokenProvider, or DefaultAzureCredential) by default. Set apiKey to authenticate inference with a resource key instead: OpenAI-compatible requests then go to the resource's /openai/v1 endpoint with the api-key header, and Claude requests send x-api-key. Claude Mythos accepts Entra ID only. Listing deployments needs Entra ID, so pass azureADTokenProvider as well for listModels(); with a key alone, address models as deployment::model (or set sourceModel) so the driver can pick the protocol without reading the deployment.

The Azure OpenAI driver (azure_openai) uses the GA v1 API (<endpoint>/openai/v1, no api-version). Setting apiVersion keeps the dated /openai/...?api-version= API. Its service is therefore typed as either client (AzureOpenAIService). An injected AzureOpenAI client keeps the AzureOpenAI type, and a pinned version can select it explicitly: new AzureOpenAIDriver<AzureOpenAI>({ endpoint, apiVersion: '2024-10-21' }). Selecting AzureOpenAIV1Client requires an endpoint and no apiVersion.

Installation

For most use cases (recommended):

pnpm add @llumiverse/core @llumiverse/drivers
  • @llumiverse/core: The interfaces, types, and base abstractions.
  • @llumiverse/drivers: The concrete implementations for all supported providers.

Quick Start

1. Initialize a Driver

Each driver takes normalized options but may require specific credentials (API keys, regions, project IDs).

// OpenAI
import { OpenAIDriver } from "@llumiverse/drivers";
const driver = new OpenAIDriver({ apiKey: process.env.OPENAI_API_KEY });

// Google Vertex AI
import { VertexAIDriver } from "@llumiverse/drivers";
const driver = new VertexAIDriver({
    project: process.env.GOOGLE_PROJECT_ID,
    region: 'us-central1'
});

// AWS Bedrock (uses local AWS credentials automatically)
import { BedrockDriver } from "@llumiverse/drivers";
const driver = new BedrockDriver({ region: 'us-west-2' });

// ... and many more!

2. Execute a Prompt (Standard)

The execution API is normalized. You send a PromptSegment[] and get a Completion object, regardless of the provider.

import { PromptRole } from "@llumiverse/core";

// 1. Create a generic prompt
const prompt = [
    {
        role: PromptRole.system,
        content: "You are a helpful coding assistant."
    },
    {
        role: PromptRole.user,
        content: "Write a hello world in Python."
    }
];

// 2. Execute against ANY driver
const response = await driver.execute(prompt, {
    model: "gpt-4o", // or "gemini-1.5-pro", "anthropic.claude-3-sonnet-20240229-v1:0", etc.
    temperature: 0.7,
    max_tokens: 1024
});

console.log(response.result[0].value);

3. Stream a Response

Streaming is standardized into an AsyncIterable.

const stream = await driver.stream(prompt, {
    model: "gpt-4o",
    temperature: 0.7
});

for await (const chunk of stream) {
    process.stdout.write(chunk);
}

// Access the full assembled response afterwards if needed
console.log("\nFull response:", stream.completion.result[0].value);

4. Generate Embeddings

Embeddings are normalized to a simple { values: number[] } structure.

const embedding = await driver.generateEmbeddings({
    content: "Llumiverse is awesome."
});

console.log(embedding.values); // [0.012, -0.34, ...]

Advanced Features

Tool Calling (Function Calling)

Llumiverse normalizes tool definitions across providers (OpenAI, Bedrock, Vertex, etc.), allowing you to define tools once and use them anywhere.

import { ToolDefinition } from "@llumiverse/core";

// 1. Define the tool
const getWeatherTool: ToolDefinition = {
    name: "get_weather",
    description: "Get the current weather in a given location",
    input_schema: {
        type: "object",
        properties: {
            location: { type: "string" },
            unit: { type: "string", enum: ["celsius", "fahrenheit"] }
        },
        required: ["location"]
    }
};

// 2. Execute with tools
const response = await driver.execute(prompt, {
    model: "gpt-4o",
    tools: [getWeatherTool]
});

// 3. Handle the tool use
if (response.tool_use) {
    for (const tool of response.tool_use) {
        console.log(`Executing ${tool.tool_name} with args:`, tool.tool_input);
        // ... call your actual function ...
    }
}

Structured Outputs (JSON Schemas)

Enforce a specific JSON structure for the response. Llumiverse handles the schema conversion for the underlying provider (e.g., using response_format for OpenAI or equivalent techniques for others).

const schema = {
    type: "object",
    properties: {
        sentiment: { type: "string", enum: ["positive", "neutral", "negative"] },
        confidence: { type: "number" }
    },
    required: ["sentiment", "confidence"]
};

const response = await driver.execute(prompt, {
    model: "gpt-4o",
    result_schema: schema
});

// The result is automatically parsed
const data = response.result[0].value;
console.log(data.sentiment); // "positive"

Testing

Tests fall into three tiers, chosen by file name and by the allow list in .github/workflows/build-and-test.yml:

Tier Files Calls providers Runs in CI
Functional *.test.ts No: provider calls are mocked Yes, all of them
Live *.live.test.ts Yes, using your credentials No; run on demand
Live in CI *.live.test.ts listed in the CI allow list Yes Yes, on every push and pull request

A test that calls a provider must live in a *.live.test.ts file. Live files in CI are the curated smoke set: model-smoke, conversation, tools and embeddings (all under drivers/test/). Other live files cover debugging or slower checks, such as max-tokens-limit.live.test.ts, which sends each smoke model a request at its documented output limit. To add a live file to CI, add its path to the allow list.

# Functional tests: no credentials needed
pnpm exec vitest run --exclude '**/*.live.test.ts'

# One live file, narrowed to a provider or model
LLUMIVERSE_LIVE_PROVIDERS=google-vertex pnpm exec vitest run drivers/test/model-smoke.live.test.ts

# On-demand GPT-6 / GPT-6.1 schemas, streaming, tool calls, and persisted reasoning replay
LLUMIVERSE_LIVE_PROVIDERS=openai LLUMIVERSE_LIVE_MODELS=gpt-6.1-terra \
pnpm exec vitest run drivers/test/gpt-6.live.test.ts

The GPT compatibility file also accepts bedrock and bedrock-mantle as provider selectors. Use their provider-specific model IDs when setting LLUMIVERSE_LIVE_MODELS. The suite exercises the streaming API, which can use the driver's blocking fallback when streaming discovery reports that native streaming is unavailable.

Live tests run only for providers whose credentials are set, for example OPENAI_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY, TOGETHER_API_KEY, GROQ_API_KEY, XAI_API_KEY, OPENROUTER_API_KEY, WATSONX_API_KEY (with WATSONX_PROJECT_ID and WATSONX_ENDPOINT_URL), AZURE_OPENAI_KEY (with AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_DEPLOYMENT), AZURE_FOUNDRY_ENDPOINT (with AZURE_FOUNDRY_MODELS), BEDROCK_REGION, BEDROCK_MANTLE_REGION, and GOOGLE_PROJECT_ID with GOOGLE_REGION. Tests for other providers are skipped.

Microsoft Foundry live tests (azure-foundry) read:

Variable Value
AZURE_FOUNDRY_ENDPOINT The project endpoint, https://<resource>.services.ai.azure.com/api/projects/<project>
AZURE_FOUNDRY_MODELS Comma-separated deployment::model IDs, for example gpt-6-luna::gpt-6-luna,haiku::claude-haiku-5-5
AZURE_FOUNDRY_API_KEY Optional resource key. Without it, DefaultAzureCredential provides a Microsoft Entra ID token

With Entra ID, the identity needs Cognitive Services User on the resource for Claude and partner models (Cognitive Services OpenAI User covers OpenAI models only). Without AZURE_FOUNDRY_MODELS the provider is skipped. CI sets none of these yet.

Contributing

We welcome contributions! Whether it's a new driver, a bug fix, or a docs improvement. Please see CONTRIBUTING.md for details.

License

Apache 2.0

Coordinated submodule updates

Add the skip-submodule-sync label to a PR before merging when a coordinated PR in composableai already handles the submodule update. The automatic sync checks the merged PR's label before creating or updating a downstream sync PR. CI still runs and failure notifications remain enabled.

The opt-out applies only to that merge commit and target branch (main or release/*); later unlabeled merges can trigger sync again. A manually dispatched downstream submodule update bypasses the label. Adding the label after a sync has already started does not cancel that sync.

Build and unit-test caching

pnpm build, pnpm typecheck:test, and pnpm test:unit use Turbo caching. test:unit excludes *.live.test.ts; pnpm test retains its full-suite behavior and its test results are not cached. CI runs the unit suite before provider authentication and runs the curated live provider suite separately on every run.

Local commands read and write local cache entries and only read the remote cache. Set TURBO_CACHE_OPTS=--cache=local:rw to use only local caching. The build/test workflow enables remote writes and authenticates with GitHub OIDC. Configure a Turborepo CLI OIDC policy for this repository on the Vercel team. The team comes from the TURBO_TEAM organization variable (Vertesia). No TURBO_TOKEN secret is needed. Unit-test hashes include the CI runner OS, architecture, and image version to separate them from local runs.

Coordinated publishing

The native publish-npm.yaml workflow accepts optional request_id, source_sha, and exact version inputs. Snapshot versions use X.Y.Z-dev.YYYYMMDD.HHMMSSZ on main; stable versions use release/X.Y. The exact version must match the selected base and bump. Registry credentials remain in the existing build environment.

Successful real runs upload sdk-publish-source/sdk-publish-source.json containing version, input source_sha, and the committed release_sha. A downstream publisher can consume that exact commit directly and include its gitlink in its version commit. Coordinated version commits carry Coordinated-Release: true, suppressing only their automatic submodule-sync notification; ordinary source commits still notify consumers.

Release retries use immutable vX.Y.Z or vX.Y.Z-dev.YYYYMMDD.HHMMSSZ source tags. Before any Git or registry write, the publisher checks source membership on the selected branch and ownership of every existing package version. Package metadata records releaseIdentity (source commit, dependency pin, and lockfile hash). After publishing, the tagged version commit is merged into the current branch tip; ordinary commits that land during the build are preserved. A rejected push retries against the new tip. Genuine version conflicts stop with the durable tag available for recovery, without changing the artifact's source identity.