Skip to content
Projects

ypollak2/llm-router has 130 GitHub stars and 17 forks. GitHot measured +1 stars in 24 hours and +5 in 7 days. Last checked 2026-11-17 02:12 UTC. Last push 2026-11-15.

How these counts are measured

What you can do

  • Automatic fallback with circuit breakers. A provider that fails or rate-limits is skipped, not retried into the ground.
  • You can see it working. A status line, terminal title and OS notification show the last model routed, savings and health.
  • Session-end summary. Savings vs baseline, tier mix, per-provider cost, latency p50/p95/p99 and top routes.
  • Media and pipelines too. llm_image / llm_video / llm_audio, and llm_orchestrate for multi-step research.

Measured star growth

in 24h
+1
in 7 days
+5
in 15 days
Collecting history
in 1 month
Collecting history

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Universal LLM router for AI coding tools. Works with Claude Code, Cursor, Codex, Gemini CLI, Copilot and more. Free-first fallback chain cuts costs 35-80%.

AI

README.md

Original

llm-router

Stop spending your Claude Pro/Max quota on questions a free model can answer.

llm-router hooks into your coding tool's own lifecycle, reads each prompt before the model does, and drafts an answer on a free or local model first. No API keys needed on a Claude subscription — routing runs through MCP tools and local models.

One caveat: routed answers are advisory by default — Claude still takes the turn unless you opt in to zero-Claude mode. By default some tool calls wait until a prompt is routed; LLM_ROUTER_ENFORCE=off turns that off (all user settings: docs/configuration.md). Draft-rate and acceptance figures live in docs/MEASUREMENT.md, with their n, window and conditions.

PyPI PyPI Downloads Tests Stars Listed on RouterArena License

📑 Table of Contents

Why install this

You are on a Claude Pro or Max plan, have not spent a cent beyond it, and still hit the five-hour limit — not because you asked too much, but because every prompt went to the premium model: "what does this error mean", "reformat this JSON", "is the service up" drew the same quota as the question that needed it.

Why a proxy cannot do this

Most routers here are a proxy: your agent forwards requests using your API keys. That has a hard limit — a proxy cannot intercept a session authenticated by a subscription, because there is no key to forward.

Pays per token Pays a subscription
What runs out your invoice your five-hour window
Needs API keys yes no
A proxy can help yes no — nothing to intercept
llm-router helps yes yes

Honesty as a feature

llm-router status reports verified and unverified savings separately, each with its own n — not a blended percentage. This project does not publish a general savings percentage; any number depends on your own workload. See Savings below.


Quick Start

pip install llm-routing        # installs the `llm-router` command
llm-router install             # wire up Claude Code (default host)
llm-router doctor               # check provider connectivity and setup
llm-router status               # verify it's routing — est. savings vs baseline

Works with no API keys on a Claude Pro/Max subscription — routing goes through MCP tools and local models. Adding provider keys widens the pool; nothing requires them. See guide/PROVIDERS.md. To install into another host, see Works With. The daily KPI job is installed only with consent (install --schedule-daily, or Y when asked; --no-schedule skips); uninstall removes it.

Python agents route in-process, no server: from llm_router import Router; r = Router(budget_usd=0.50).run("...") (await .arun(...) in async code; BudgetExceeded when the cap is spent).


Works With

Host Install
Claude Code (default) llm-router install
Codex CLI llm-router install --host codex
OpenCode llm-router install --host opencode
Gemini CLI llm-router install --host gemini-cli
GitHub Copilot CLI llm-router install --host copilot-cli
OpenClaw llm-router install --host openclaw
Trae IDE llm-router install --host trae
Pi (pi.dev) (MCP server only; end-of-turn line planned in 16.1) llm-router install --host pi
Factory Droid llm-router install --host factory
Claude Desktop llm-router install --host desktop
VS Code (native MCP) llm-router install --host vscode
Cursor llm-router install --host cursor
GitHub Copilot in VS Code (capability extension, no cost-routing) llm-router install --host copilot
Windsurf / Cascade llm-router install --host windsurf
Kimi Code (Moonshot AI) planned (16.1); --host kimi reports why and exits 2

llm-router install --host all installs or prints every host config in one pass. Full per-host detail, including what each host genuinely cannot do: guide/HOST_SUPPORT_MATRIX.md.


How It Works

Hooks intercept the prompt before your coding tool's own model sees it. A free regex heuristic classifies it instantly, scoring about half of real prompts with confidence — 788 of 1,571 measured (scripts/measure_low_signal_rate.py, run 2026-09-23); the rest fall back to a default. A routable prompt gets a draft from a free/local model first, then walks up the chain toward paid models only if needed.

The draft is advisory — handed to Claude as an unverified hint, not a turn replacement, unless you opt in to zero-Claude mode. Enforcement acts at the tool-call level: smart (default) holds selected tool calls until the prompt is routed; off disables that. Full modes and per-host overrides: guide/GETTING_STARTED.md.


Features

  • A prompt with a secret goes to a local model or nowhere. When a routed call (the llm_* text tools, the gateway, streaming) contains an API key, token or private key, llm-router sends it only to Ollama models on a loopback daemon — direct, not through an HTTP proxy, with no semantic cache, judge queue or cloud fallback — or refuses it with the secret's category and how to proceed (cloud: prefix to override one request). Placeholders such as AKIA...EXAMPLE or postgres:postgres@ are not refused. Not every door enforces this: the router, gateway, SDK, llm_stream, llm_codex, llm_gemini and the agent-route hook do; the Claude Code proxy (logs only), the opt-in auto-route cloud classifiers, the hook's direct-execution Gemini/OpenAI calls, agent delegation and other opt-in features do not, and what a host already sent to its own model is out of reach. The per-door table, details and limits.
  • Automatic fallback with circuit breakers. A provider that fails or rate-limits is skipped, not retried into the ground.
  • You can see it working. A status line, terminal title and OS notification show the last model routed, savings and health.
  • Session-end summary. Savings vs baseline, tier mix, per-provider cost, latency p50/p95/p99 and top routes.
  • Media and pipelines too. llm_image / llm_video / llm_audio, and llm_orchestrate for multi-step research.

Reference

The default MCP surface shows 12 front-door tools (the CONSOLIDATED_TOOLS set in src/llm_router/tool_surface.py; tests/test_doc_tool_counts_match_registration.py checks this count against it); an opt-out shows everything registered (see docs/configuration.md for user settings).

Topic What Guide
CLI install, status, gain, doctor, okf index/status, sessions status and more guide/GETTING_STARTED.md
Providers Free-first — Ollama (local), OpenRouter, Gemini, Groq, your Claude subscription guide/PROVIDERS.md
Routing Policies conservative → balanced (default) → cost_aggressive; thresholds and YAML schema guide/POLICIES.md
MCP Tools Every tool with its signature guide/TOOLS.md
llm-router gain today           # savings so far today
llm-router okf status           # knowledge base freshness
llm-router sessions status      # per-session summary

Savings

Savings figures are a counterfactual — what the same tokens would have cost at API list price, against what was actually spent — not money saved on a flat subscription. llm-router status and llm-router savings-report split verified from unverified savings, each with its own n. Methodology, assumptions and limitations: docs/MEASUREMENT.md.


Trust and Security

llm-router runs entirely on your machine — no hosted proxy, nothing sent to an llm-router service, no account required. A grounding check (src/llm_router/grounding.py) discards any draft citing a file or function absent from its context and the indexed repo, falling through to Claude.

LLM_ROUTER_DIRECT_EXECUTION is default on — a local model gets write_file, edit_file and run_command under a confined, allowlisted path, and none of that stops targeted deletes, git push --force, or exfiltration. Shell redirects and copies that write outside the project root are refused by a static tripwire, not a sandbox; an OS sandbox is 16.1. A command the tripwire cannot judge (an interpreter one-liner, sudo, a $(...)) returns needs_approval, but approval is not wired anywhere yet, so both command paths treat it as refused and do not run it. Turn it off with LLM_ROUTER_DIRECT_EXECUTION=false. Full analysis: SECURITY.md.

Self-audits are published in audit/ and docs/repo_goals/AUDIT-2026-09-25.md (on 2026-09-25, of 20 goals: 13 met, 6 partial, 1 not met), including negative results.

Ground Truth accumulation (opt-in) builds a corpus of replayable routing decisions instead of unlinked telemetry; off by default since it's the only part that writes prompt text to disk. Details: guide/GROUND_TRUTH.md.


Documentation

Full index: guide/README.md

Document Purpose
Quick Start Fastest path to working routing
Getting Started Full setup walkthrough
Host Support Matrix Per-host feature comparison
Providers Provider setup and model recommendations
Routing Policies routing.yaml schema and authoring your own policy
Tool Reference All MCP tools with examples
Architecture Internal design and module structure
Troubleshooting Common issues and fixes
Testing the Router Isolation suite for verifying routing health
Measurement What "savings" means and how it's computed
RouterArena Benchmark methodology, results and negative findings
Ground Truth Replayable routing corpus, opt-in
Benchmarks Model cost/latency/quality table, regenerated by CI
Changelog Release notes (archive)

llm-router is scored on the RouterArena benchmark, graded locally with this repo's harness and not yet independently verified. Results with their n and source files, methodology and negative results (including a proxy split that misled tuning): docs/ROUTERARENA.md.


Enterprise, Contributing, License

llm-router is built for individual developers and small teams: local cost savings, zero ops overhead, no hosted anything. For team-wide policy enforcement, audit export, SSO or per-org budgets, see Chuzom — a separate, sibling product.

Contributions welcome — see CONTRIBUTING.md.

git clone https://github.com/ypollak2/llm-router.git
cd llm-router
uv sync --extra dev
uv run pytest tests/ -q         # Run tests
uv run ruff check src/ tests/   # Lint

MIT License. Issues · Discussions · PyPI · Changelog