⚡ llmspeed

Feel how fast a local LLM runs before you buy the hardware.

Pick a GPU or Mac and a model, and watch the answer stream at the real speed, including the wait before the first token.

https://github.com/user-attachments/assets/534b4736-6b59-426e-bc0a-e93a196156b9

RTX 5090 vs Mac Studio M5 Ultra 512GB on GLM-5.3 Flash (Q8). The 5090 can't load it.

Features

• Real-time simulation: text streams at the predicted tokens/sec, after a realistic time to first token for your prompt length
• Race mode: run two setups side by side, e.g. a 5090 against a Mac Studio
• Memory fit: see whether a model fits at each quantization, needs partial offload to system RAM, or won't run at all
• 30 GPUs, Macs and CPUs × 15 popular models, across CUDA, Metal, ROCm, Vulkan and CPU
• Shareable links to any setup or race

How the numbers work

Every estimate is labeled with how much to trust it:

Label	Meaning
Measured	Benchmarked on this exact hardware, model and quantization
Derived	Interpolated from measured benchmarks of similar setups
Projected	Calculated from hardware specs, e.g. for announced but unreleased hardware

Sources. Benchmarks are collected and verified from r/LocalLLaMA, localscore.ai, llama.cpp discussions and more.

The model. When no benchmark exists, llmspeed falls back to first principles:

• Generation speed is bound by memory bandwidth: every token reads the active weights once. For mixture-of-experts models only the active parameters count.
• Prompt processing is bound by compute (FP16 TFLOPS).
• Both use per-backend efficiency coefficients calibrated against the measured benchmarks, plus a slowdown as context grows.
• Partial offload uses a hybrid model: the GPU-resident share streams at GPU bandwidth and the spilled share at an effective system-RAM bandwidth, calibrated from measured offload runs.
• Memory fit counts weights, KV cache and runtime overhead against usable memory: about 75% of unified memory on Macs and 93% of VRAM on discrete GPUs.

Contributing benchmarks

The most valuable contribution is a real measurement. To add one, open a PR that appends a row to benchmarks/submissions.csv.

1. Run llama.cpp's benchmark tool on your machine:

   llama-bench -m your-model.gguf -p 512 -n 128
   

2. Add a row with your results:

   Column	Example
   hardware	NVIDIA RTX 4090 24GB
   model	Llama 3.1 8B Instruct
   quant	Q4_K_M
   backend	CUDA, Metal, ROCm, Vulkan or CPU
   engine_version	llama.cpp b6500
   gen_tps	128.4 (the tg128 result)
   prefill_tps	9850 (the pp512 result)
   prompt_tokens	512
   source_url	a link to your post, issue or gist with the full output
   notes	anything unusual: power limits, flash attention, offload, …

3. In the PR description, paste the full llama-bench output.

Results from other engines (MLX, vLLM, ExLlama, …) are welcome too; just name the engine in engine_version. Want a model or GPU added? Open an issue.

Development

npm install
npm run dev      # local dev server
npm run build    # production build + prerendered landing pages + sitemap

Built with Vite, Preact and TypeScript. The site is fully static and deployed to GitHub Pages on every push to main. All predictions live in data/app-data.json, which is generated by the benchmark pipeline.

Limitations

• Single device only. Multi-GPU setups (tensor or layer split) aren't simulated yet.
• Estimates, not guarantees. Drivers, engine versions, flash attention, power limits and thermals can all move real numbers by 10–30%.
• Projected numbers for unreleased hardware will be replaced with measurements as they appear.

Star history

License

MIT. Benchmark data credits go to the r/LocalLLaMA and localscore.ai communities and everyone who has shared their numbers.