kekzl/imp has 83 GitHub stars and 7 forks. GitHot measured 0 stars in 24 hours and +8 in 7 days. Last checked 2026-11-15 06:51 UTC. Last push 2026-11-02.
Measured star growth
- in 24h
- 0
- in 7 days
- +8
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.
Self-hostedREADME.md
Original
Built for Blackwell. Nothing else.
RTX 5050 to 5090 · RTX PRO 2000 to 6000 · one docker run · OpenAI and Anthropic API · built-in chat · Linux or WSL2
imp runs AI chat models on your own NVIDIA graphics card (any GeForce RTX 50 or RTX PRO Blackwell). Chat apps and coding agents talk to it the way they talk to ChatGPT or Claude, but nothing leaves your machine. It is built for this one card generation only, and that is why it is fast. One container, nothing else to install, MIT license.
How fast
| Qwen3.8-27B on one RTX 5090, tokens/s | imp | vLLM 0.31.0 |
|---|---|---|
| one user | 95.5 | 70.4 |
| 32 users at once (total) | 1974.8 | 1380.1 |
Both engines read the same model file. A token is about ¾ of a word, so 95 tokens/s is roughly 70 words per second. Every model, method and raw log: docs/BENCHMARKS.md.
Bigger than the card: the 123.6 GiB Qwen3.8-Flash-Next answers at 91.8 tokens/s on a 32 GB card, its experts in host RAM (how).
Quickstart (Docker)
You need: an RTX 50 or RTX PRO Blackwell card, Linux or Windows with WSL2, Docker with the NVIDIA Container Toolkit. No CUDA toolkit, no Python.
Qwen3.8-27B, the default: text and images, fits a 24-32 GB card.
git clone https://github.com/kekzl/imp && cd imp
scripts/stage-model.sh kekzle/Qwen3.8-27B-NVFP4-vllm ~/models/Qwen3.8-27B-NVFP4-vllm # 1. download the model, 19.2 GiB
docker run --gpus all -v ~/models:/models -v imp-cache:/home/imp/.cache/imp \
-p 127.0.0.1:8080:8080 ghcr.io/kekzl/imp:latest --model /models/Qwen3.8-27B-NVFP4-vllm # 2. start
When the log says Server listening, open http://localhost:8080 and chat. On an RTX 5090 this setup answers at 96.3 tokens/s and uses 19.7 GiB of the card.
Qwen3.8-Flash-Next, the biggest: 123.6 GiB on a 32 GB card, needs ~67 GiB free RAM and 124 GiB of disk.
STAGE_DIR=~/models/Qwen3.8-Flash-Next-NVFP4 scripts/stage-model.sh nvidia/Qwen3.8-Flash-Next-NVFP4 ~/models/Qwen3.8-Flash-Next-NVFP4
docker run --gpus all -v ~/models:/models -v imp-cache:/home/imp/.cache/imp \
-p 127.0.0.1:8080:8080 ghcr.io/kekzl/imp:latest --model /models/Qwen3.8-Flash-Next-NVFP4
Ready after about 50 s. How a model bigger than the card runs: docs/BIG_MODELS.md.
Step by step, what the log should say, other models: docs/QUICKSTART.md. Build the image yourself or run without Docker: docs/BUILD.md.
Which model?
| You want | Take |
|---|---|
| not sure | Qwen3.8-27B (the first example above): reads text and images, fits a 24-32 GB card |
| fast answers on a 12-16 GB card | Qwen3-8B |
| code and agents, fast | Qwen3-Coder-30B-A3B |
| the biggest, with ~67 GiB free RAM | Qwen3.8-Flash-Next (how) |
imp is built for models in SafeTensors format (NVFP4 or FP8). Which models work and how big they are: docs/MODELS.md. Which card holds what: docs/QUICKSTART.md.
Using it
- In the browser: http://localhost:8080, a built-in chat with saved conversations and live speed.
- Your apps and coding agents (Claude Code, Codex, any OpenAI app): set the base URL to
http://localhost:8080/v1. - Tool calls, JSON output, thinking models, streaming:
docs/API.md. - Something went wrong?
docs/TROUBLESHOOTING.md, or open an issue with the server log.
What imp is not
- Not for other hardware: one GPU, NVIDIA Blackwell only. No multi-GPU, no CPU, no AMD: use vLLM for those.
- Not always ahead: vLLM is faster on Qwen3-14B with 32 users at once sending long prompts (~1000 tokens each: -4.2 %, 2026-10-31,
BENCHMARKS.md); with short prompts imp leads (+7.2 %).
- Not a company product: one maintainer, tested on one RTX 5090; issues are welcome, answers are not guaranteed.
More
| I want to … | Read |
|---|---|
| know how it works | docs/HOW_IT_WORKS.md |
| run it behind a proxy, with an API key | docs/DEPLOYMENT.md |
| see every setting | docs/CONFIG.md |
| build from source | docs/BUILD.md |
| contribute | CONTRIBUTING.md |
| browse all docs | docs/README.md |
Every line of imp was written by an AI coding agent (Claude Code); the audit trail is in docs/audit/.
License
MIT, see LICENSE. One kernel file is adapted from Apache-2.0 code (SageAttention); the APA prefill (src/compute/apa/) is kekzl/apa under MPL-2.0. Notices: THIRD_PARTY_LICENSES.md, shipped in the image at /usr/share/doc/imp/.