starlinghq/starling
starlinghq/starling has 169 GitHub stars and 15 forks. GitHot measured +43 stars in 24 hours. Last checked 2026-11-18 09:14 UTC. Last push 2026-11-15.
Measured star growth
- in 24h
- +43
- in 7 days
- Collecting history
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Local AI in .NET, with no Python sidecar and no API key: embeddings, entity extraction, image and video search and chat on ONNX Runtime (CPU/CUDA), through Microsoft.Extensions.AI.
AIREADME.md
OriginalStarling
Local AI in .NET, with no Python sidecar and no API key.

Starling packages open models as ordinary NuGet libraries: embeddings, entity extraction, image and video search, and chat. They run on ONNX Runtime on CPU or CUDA and plug into Microsoft.Extensions.AI (IEmbeddingGenerator, IChatClient), so code you wrote against a hosted provider can move to a local model by changing one line at the call site (moving from a hosted provider walks through the whole move). Embeddings still need re-indexing, and a small local model is not a large hosted one. Each model is pinned to an exact upstream commit, verified by SHA-256, and checked against the original PyTorch/Python implementation before it is released.
Try it in your browser: multilingual search with nothing to install. It downloads a 182 MB model once, and what you type stays on the page.
⭐ If this is useful, star the repository so other .NET developers can find it.
Using an AI coding agent? Point it at llms.txt or install the Starling skill.
What you can build
Three complete programs in ai-dotnet/samples/. Each downloads its models once, then runs with the network off. The output below is from a run on Linux x64 CPU with no network interface, on the published 0.5.0 packages.
Ask a folder of docs
LocalRag answers from your own Markdown and text files, using EmbeddingGemma 2 and Qwen3.5 through Microsoft.Extensions.AI.
cd ai-dotnet/samples/LocalRag
dotnet run -c Release -- download # about 4 GB, once
dotnet run -c Release -- ask --docs docs "When is the Willow Bookshop open?"
Indexed 4 paragraphs.
[1] bookshop.md paragraph 1 (cosine 0.9016)
[2] travel.txt paragraph 1 (cosine 0.7852)
[3] bookshop.md paragraph 3 (cosine 0.7462)
Question: When is the Willow Bookshop open?
Answer: The Willow Bookshop is open Tuesday through Saturday from 10:00 to 18:00.
It's a small-folder demonstration: it re-embeds every paragraph on each question, and a 0.8B model can be wrong, so it prints its sources.
Triage tickets in any language
TicketTriage trains a router on English tickets in an ordinary ML.NET pipeline, then routes German, Spanish, French and Japanese tickets with the same model. GLiNER2 pulls out the entities.
cd ai-dotnet/samples/TicketTriage
dotnet run -c Release -- download # about 1.3 GB, once
dotnet run -c Release -- train
dotnet run -c Release -- triage "Mon colis est arrivé ouvert et il manque un article."
ticket: Mon colis est arrivé ouvert et il manque un article.
queue: shipping 56 %, then returns 27 %
entities: product: colis
similar: [shipping] My order was split into two shipments and only one has arrived. (cosine 0.902)
Search photos with text
EmbeddingGemma2Search puts text and images in one embedding space, so a text query can rank a folder of photos.
cd ai-dotnet/samples/EmbeddingGemma2Search
dotnet run -- download-images # about 2.9 GB, once
dotnet run -- images --images <your photos> "a photo of a cat"
image query: a photo of a cat
top: chelsea.png (0.7051); next: astronaut.png (0.5782)
Vision takes about 4.5 s per photo on a 12-thread CPU, so start with a small folder or use CUDA.
Quickstart
Requires the .NET 10 SDK. Multilingual semantic search in four commands:
dotnet new console -o search && cd search
dotnet add package Starlings.Models.E5.Extensions.AI
dotnet add package Microsoft.ML.OnnxRuntime --version 1.30.0
Replace Program.cs with:
using System.Numerics.Tensors;
using Microsoft.Extensions.AI;
using Starlings.Models.E5;
using Starlings.Models.E5.Extensions.AI;
// One-time download, pinned by commit and checked by SHA-256. Inference never touches the network.
E5ModelFiles files = await E5ModelAssets.DownloadAsync();
using var embedder = E5EmbeddingGenerator.Create(files);
// e5 expects "passage: " on documents and "query: " on queries.
string[] docs = ["passage: The Eiffel Tower is in Paris.",
"passage: Der Rhein fließt durch Köln.",
"passage: 富士山は日本一高い山です。"];
var docVectors = await embedder.GenerateAsync(docs);
var query = await embedder.GenerateVectorAsync("query: Which river runs through Cologne?");
foreach (var (doc, v) in docs.Zip(docVectors))
Console.WriteLine($"{TensorPrimitives.CosineSimilarity(query.Span, v.Vector.Span):F3} {doc}");
dotnet run
0.680 passage: The Eiffel Tower is in Paris.
0.807 passage: Der Rhein fließt durch Köln.
0.688 passage: 富士山は日本一高い山です。
The first run downloads the model (about 487 MB) into your user cache; later runs work offline. The application chooses the ONNX Runtime native package: Microsoft.ML.OnnxRuntime for CPU, or Microsoft.ML.OnnxRuntime.Gpu.Linux for CUDA on Linux x64. If this or a later step fails, troubleshooting collects the errors people hit on first runs, with the fix for each.
Local chat through IChatClient
dotnet add package Starlings.Models.Qwen35.Extensions.AI
dotnet add package Microsoft.ML.OnnxRuntimeGenAI --version 0.17.1
using Microsoft.Extensions.AI;
using Starlings.Models.Qwen35;
using Starlings.Models.Qwen35.Extensions.AI;
Qwen35ModelFiles files = await Qwen35ModelAssets.DownloadAsync(Qwen35Variant.Fp32x); // about 3 GB, once
using IChatClient chat = Qwen35ChatClient.Create(files);
await foreach (var update in chat.GetStreamingResponseAsync("In one sentence, why is the sky blue?"))
Console.Write(update.Text);
Ten complete programs are in ai-dotnet/samples/: semantic search with e5 (E5Search), with an in-memory Microsoft.Extensions.VectorData collection (E5VectorStore) and with a reranking second stage (E5RerankSearch, Experimental), text, image and video search with EmbeddingGemma 2 (EmbeddingGemma2Search), entity extraction with GLiNER2 (Gliner2Ner), streaming chat with Qwen3.5 (QwenChat), local question answering over your own files (LocalRag), multilingual ticket triage in an ML.NET pipeline (TicketTriage), and e5 search in the browser from Blazor WebAssembly (E5BrowserSearch) and from a JavaScript page (E5BrowserJs), both on the Experimental browser packages.
Models
Choosing a text embedder? See choosing an embedding model.
| Model | What it does | Package | CPU | CUDA |
|---|---|---|---|---|
| multilingual-e5-small | Text embeddings, 100 languages | Starlings.Models.E5 (+ .Extensions.AI, .MLNet) |
Stable | Stable |
| EmbeddingGemma 2 | Text, image and video embeddings in one space | Starlings.Models.EmbeddingGemma2 (+ .Extensions.AI, .MLNet, .Imaging.SkiaSharp, .Video.FFmpeg, .Video.FFmpeg9) |
Stable (text, image, video) | Stable (text, image, video) |
| GLiNER2 | Zero-shot entity extraction with any labels | Starlings.Models.Gliner2 |
Stable | Stable |
| Qwen3.5-0.8B | Chat and text generation | Starlings.Models.Qwen35.OnnxGenAI (+ .Extensions.AI; model definition Starlings.Models.Qwen35) |
Stable | Stable (fp32x and fp16x) |
Experimental, on nuget.org from 0.3.0: Starlings.Models.Kev (Kev-0.8B and Kev-4B decision models, CPU and CUDA) and Starlings.Models.QwenChat (Qwen3-0.6B and Qwen2.5-0.5B-Instruct text chat, CPU) have no Stable profile; their model files download from public hosting (release record). Starlings.Models.Bert is on nuget.org from 0.4.0 with Experimental profiles only. Starlings.Models.Whisper builds and tests on main but is not published; build it from this repository.
| Model | What it does | Project | Status |
|---|---|---|---|
| Kev-0.8B, Kev-4B | Decision models: a calibrated probability per option, no generated text | Starlings.Models.Kev.Onnx (model definition Starlings.Models.Kev) |
Experimental, CPU and CUDA fp32; four registry profiles (kev-0.8b, kev-4b: onnx-fp32-cpu, onnx-fp32-cuda) |
| all-MiniLM-L6-v2, bge-small-en-v1.5, bge-base-en-v1.5, nomic-embed-text-v1.5, paraphrase-multilingual-MiniLM-L12-v2, ms-marco-MiniLM-L6-v2, gte-reranker-modernbert-base | BERT text embeddings and reranking | Starlings.Models.Bert |
Experimental, CPU fp32; proposed evidence only. paraphrase-multilingual-MiniLM-L12-v2 and gte-reranker-modernbert-base are not in the 0.5.0 package |
| Qwen3-0.6B, Qwen2.5-0.5B-Instruct | Text-only chat | Starlings.Models.QwenChat.OnnxGenAI (model definition Starlings.Models.QwenChat) |
Experimental, CPU fp32; proposed evidence only |
| whisper-large-v3-turbo | Speech-to-text: 30 s of 16 kHz audio to an English transcript (greedy) | Starlings.Models.Whisper |
Experimental, CPU fp16 graphs, source only; one registry profile (whisper-large-v3-turbo/onnx-fp16-cpu), no release gate run |
Their candidate contracts are in model-pipeline/candidates/ and the BERT and Qwen chat onboarding workflow is in model-pipeline/FIVE_MODELS.md. The QwenChat sample uses the published Qwen3.5 packages, not Starlings.Models.QwenChat.
Stable means every release gate passed: provenance, numerical parity and task quality against the Python reference, a one-hour leak soak, benchmarks, a clean install from the packages, and integration. It also means regression runs continue: weekly on CPU, and on a GPU before every release for CUDA. Stable on CUDA means Linux x64 with an NVIDIA GeForce RTX 3090, where every CUDA gate and the per-release GPU run take place; other GPUs are untested. On Linux, GLiNER2 (CPU and CUDA) and EmbeddingGemma 2 video on CUDA also need MALLOC_MMAP_THRESHOLD_=65536 in the process environment; see each package README. The model catalog links each gate to its evidence, and the compatibility matrix lists each platform and provider.
Release status: 0.5.0, published on nuget.org on 2026-11-03 (GitHub release), adds Qwen3.5 int4 and guided decoding on the hosted files, GLiNER2 text classification, reranking helpers and generated capability records; every Stable profile of 0.4.0 is unchanged (release record). It depends on Starlings.Tokenizers 0.3.4.
How a model becomes a package
Every model goes through the same twelve stages of the delivery pipeline, drawn here as a production line. The original Python implementation runs once, from pinned code and weights, to cut a golden master: protected fixtures and tolerances that only the maintainer can change. Later stages are measured against it: the tokenizer, the exported graph in Python ONNX Runtime, then the same graph from C#. model-pipeline evaluate names the first stage that diverges. Gates G0 to G7 decide whether a profile is Stable, a maintainer approves each release, and the packages carry asset locks rather than weights. Your app downloads the weights once, verifies their hashes and then runs offline. Open the image for the full-size animated view.
Why Starling
| If you would otherwise... | The trade-off | With Starling |
|---|---|---|
| Call a hosted embedding or chat API | Data leaves the machine, every call costs money, and the model can change under you | The model runs in-process; weights are pinned by commit and hash |
| Run a Python sidecar (sentence-transformers, transformers) | A second runtime to deploy, secure and scale | One PackageReference; the Python reference is what we test against |
| Call ONNX Runtime directly from C# | You write the tokenizer, pre- and post-processing and parity checks yourself | All of that is included and checked against upstream; measured warm latency is within noise of Python on the same backend |
| Use a GGUF chat runtime | Usually chat only, and quantized | Embeddings, extraction, image and video as well as chat; fp32 graphs checked against the original weights |
Starling depends only on the vendor-neutral Microsoft.Extensions.AI.Abstractions. CI rejects any Azure or OpenAI SDK dependency in the libraries, so you can adopt it, or leave it, without lock-in.
Use it with your AI coding agent
Starling includes three files written for coding agents such as Claude Code, Copilot, Cursor and Codex:
llms.txtis a short map of the repository: what Starling is, the facts that matter when writing code, and links to the rest.skills/starling-dotnet/SKILL.mdis an Agent Skill that teaches an agent to use the packages. It covers which package fits which task, the native runtime the app must add, offline loading, verified code for every model, CUDA, and the pitfalls (e5 prefixes, EmbeddingGemma 2 prompts, input limits).skills/starling-add-model/SKILL.mdis for contributors. It walks an agent through adding a new model in a pull request: choosing the onboarding route, writing the candidate contract, running themodel-pipelinecomparison against a fresh Python reference, and opening a draft PR. It also says what only the maintainer may change.
To give your agent context for a single task, paste this:
Read https://raw.githubusercontent.com/starlinghq/starling/main/llms.txt and
https://raw.githubusercontent.com/starlinghq/starling/main/skills/starling-dotnet/SKILL.md,
then use the Starlings.* NuGet packages to <what you want to build>.
To install the skill so it loads whenever it's relevant, for Claude Code:
mkdir -p .claude/skills/starling-dotnet
curl -fsSL https://raw.githubusercontent.com/starlinghq/starling/main/skills/starling-dotnet/SKILL.md \
-o .claude/skills/starling-dotnet/SKILL.md
Use ~/.claude/skills/ instead of .claude/skills/ to install it for all your projects. Other agents that support Agent Skills take the same folder. For agents that don't, paste SKILL.md into your AGENTS.md or rules file.
Contributing
Issues and pull requests are welcome. Start with a good first issue, read CONTRIBUTING.md, and use the issue templates to request a model or report a regression. Security reports go through SECURITY.md.
Build and test
Requires the .NET 10 SDK (pinned in global.json) and Python 3.11+.
dotnet test --solution ai-dotnet/AI.slnx
python3 -I tools/check-boundary.py
cd model-pipeline
pip install -e '.[dev]'
python3 -m pytest
ruff check .
ruff format --check .
model-pipeline validate registry
Repository layout
| Directory | Contents |
|---|---|
ai-dotnet/ |
.NET libraries, backend adapters, model implementations, samples, benchmarks. |
model-pipeline/ |
Manifest schema, model registry, admissions, candidate contracts and onboarding recipes, reference runners, evaluation, protected fixtures. |
browser/ |
The WebAssembly spike for e5 in the browser (browser/spike/) and the conformance harness of the browser packages (browser/conformance/). The packages themselves (Starlings.Runtime.OnnxWeb, Starlings.Runtime.OnnxWeb.WebGpu, Starlings.Models.E5.Browser) are under ai-dotnet/src/: Experimental, not yet published. |
docs/ |
Plan, generated catalog, guides, benchmarks, evidence and release records (index). |
skills/ |
Agent Skills for using Starling and for adding a model. |
.github/ |
Charter, contribution and support policy, security reporting, issue templates, CI. |
tools/ |
Repository checks (boundary, provenance), model acquisition, release drills, release gates, qualification and the CUDA CI pod scripts (tools/ci/). |
The tokenizer is its own product, Starlings.Tokenizers from starlinghq/tokenizers. The execution plan is docs/plan/fast-faithful-ai-dotnet.md, and open decisions are in docs/kickoff.md. An experimental five-model CPU onboarding workflow covers additional Qwen and BERT models; its references are still proposals. nomic-embed-text-v1.5 joined the BERT route afterwards (#120).
License
Original code is licensed under the MIT License or the Apache License, Version 2.0, at your option (LICENSE-MIT, LICENSE-APACHE), copyright StarlingHq. Model weights, tokenizer assets, and datasets keep their own terms; the code license does not cover them.