PerryLink/dsh-library
PerryLink/dsh-library has 28 GitHub stars and 0 forks. GitHot measured +1 stars in 24 hours. Last checked 2026-11-15 15:13 UTC. Last push 2026-11-13.
Measured star growth
- in 24h
- +1
- in 7 days
- Collecting history
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Local document knowledge base for DeepSeek Harness: library_add/remove/list, hybrid semantic+keyword library_search with diversity re-ranking, relevance filtering and lost-in-the-middle avoidance, citation-aware injection, library_cite_check and library_diagnose — SQLite-backed index via the storage domain, local embedding, zero model downloads.
AIREADME.en.md
Original
dsh-library
Turn your local md/txt documents into a citable, verifiable retrieval library.
A local document knowledge base for DeepSeek Harness: import by path, chunk with a sliding window, embed locally, and get hybrid search results whose [n] citations — and deletions — can be checked.
Install · Configuration · FAQ · 简体中文 · English · Español · Português · हिन्दी
dsh plugin --profile <name> add dsh-library
Community plugin, not affiliated with DeepSeek. Canonical repository: https://github.com/PerryLink/dsh-library (Gitee is a mirror).
What it is
dsh-library turns your md/txt documents into a local knowledge base your agent can query. library_add reads a document by path, chunks it with a sliding window, and writes it into the host's dsh_library storage domain; library_search combines semantic and keyword scoring, maximal-marginal-relevance diversity re-ranking, relevance filtering, and lost-in-the-middle avoidance to return a result page marked with [n] sources. Citations can be checked one by one with library_cite_check, and deletions are re-probed for residue by library_remove's purge verification. Zero model downloads and zero network requests by default — scoring is deterministic hashing and token math.
Three-step setup
Install (pick one channel):
# git channel (latest main) — the prepare script builds with production dependencies only dsh plugin --profile web add github:PerryLink/dsh-library # npm channel (published releases) dsh plugin --profile web add dsh-library # tarball channel: pnpm pack in this repo, then dsh plugin --profile web add ./dsh-library-<version>.tgzIf pnpm reports
ERR_PNPM_IGNORED_BUILDSfor this package (esbuild's harmless platform-binary validation), addallowBuilds: { esbuild: true }to yourpnpm-workspace.yaml— thedshCLI prints the exact snippet.Minimal config: the defaults are enough — the bundle patch inserts the
dsh-libraryrow, and the index lives in the host'sdsh_librarystorage domain (JSON or SQLite backend). You only need an override to change retrieval taste; see Configuration.Verify it mounts:
dsh --profile web --dump-config | grep -A2 'id: dsh-library'Then ask the agent to import and use a document:
> Add ./docs/spec.md to library docs, then answer: what does the spec say about retries? Cite [n] markers.
Uninstall: dsh plugin --profile web remove dsh-library (or remove the row from the profile patch).
Install from the DSH Desktop market
Every PerryLink plugin is listed in the built-in DSH Desktop market: Market → Sources → Add source → paste https://perrylink-dsh-catalog.perrylink.workers.dev/catalog-source.json → select. Installation still requires the market's npm identity check and your confirmation.
Core capabilities
dsh-library turns local md/txt documents into a queryable knowledge base with a quality pipeline your agent can trust:
library_add/library_remove/library_list— import a document by path (chunked and embedded), remove one with purge verification (signatures of the removed content are probed against the remaining index and any residue is reported), and list document metadata.library_search— BM25 + semantic reciprocal-rank fusion (default;search.fusion: weightedkeeps the legacy hybrid weight), an auditable Chinese↔English query bridge, maximal-marginal-relevance diversity re-rank, relevance filtering, and lost-in-the-middle avoidance (strongest chunks pinned to head and tail). Withinject: truethe result page is injected into the calling agent; every hit carries a[n]source marker and the injection is reconstructable from thelibrary/injectsession event (host-gated; see Permissions & data).library_cite_check— verify the[n]citations in an answer against the search result page with a fuzzy token match AND a semantic similarity check.library_diagnose— chunk-size histogram, near-duplicate chunk pairs, a self-retrieval probe, and the middle-penalty signal./library— one-line index summaries per library.
Commands & tools surface
| Tool | Notes |
|---|---|
library_add |
{ path, library, name? } → document id; file read through the harness filesystem service |
library_remove |
{ library, documentId } → removal summary + purge verdict (residue reported) |
library_list |
{ library? } → document metadata (never text) |
library_search |
{ query, library, topK?, inject? } → ranked hits with [n] markers; inject: true seeds the calling agent |
library_cite_check |
{ library, query, answer } → per-citation valid/invalid verdicts (fuzzy + semantic) |
library_diagnose |
{ library } → chunk stats, duplicates, self-retrieval, middle penalty |
/library [name] |
Command: per-library document/chunk summaries |
Examples
These are the plugin's real entry points; install once, then call one.
Detailed example
Command
library_search({ library: "docs", query: "what is the harness built on", inject: true })
Output
Library docs: 2 result(s) for "what is the harness built on"
[1] DeepSeek Harness is a plugin-based agent harness built on vendored Cordis. Everything in the harness is a plugin row inside a cordis.yml composition. The tools/pre-execute waterfall lets plugins gate…
[2] The recipe for sourdough starts with a levain fed the night before. Bulk fermentation takes four to six hours at room temperature. Bake in a dutch oven at 230 degrees Celsius for the first twenty min…
Observed from README.md:161-167; tests/index.spec.ts:46-64; src/index.ts:830-836
Failure branch
Input
{"path":"/nonexistent/dsh-library-missing.txt","library":"docs"}
Output
isError=true; error.message=cannot read "/nonexistent/dsh-library-missing.txt": not found; error.info={"name":"FsError","code":"FS_NOT_FOUND"}
Observed from plugins/dsh-library/src/index.ts:695-696 (fs.resolve/readText); plugins/dsh-library/tests/index.spec.ts:143-151 (unreadable path); plugins/dsh-library/README.md:195 (unreadable files fail loud); exercised through plugins/dsh-library/tests/harness.ts
Brief examples: library_search · library_list
Configuration
All tunables are Schemastery Config fields (changeable from cordis.yml). An id-targeted override replaces the whole row — restate every key you need. cordis.patch.yml documents each key inline.
Most-used keys:
| Key | Default | Meaning |
|---|---|---|
chunkSize |
900 |
Sliding-window chunk size in characters (≤ 4000) |
chunkOverlap |
120 |
Overlap between consecutive windows; must be smaller than chunkSize |
maxFileBytes |
5242880 |
Files larger than this are rejected on library_add |
embedding.preset |
custom |
hash / ollama / openai / custom; fills defaults, explicit fields win |
embedding.provider |
hash |
Embedder backend: hash (built-in), command (subprocess), ollama (local, degrades to hash), or openai (OpenAI-compatible /embeddings) |
embedding.openaiBaseUrl / openaiModel |
https://api.openai.com/v1 / text-embedding-3-small |
OpenAI-compatible endpoint and model |
embedding.openaiApiKeyEnv |
OPENAI_API_KEY |
environment variable holding the key; the key is never stored |
search.topK |
8 |
Results returned after the full pipeline |
search.fusion |
rrf |
rrf = BM25 + semantic reciprocal-rank fusion (normalized to the best hit); weighted = legacy hybrid weight |
search.hybridWeight |
0.6 |
legacy weighting, used only when search.fusion = weighted |
search.minRelevance |
0.15 |
Chunks below this relevance threshold are filtered out |
injection.enabled / maxChars |
true / 12000 |
library_search inject behavior and budget |
Full configuration table (18 key groups)
| Key | Default | Meaning |
|---|---|---|
chunkSize |
900 |
Sliding-window chunk size in characters (≤ 4000) |
chunkOverlap |
120 |
Overlap between consecutive windows; must be smaller than chunkSize |
maxFileBytes |
5242880 |
Files larger than this are rejected on library_add |
embedding.dims |
256 |
Hash-embedding dimensionality (≥ 8) |
embedding.preset |
custom |
hash / ollama / openai / custom; fills defaults, explicit fields win |
embedding.provider |
hash |
Embedder backend: hash (built-in), command (subprocess), ollama (local, degrades to hash), or openai (OpenAI-compatible /embeddings) |
embedding.openaiBaseUrl / openaiModel |
https://api.openai.com/v1 / text-embedding-3-small |
OpenAI-compatible endpoint and model |
embedding.openaiApiKeyEnv |
OPENAI_API_KEY |
environment variable holding the key; the key is never stored |
embedding.command |
'' |
Optional external embedder command (space-separated argv, no shell) over ctx.subprocess; setting it selects the command provider |
embedding.ollamaUrl / ollamaModel |
http://127.0.0.1:11434 / nomic-embed-text |
Local Ollama endpoint + model for the ollama provider (zero cloud) |
embedding.timeoutMs / graceMs / maxOutputBytes / maxBatchItems |
30000 / 1000 / 1048576 / 64 |
Embedder subprocess budget |
search.topK |
8 |
Results returned after the full pipeline |
search.fusion |
rrf |
rrf = BM25 + semantic reciprocal-rank fusion (normalized to the best hit); weighted = legacy hybrid weight |
search.hybridWeight |
0.6 |
legacy weighting, used only when search.fusion = weighted |
search.minRelevance |
0.15 |
Chunks below this relevance threshold are filtered out |
search.diversityLambda |
0.5 |
MMR trade-off: 1 = pure relevance, 0 = pure diversity |
search.lostMiddleHead / lostMiddleTail |
1 / 1 |
Strongest chunks pinned to head / tail |
search.maxResultChars |
16000 |
Character budget of the model-facing result page |
injection.enabled / maxChars |
true / 12000 |
library_search inject behavior and budget |
citation.windowChars / minScore / minSemantic |
150 / 40 / 0.1 |
library_cite_check thresholds |
purge.signatureLength / maxProbes |
4 / 24 |
Purge verification signatures and probe budget |
diagnose.maxDuplicatePairs / sampleCap / positionBins |
24 / 200 / 5 |
library_diagnose budget caps |
Example override in your profile patch:
- insert:
- id: dsh-library
name: dsh-library
config:
chunkSize: 900
embedding:
provider: ollama
ollamaModel: nomic-embed-text
search:
hybridWeight: 0.7
FAQ
Where do the document text and the index actually live? Can they reach the session log?
Chunk text and embeddings live in the host's storage backend (JSON or SQLite), with the same trust as the deployment's other durable data; the plugin adds no encryption. Document paths and embeddings never enter the session log — library/inject (id, query, chunk ids, page size) and library/purge (verdict) are log-only audit events, and the model-visible injected page is reconstructable from them. On hosts that record neither event type, the logged tool/call + tool/result events remain the reconstructable audit trail.
Does it work without a model or a network?
Yes. The built-in embedder is deterministic hashing (word tokens plus character tri-grams into a signed bucket vector, L2-normalized), so there are zero downloads and zero network requests; scoring is hashing and token math. The trade-off is lexical-grade semantics: retrieval quality on paraphrases is lower than a real embedding model — configure embedding.command (any subprocess embedder) or embedding.provider: ollama (a local Ollama model) for stronger semantics.
Is a removed document really gone?
library_remove does not assume success: it probes the remaining index with deterministic signatures of the removed content (purge.signatureLength, at most purge.maxProbes probes) and reports residue instead of claiming the content was purged.
Does it fight other plugins over tool names or services?
No. It only consumes host services (storageDomain / tools / commands); every tool it registers carries the library_ prefix, its only command is /library, it registers no waterfall listener, and it provides no service key another plugin could contend for — so it occupies no bare tool name.
How big is the performance overhead?
The default path makes no model calls and no network requests; the cost is one sliding-window chunking pass plus hash embedding, then scoring and re-ranking at query time. The bounds are configurable: maxFileBytes caps document size, search.topK caps results, search.maxResultChars and injection.maxChars cap the model-visible text budget, and library_diagnose shows chunk distribution and duplicates on demand.
Security boundaries
- Local by default. Zero model downloads, zero network calls — scoring is deterministic hashing and token math. Only an explicitly configured embedder command runs code, and its protocol is completeness-checked and output-capped.
- No fabrication. Citation checks report what the pipeline can verify; suspicious citations are surfaced honestly, never guessed.
- Purge is verified.
library_removeprobes the remaining index with deterministic signatures of the removed content and reports residue instead of assuming success. - Fail loud. Invalid library names, oversized documents, unreadable files, and a configured-but-absent embedder seam all fail the call with a clear error.
Permissions & data
- Permissions: the plugin only reads files you point
library_addat (through the harness filesystem service and its policy) and writes into its owndsh_librarystorage domain. No network requests; an optional external embedder runs throughctx.subprocesswithout shell interpretation. - Data: chunk text and embeddings live in the host's storage backend (same trust as the deployment's other durable data); the plugin adds no encryption. Document paths and embeddings never enter the session log.
- Session log:
library/inject(id, query, chunk ids, page size) andlibrary/purge(verdict) are log-only audit events — the model-visible injected page is reconstructable from them. The append is host-gated: harnesses whose known-type set covers the vocabulary get the events,ignorable-envelope builds get them with the marker, and envelope-less builds (0.1.1-rc.2, 0.1.2-rc.1) skip the append — the loggedtool/call+tool/resultevents remain the reconstructable audit trail there. 0.1.2-rc.1 (adapted 2026-09-02): the session envelope keeps its ignorable field for stored-log read compatibility only - Session.append still cannot stamp it, so audit-gate behavior is unchanged.
Known limitations
- Lexical-grade default embeddings. The built-in hash embedder scores surface similarity, not meaning; configure
embedding.preset: openai(orollama/command) for a real model. BM25 covers exact terms even with the hash default. - Small bilingual dictionary. The query bridge is deliberately tiny and auditable; unknown terms are not translated.
- Local citation model.
library_cite_checkvalidates against the search result page (the[n]numbering), not against free-form source names; the fuzzy score is a bounded token-sequence partial ratio. - No ingestion pipeline. Documents must be imported by path (
md/txt); PDF/docx extraction is out of scope for v0.1.0. - Host-gated audit events.
library/inject/library/purgeare only appended on harnesses that can carry them (see Permissions & data); on the published0.1.2-rc.1line (as on earlier envelope-less lines) they are not appended, and every fact stays reconstructable from the tool call/result log.
Compatibility
| Surface | Status |
|---|---|
| Harness | DeepSeek Harness dsh-v0.2.1-alpha.1 (verified 2026-09-25: dual typecheck rulers + 108 tests + self-contained/artifacts gates; peer range >=0.1.2-rc.1 <0.2.0 || >=0.1.5-alpha.1 <0.2.0 || >=0.1.6-0 <0.2.0 || >=0.1.7-0 <0.2.0). Home of the family's only exercised alpha.2 host smoke (2026-09-11). |
| Node | ^22.19.0 || >=24.0.0 |
| Storage | Any storage-domain backend (JSON or SQLite); the index lives in the host's storage domain |
| Models | None required — the built-in embedder is deterministic hashing (zero downloads) |
Ecosystem & quality
- Gitee mirror (sync mirror; GitHub is canonical)
- dshfind plugin page (installs and ranking)
- OpenSSF Scorecard report (driven by .github/workflows/scorecard.yml)
- dsh-plugin-doctor verification report
- DSH Market (listed)
- 1024 store channel:
npm i -g dsh1024once, thendsh1024 plugin --profile web add dsh-library(counts toward the deepseek1024.com install ranking) - Ecosystem knowledge base — measured data, not marketing: plugin development guide · plugin-selection data · maintenance criteria
Development
pnpm install # node ^22.19 || >=24
pnpm run typecheck # tsc: src + tests against the local harness checkout
pnpm run typecheck:ci # tsc against the published 0.1.7-rc.2 types (no paths)
pnpm test # vitest: quality ports, core vocabulary, real-stack assembly
pnpm run build # tsdown bundle + tsc declarations (lib/)
pnpm run verify:self-contained # dependency specs resolve from the registry
pnpm run verify:artifacts # built ESM face + bundle patch present
pnpm run check:readmes # five-language README consistency
pnpm pack # the published tarball
The README images are generated programmatically by scripts/gen-readme-assets.py: Pillow draws the hero banners (docs/assets/dsh-library-hero-{zh,en}-{light,dark}.webp) and a hand-written SVG draws the structure diagram (docs/assets/dsh-library-flow-v5-{light,dark}.svg); both are safe to re-run. The older single-panel flow diagram stays in docs/assets/ for external links; the READMEs only reference the new hero and flow-v5 pairs.
Topics: dsh, dsh-plugin, deepseek-harness, deepseek, cordis, rag, knowledge-base, retrieval, embedding, vector-search, citation-validation, document-library
Family & thanks
This project is part of the DeepSeek Harness plugin family from PerryLink (95 plugins, all Apache-2.0). If it is useful, a star helps the next person find it — nothing is gated behind it.
- @PerryLink — creator and maintainer: the eight quality ports, storage-domain index, hybrid retrieval pipeline, citation/purge verification, and the five-language docs.
Expand all plugins
License
LICENSE (Apache License 2.0) © 2026 dsh-library contributors
Apache-2.0 · Community plugin, not affiliated with DeepSeek.