Skip to content
Projects

boromat/polanon has 29 GitHub stars and 9 forks. GitHot measured +3 stars in 24 hours. Last checked 2026-11-15 20:15 UTC. Last push 2026-11-14.

How these counts are measured

29 stars9 Forks +3 in 24h

Measured star growth

in 24h
+3
in 7 days
Update delayed
in 15 days
Update delayed
in 1 month
Update delayed

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Reversible PII masking for LLM calls and RAG, Polish-first: checksum-validated PESEL, NIP and IBAN, Polish names in every case, stable tokens, streaming restore and Haystack components.

AI

README.md

Original

polanon

CI Python 3.11+ License: MIT

English | Polski

Reversible PII masking for LLM calls, Polish-first. Mask personal data before a prompt leaves your network, send only tokens to the model, and put the originals back into its answer — even while it streams. Pure Python: the standard library is the whole core.

polanon masking a Polish invoice request in a terminal, then restoring the model's answer in Python

pip install polanon
from polanon import Anonymizer, default_guards

anonymizer = Anonymizer(default_guards(with_ner=False))

result = anonymizer.anonymize_sync(
    "Proszę o korektę faktury dla Jana Kowalskiego, PESEL 50810112354, "
    "konto PL61109010140000071219812874, mail [email protected]."
)
print(result.anonymized)
# Proszę o korektę faktury dla <OSOBA_1>, PESEL <PESEL_1>, konto <IBAN_1>, mail <EMAIL_1>.

answer = "Korekta dla <OSOBA_1> (PESEL <PESEL_1>) jest gotowa."  # what the model replied
print(Anonymizer.restore(answer, result.mapping))
# Korekta dla Jana Kowalskiego (PESEL 50810112354) jest gotowa.

The model only ever sees <OSOBA_1>. The mapping that turns it back into a name never leaves your process. anonymize_sync works in scripts, Django and Flask views, Celery tasks and notebooks; in async code, await anonymizer.anonymize(...) does the same work in a worker thread.

Why another PII library

General-purpose tools are built around English. Polish data needs three things they do not do out of the box:

  • Checksums. A PESEL, NIP, REGON, ID card number, IBAN or payment card is validated, not just pattern-matched: the checksum rejects most random digit strings — for a PESEL, together with the birth date, about 98% of them — while a valid number always passes. KRS numbers, which have no public checksum, and passports are recognised next to their label as well. Eleven digits right after the word PESEL are masked even with a typo in them.
  • Seven grammatical cases. "Jan Kowalski", "Jana Kowalskiego" and "z Janem Kowalskim" are the same person. Polish first names are recognised in every case, in lowercase and typed without Polish letters, together with the surname that agrees with them, without any language model.
  • Polish model labels. Polish spaCy models tag people as persName, not PERSON. A pipeline configured for English labels finds no names at all, and says nothing about it.

It also reads foreign invoices: EU, UK and Swiss VAT numbers, company register numbers, Bankgiro, Plusgiro and sort codes, international phone numbers, street addresses and postcodes.

How it compares

polanon Presidio pii-veil piimask
Restores the model's answer Yes, from stable tokens such as <OSOBA_1>, also while it streams Not built in: its deanonymizer decrypts spans at known positions; its docs include a sample that restores an LLM answer Yes, from stable tokens and a saved mapping Yes, from stable tokens
Polish identifiers with a checksum PESEL, NIP, REGON, ID card, IBAN, payment cards PESEL; IBAN and payment cards in general PESEL, NIP, REGON, IBAN, payment cards; ID card and passport by pattern only None; IBAN and payment cards in general
Polish names in every grammatical case For listed first names and the surnames that agree with them, without a language model; spaCy adds the rest With a Polish spaCy model and its persName label mapped Not detected Not detected; its README suggests adding an NER model
What the core needs Nothing beyond the standard library spaCy, numpy, pydantic and a few more; a language model only for names and places pii-core and typer Nothing beyond the standard library
On polanon's sample documents 98% fully masked with pl_core_news_md, 86% with rules alone 66% with the same model (presidio-analyzer 2.2.366) Not measured Not measured
License MIT MIT Apache-2.0 MIT

Compared from each project's own documentation and source on 3 November 2026. Only Presidio was run on the sample documents, with the script in benchmarks/, and that is not neutral ground: the eleven synthetic documents were written by polanon's author, who tuned its rules on them, and Presidio had only the Polish model, also for the three invoices from abroad. Each has its place: Presidio for many languages, structured data and its Azure integration; pii-veil for Polish identifiers under Apache-2.0 when names do not matter; piimask for language-neutral identifiers (e-mail, phone numbers, payment cards, IBAN, IP addresses, URLs) and US social security numbers, with fake-value, hashing and partial strategies and an encrypted vault. polanon is for Polish text on its way to a model, where the names in it and the restored answer matter as much as the numbers.

Install

You want Install
Identifiers, contacts, secrets, Polish names, companies by legal form pip install polanon
Also places, and names and organisations no rule can see pip install "polanon[ner]" then python -m spacy download pl_core_news_md
The stage-2 judges (a local Ollama model, or a remote one) pip install "polanon[judge]"
Components for Haystack 3 pipelines pip install "polanon[haystack]"
A gateway for tools that use the OpenAI API pip install "polanon[gateway]"

Then use default_guards() with the model, or default_guards(with_ner=False) without it. Another spaCy model is one argument away: default_guards(spacy_model="pl_core_news_lg"). Package names copied from pip work too (pl-core-news-lg). A local model directory can be passed as a path; it must contain meta.json and config.cfg. Missing packages or model directories are reported when the guards are built, while the model itself loads on first use. An installed model package takes priority over a local path with the same name.

From the command line

Text comes from the arguments, from a file with --file, or from standard input. Masked text goes to stdout and the mapping to stderr, or to a file with --mapping:

polanon --version  # print the installed version
polanon "Jan Kowalski, PESEL 50810112354"
polanon --file umowa.txt --ner --mapping mapping.json > masked.txt
cat umowa.txt | polanon --ner > masked.txt 2> mapping.json

A non-zero exit status means something went wrong and stderr says what. --file reads the file's bytes itself and tells UTF-8, UTF-16 (what Windows PowerShell 5.1 writes with >) and cp1250 apart; text it cannot decode for certain is refused rather than masked half-blind.

On Windows, prefer --file and --mapping to pipes and redirection. Windows PowerShell 5.1 re-encodes text piped into a program as ASCII, so Polish letters arrive as ? and a garbled name is not recognised, and its 2> wraps stderr in error records. If you do pipe, run $OutputEncoding = [System.Text.UTF8Encoding]::new($false) first; in cmd.exe, run chcp 65001.

Streaming

A provider streams arbitrary fragments, so <OSOBA_1> can arrive as <OSO and BA_1>. The restorer holds back anything that could still grow into a token:

restorer = Anonymizer.stream_restorer(result.mapping)
for chunk in stream:  # "Dzień dobry <OSO", "BA_1>, PESEL <PESEL", "_1> zweryfikowany."
    print(restorer.feed(chunk), end="")
print(restorer.flush())
# Dzień dobry Jana Kowalskiego, PESEL 50810112354 zweryfikowany.

With Claude, GPT or a local model

Mask before the call, restore while the answer streams. With the Anthropic SDK:

import anthropic
from polanon import Anonymizer, default_guards

anonymizer = Anonymizer(default_guards(with_ner=False))
masked = anonymizer.anonymize_sync(question)
restorer = Anonymizer.stream_restorer(masked.mapping)

with anthropic.Anthropic().messages.stream(
    model="claude-opus-5-5",
    max_tokens=64000,
    system="Personal data is replaced by tokens such as <OSOBA_1>. Copy a token exactly as written.",
    messages=[{"role": "user", "content": masked.anonymized}],
) as stream:
    for text in stream.text_stream:
        print(restorer.feed(text), end="")
print(restorer.flush())

Telling the model what the tokens are keeps it from rewriting them. The examples go further: openai_chat.py does the same for any OpenAI-compatible endpoint (OpenAI, OpenRouter, Azure, vLLM, a local Ollama), and anthropic_chat.py holds a conversation whose history keeps masked text only, so an answer restored for the reader is never sent back in the clear. Both are tested in CI against the real SDKs, offline.

A chat turn with tools

A turn of a chat with tools sends more than the question: the history, and the result of every tool the model calls. anonymizer.turn() masks all of it with one mapping, puts the originals back into the tool arguments and the answer, and refuses instead of sending anything it could not mask:

import json

import anthropic
from polanon import Anonymizer, MaskingUnavailableError, default_guards, token_instructions

anonymizer = Anonymizer(default_guards(with_ner=False))
client = anthropic.Anthropic()


def chat(history: list[dict], clients: list[str]) -> None:
    """One turn. `history` ends with the question; SYSTEM, TOOLS and run_tool are your own."""
    try:
        turn = anonymizer.turn(known=clients, unknown_blocks="drop")
        messages = turn.mask_messages(history)
        while True:
            restorer = turn.stream_restorer()
            with client.messages.stream(
                model="claude-opus-5-5",
                max_tokens=64000,
                system=SYSTEM + "\n\n" + token_instructions("en"),
                tools=TOOLS,
                messages=messages,
            ) as stream:
                for text in stream.text_stream:
                    print(restorer.feed(text), end="")
                response = stream.get_final_message()
            print(restorer.flush())
            messages.append({"role": "assistant", "content": response.content})
            if response.stop_reason != "tool_use":
                return
            results = []
            for block in response.content:
                if block.type == "tool_use":
                    result = run_tool(block.name, turn.restore_data(block.input))
                    masked = json.dumps(turn.mask_data(result, default=str), ensure_ascii=False)
                    results.append(
                        {"type": "tool_result", "tool_use_id": block.id, "content": masked}
                    )
            messages.append({"role": "user", "content": results})
    except MaskingUnavailableError:
        print("Masking is on but not working, so nothing more was sent to the model.")

turn.stream_restorer() puts the originals back as each response streams in, also a token cut between two events, as in Streaming. The tests run this example as written here against the real anthropic SDK, offline, so it cannot drift from the library.

A turn or the lower API. Take a turn whenever a model gets a list of messages, with tools or without: it keeps one mapping for all of them, masks a value found in a later message in the earlier ones too, and refuses on any failure, all of which you would otherwise write yourself. Call anonymize_sync(), Anonymizer.restore() and StreamRestorer directly for a single text, such as a document or a prompt you build yourself, when you need the spans of the result, or when the stage-2 judge must run, which a turn does not take.

One mapping for the turn. A client in the history, in the question and in a tool result is the same <FIRMA_1>. To keep the tokens from one turn to the next, start the next one with anonymizer.turn(mapping=turn.mapping).

What mask_messages masks. Messages in the Anthropic or the OpenAI format: their text, the input of tool_use, tool results and the arguments of tool_calls, with JSON kept valid and ids and tool names left alone. A value recognised only in the question is masked where it stood earlier in the history, unrecognised. A block nothing can mask, such as an image or a model's signed reasoning, is refused, or left out with unknown_blocks="drop". mask_data masks the strings in a value; keys and numbers stay as they are, so keep personal data in string values, and an object such as a date needs default=str.

Values you already know. known= takes what you know is sensitive, such as your clients' names from the database. They are masked wherever they appear, also when they are on the allowlist.

Refusing instead of leaking. Whatever fails while masking raises MaskingUnavailableError, and nothing half masked comes back. default_guards() raises NerModelUnavailableError, which is one, when spaCy or the model is missing, and ner_available() tells a settings page whether they are installed. Once the answer is in, turn.unknown_tokens(answer) lists the tokens the model made up, and turn.sweep(restored) also looks for personal data the answer added.

Masking for RAG and search indexes

Tokens numbered per call mean nothing outside their conversation: a contract masked for a search index and a question masked a week later can each call a different person <OSOBA_1>, and the search finds nothing. Give the anonymizer a secret key and every token is computed instead, from HMAC-SHA256 of the value under the key: the same in every document, every process and every question, with no shared state. "Jan Kowalski", "Jana Kowalskiego" and "z Janem Kowalskim" get one token, and so does a NIP with its dashes or without them.

import os

from polanon import Anonymizer, InMemoryMappingStore, Pseudonyms, default_guards, stored_mapping

key = bytes.fromhex(os.environ["POLANON_KEY"])  # 32 random bytes per company, from a secret store
store = InMemoryMappingStore()  # in production, an encrypted table of yours, apart from the index
pseudonyms = Pseudonyms(key, collisions=store.collisions)
anonymizer = Anonymizer(default_guards(with_ner=False), pseudonyms=pseudonyms)

document = anonymizer.index_chunks(
    [
        "Umowę z Firmą Kot Sp. z o.o. podpisał Jan Kowalski.",
        "Pełnomocnikiem Jana Kowalskiego jest Anna Nowak, tel. 600 100 200.",
    ],
    store,
)
for text in document.texts:  # tokens only: these go to the embedder and the index
    index.add(text, embed(text))  # index and embed are your own

question = anonymizer.mask_query("Kto jest pełnomocnikiem Jana Kowalskiego?", store)
passages = index.search(embed(question.anonymized))  # the question has the passages' tokens

mapping = question.mapping | stored_mapping(store, question.anonymized, *passages)
restorer = Anonymizer.stream_restorer(mapping)  # restores the answer, as in Streaming

The tests run this example as written here, with a stand-in index and embedder. Each part:

Nothing in the index names a value. document.texts holds tokens where the values were; what each token stands for goes to the store. Never copy a mapping into a passage's metadata, which is stored and searched with it. document.mappings[i] restores passage i exactly as it was written, "Jana Kowalskiego" in the genitive; keep it with the passage, encrypted, outside the index.

A store of your own. MappingStore has two methods: get(tokens) and add(entries), which stores only the tokens it does not hold yet, atomically per token, as a unique key on the token does. It returns what it already holds for the others. Encrypt StoredValue.value, which is the personal data; its fingerprint names nothing without the key. Keep store.collisions next to it and pass it to every Pseudonyms: it holds tokens and fingerprints, nothing else.

Which form a name comes back in. One token stands for "Jan Kowalski" and "Jana Kowalskiego". The store keeps the nominative, because the model writes the token in a sentence of its own, and "Umowę podpisał Jana Kowalskiego" would be wrong whichever document came first. A passage restored with its own mapping keeps the case it was written in.

Names the store already holds. A surname on its own, as in "Co podpisał Kowalczyk?", is a name no rule reads. index_chunks stores the surname of every person whose first name the rules know, next to the full name, and mask_query and index_chunks look up every run of up to four words in the store and mask the ones it holds a token for. A question then gets the surname's token, and a surname masked in one document stays masked in the next. Pass lookup=False for a store too slow for that.

The key. At least 32 random bytes, one key per company and a scope= per index that must not be linked to another. Keep it in a secret manager, out of the index, the code and the logs; Pseudonyms never shows, pickles or copies it. A new key, scope or number of digits changes every token, so the index has to be built again: store pseudonyms.settings with the index and compare them at startup. To erase a person, delete the passages holding pseudonyms.token(name, "PERSON"), and those holding the surname's token where nobody else with that surname is indexed. Deleting the store's entries alone does not unlink them, since the key turns a guess back into the token.

Collisions. Twelve digits make two values of one type share a token with a chance of about 1 in 20,000 at 10,000 values, and 1 in 200 at 100,000. Wherever two such values meet, in the store or in one text, the one that came second gets four digits more.

Limits. A name the rules cannot decline, such as a rare first name or "Krawca" from "Krawiec", and a company name in another case, "Firmą Kot" beside "Firma Kot", keep a token per form, which costs a missed search but leaks nothing. A word alone in lowercase is never looked up in the store. And one token everywhere is the point and the price: whoever reads many masked passages can tell which of them are about the same person. The threat model says what that means.

With Haystack

pip install "polanon[haystack]" adds components that put the masking into a Haystack 3 pipeline. The key is read from POLANON_KEY, 32 random bytes in hexadecimal:

from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from polanon import InMemoryMappingStore
from polanon.integrations.haystack import (
    PolanonDocumentMasker,
    PolanonQueryMasker,
    PolanonRestorer,
    PolanonStreamRestorer,
)

mappings = InMemoryMappingStore()  # in production, an encrypted table of yours

indexing = Pipeline()
indexing.add_component("mask", PolanonDocumentMasker(store=mappings))
indexing.add_component("split", DocumentSplitter(split_by="period"))
indexing.add_component("embed", document_embedder)  # any embedder: it gets masked text
indexing.add_component("write", DocumentWriter(document_store))
indexing.connect("mask", "split")
indexing.connect("split", "embed")
indexing.connect("embed", "write")

asking = Pipeline()  # with an embedder, a retriever, a prompt builder and a generator of yours
asking.add_component("mask", PolanonQueryMasker(store=mappings))  # first
asking.add_component("restore", PolanonRestorer(store=mappings))  # after the generator
asking.connect("mask.query", "embed.text")
asking.connect("mask.query", "prompt.query")
asking.connect("retrieve.documents", "restore.documents")
asking.connect("llm.replies", "restore.replies")
asking.connect("mask.mapping", "restore.mapping")

stream = PolanonStreamRestorer(mappings, on_text=lambda text: print(text, end=""))
asking.run({"mask": {"query": question, "stream": stream}, "llm": {"streaming_callback": stream}})

examples/haystack_rag.py is the whole of it, offline, with Haystack's stand-in embedder and generator, and the tests run it. examples/haystack_openai.py is the same with OpenAI's embedder and model: it is paid, so the tests only build it.

Mask first. The document masker comes before the splitter, so a whole document is read and a company or an address is masked where it stands: cut at full stops first, "Firmą Kot Sp. z o.o." fell apart and went out unmasked. Passages that come split already are masked together when they name one source in source_id, parent_id or __parent_id. Split by word, sentence, period or passage: a splitter that counts characters cuts tokens in two, and a half matches nothing.

Nothing in the document store names a value. The masked documents are new ones: tokens in the content and in every string of meta, nested keys and the members of sets included. A document's id is a plain SHA-256 of its text, so every string of meta shaped like one, which is what splitters write there, points to the masked twin of its document or becomes a keyed fingerprint. An id Haystack computed is computed again from the masked document; one you chose is masked like text and kept, so upserts and deleting by id still work. What the tokens stand for goes to the mapping store only, never into meta. A document with a blob or an embedding is refused, since both were made from the original. The tests check every document in the store, and the file it saves, for each original value.

The answer, for the reader. PolanonQueryMasker sends the masked question to the embedder and the prompt builder, and its mapping to PolanonRestorer, which restores the replies and the documents found. Its documents are for display only: never write them to a store or send them to a model. PolanonStreamRestorer restores a streamed answer as the generator's streaming_callback; with Pipeline.stream(), feed it the chunks instead: stream.feed(chunk.content). Values you know are sensitive, such as client names, go in at run time, {"mask": {"known": clients}}, so they never end up in a saved pipeline.

Saving a pipeline. The key is a Haystack Secret, and a saved pipeline holds the name of its variable only. A mapping store is saved through its own to_dict, which should say where it connects and never what it holds; InMemoryMappingStore has none, so a pipeline with it is built in code. Haystack 3 loads classes from its own modules only, so allow this one and your store's: Pipeline.loads(text, allowed_modules=["polanon.integrations.haystack", "myapp.vault"]).

Keep content tracing off. With HAYSTACK_CONTENT_TRACING_ENABLED, Haystack records what every component receives, and the masker receives the originals. A run that fails keeps its inputs in the error's snapshot, written to disk with HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED. Both are off by default.

A gateway for any OpenAI-compatible tool

Tools such as Hermes Agent, n8n, Open WebUI or LibreChat talk to a model through the OpenAI API. polanon-gateway stands between such a tool and the provider, on the machine of whoever runs it: every message is masked before it leaves, and the answer comes back with the original values, streamed or not, tool calls included.

pip install "polanon[gateway,ner]"
python -m spacy download pl_core_news_md
polanon-gateway --upstream https://api.openai.com/v1

Then point the tool's base URL at http://127.0.0.1:8787/v1 and keep its API key: the gateway passes it on to the provider. A local model works the same way, --upstream http://localhost:11434/v1 for Ollama.

Tool Where the base URL goes
Hermes Agent base_url of the model or of a named provider in config.yaml
n8n the Base URL of the OpenAI credential
Open WebUI the URL of an OpenAI API connection
LibreChat the baseURL of a custom endpoint in librechat.yaml

What it does with a call:

  • It serves /v1/chat/completions and the Responses API, /v1/responses, streamed or not, /v1/embeddings and /v1/models, and refuses every other endpoint, so nothing it cannot mask leaves as it came.
  • With the Responses API the whole conversation comes with every call, as Hermes Agent sends it with store: false. A call that leaves it at the provider, with previous_response_id or conversation, or asks for a background response to fetch later, would bring back tokens no one can restore, and is refused. Encrypted reasoning passes as it is: the provider sealed it from masked text.
  • Every message, tool result and earlier tool call is masked, and so is the prose of the tools and of an answer format: descriptions, titles and example values. Tool names, types and parameter names stay, so the tools keep working. A participant's name is left out.
  • An image, a file or audio cannot be masked, so the call is refused; --drop-unsupported leaves such content out and sends the rest.
  • Before anything leaves, the payload is searched once more for every original value, and the call is refused if one is still there.
  • A note telling the model what the tokens are joins the system message: --instructions en, or none to leave it out.
  • It keeps nothing on disk. A tool sends the whole conversation with every call, and each call is masked and restored on its own. With POLANON_KEY set to 64 hexadecimal characters, a person keeps one token in every call and in every grammatical case.

Keys stay with whoever runs the gateway, and come from the environment only. By default the tool's own key is passed on; POLANON_UPSTREAM_KEY sends another one instead. The gateway listens on 127.0.0.1. To reach it from another machine, set --host and POLANON_GATEWAY_KEY, which every client must then send: without the key it refuses to start. The log names the path, the status, the time and how many values of each type were masked, never a value.

There is no hosted service: the gateway runs where it is installed, and whoever runs it answers for that machine, its keys and the provider the calls go to. Not supported yet: Anthropic's Messages API. The threat model says what the gateway protects and what it does not.

What it detects

Checksum-validated Polish identifiers (PESEL, NIP, REGON, ID card, IBAN, payment cards), KRS numbers next to their label, e-mail addresses and phone numbers, street addresses and postcodes, links to a person's profile (LinkedIn, GitHub, GitHub Pages, GitLab, social media) up to the handle, IPv4 and IPv6, MAC addresses, vehicle registrations after their label ("nr rej."), VINs, passport numbers, birth dates, API keys, tokens and passwords, Polish first names with their surnames, and company names anchored on a legal form ("Sp. z o.o.", "S.A.", "Ltd", "GmbH"). Once a person or a profile link is masked, the surname or the handle alone is masked wherever it appears in the same text: "Kowalski" after "Jan Kowalski", "jkowalski" in a JSON field after "github.com/jkowalski". With the ner extra, a spaCy model adds places and the names nothing else marks. An optional second stage lets a small local model review the masked text for what still identifies someone by context. Other dates are recognised too, but left readable unless you ask for them.

Text copied from PDFs, Word or macOS is read as a person would read it: letters written as a base letter plus an accent, no-break spaces and en dashes inside numbers, and invisible zero-width characters do not hide a value. The mapping keeps each value as it was first written.

On the annotated sample documents in the repository, the rules alone fully mask 86% of the values and the rules with pl_core_news_md mask 98%, with nothing left half-masked. Twelve short documents are a regression check, not a research benchmark: see the benchmark notes for what is missed and why, and for Presidio on the first eleven of them with the same model: 66% fully masked, 76% at its most lenient.

Worth knowing

Tokens are stable per conversation, not global, unless you give a key, as for a search index. Pass the mapping from the latest result to the next call: the same value keeps the same token, and a token is never reused for another value. The mapping you pass in is read, not changed, so keep the one each result returns:

mapping = result.mapping
second = anonymizer.anonymize_sync("Ten sam PESEL 50810112354", mapping)
second.anonymized  # "Ten sam PESEL <PESEL_1>"
mapping = second.mapping  # everything so far, for the next turn

A token restores the form it replaced, unless the model names a case. Masking happens on the text as written, so a name masked in the genitive comes back in the genitive. The model may write the case its own sentence needs after a colon, and a person's name comes back declined: Szanowny Panie <OSOBA_1:wołacz> gives "Szanowny Panie Janie Kowalski" where the text said "Jana Kowalskiego". token_instructions("pl") asks for it, and every restore understands it, streams and the gateway included. Declined are names of one or two words the rules read for sure: a listed first name and a surname such as Kowalski, Nowak, Bednarek, Mickiewicz or Duda. A surname alone that may be a man's or a woman's, a foreign name, a company and a place come back as written, and the case is dropped. Spellings that differ only in letter case, in the kind of whitespace (a no-break space, a line break, a run of spaces) or in invisible characters are the same value, and so are an identifier's groups written apart or together: they share its token, and restore writes the spelling seen first. Passwords and custom types are the exception, since two passwords that differ in case are two passwords, and a password never shares a token with a name that reads alike. A one-word name found in all lowercase keeps a token of its own too, for the reason below.

Every occurrence of a masked value is masked, also where no detector would recognise it on its own: as written, in capitals, across a line break, with a zero-width character in it, with its groups spaced differently, and around whatever a detector found inside it. Two exceptions keep ordinary words readable. A capitalised one-word name is not matched in all lowercase, because Zima, Kot and Wilk are surnames and "zima" is winter; identifiers such as a VIN or an e-mail address are matched in any case. A number shorter than six digits, such as a PIN, is not matched inside a longer number, a date, a time or an amount: "15.04.2025", "2024–2025", "2025,50" and "20250" stay readable when the PIN is 2025, while "w 2025 roku" and "PIN/PUK 2025/36912874" do not.

A token you write stays yours. A placeholder such as <PESEL_1> in your own text is never handed to real data. It joins the mapping as standing for itself, so the reservation holds in later turns too.

The mapping is the sensitive half. result.anonymized holds a token for everything the detectors found, and only what they missed in the clear; result.mapping is the key that undoes it. Store the mapping the way you would store the original document.

Replaying a conversation? mask_known_values(text, mapping) masks every value a mapping already holds in text that was masked earlier, such as history sent again after a new value got its token. The same call checks a payload before it leaves: mask_known_values(payload, mapping) == payload means no value the mapping holds is left where the library would mask it.

Words of your own. In-house names no rule can know, such as a project codename or a client, go on a denylist and are masked wherever they appear, in any case and across line breaks:

guards = default_guards(denylist=["Projekt Wichura", "Zakład Nowak"], with_ner=False)

Checking the answer. After restoring, anonymizer.sweep(restored, mapping) reports tokens the model invented and personal data its answer added: report.is_clean, or report.unknown_tokens and report.new_spans to look at.

Choosing what to mask

Anonymizer(guards) masks DEFAULT_ENABLED: every type in ENTITY_TYPES except DATE, because a date rarely identifies anyone and masking every deadline leaves the model unable to answer. A birth date does identify someone, so a date after "ur.", "data urodzenia", "DOB" or "geb." is its own type, BIRTH_DATE, and stays masked. Pass enabled to choose for yourself:

from polanon import DEFAULT_ENABLED, ENTITY_TYPES, Anonymizer, default_guards

guards = default_guards(with_ner=False)
Anonymizer(guards, enabled=DEFAULT_ENABLED | {"DATE"})  # dates too
Anonymizer(guards, enabled={"PESEL", "NIP", "IBAN"})  # identifiers only
list(ENTITY_TYPES)  # every name you can use

A name that is neither built in nor declared in a guard's entity_types raises ValueError, so a typo such as "PERSONS" cannot quietly switch masking off.

A second opinion: the judge

Rules and a language model miss what identifies someone only by context: "the finance director of the only steelworks in town". A judge is a small model that reads the masked text after stage 1 and flags such fragments. Run it locally with Ollama, so the review never leaves your network:

import asyncio

from polanon import Anonymizer, JudgeError, default_guards
from polanon.judge import OllamaJudge

anonymizer = Anonymizer(
    default_guards(),
    judge=OllamaJudge(model="qwen2.5:1.5b"),
    judge_fallback="block",  # refuse to go on if the judge is down
)


async def main(text: str) -> None:
    try:
        result = await anonymizer.anonymize(text)
    except JudgeError:
        ...  # the judge was down, and "block" means nothing goes out


asyncio.run(main("..."))

It needs pip install "polanon[ner,judge]", the spaCy model, a running Ollama and ollama pull qwen2.5:1.5b. With judge_fallback="send_stage1" the call goes on when the judge is down, and result.judge_failed says that stage 1 went out alone.

Judges are asynchronous, so anonymize_sync refuses an anonymizer that has one. RemoteJudge asks an OpenAI-compatible endpoint instead; the masked text then leaves your network, which makes it a testing aid rather than a privacy control.

Names that should stay readable

A language model flags "Facebook" or "Google Ads" as an organisation, which masks the very thing a question is about. Pass an allowlist and those detections are dropped before tokens are assigned:

from polanon import WELL_KNOWN_NAMES, Allowlist, Anonymizer, default_guards

anonymizer = Anonymizer(default_guards(), allowlist=Allowlist(WELL_KNOWN_NAMES))

WELL_KNOWN_NAMES holds platforms and office tools, and public institutions and filing programs such as ZUS, Płatnik, NFZ and KSeF. A one-word name also matches its Polish case forms ("Facebooku", "Canvie", "ZUS-ie"). Only detections of names are dropped, and only when every word in them is on the list; identifiers, the denylist and values a conversation has already masked stay masked.

The pieces

Module What it is for
guards/ deterministic detectors. Each returns Spans in original-text coordinates
pipeline.py runs the guards in a thread, resolves overlaps, assigns stable tokens
policy.py the entity taxonomy and the token naming (<OSOBA_1>, <PESEL_1>)
judge/ the optional second stage: a model reviews the masked text for what still identifies
stream_restore.py puts originals back into a streamed reply without splitting a token
turn.py masks everything one call to a model sends, with one mapping, and restores what it gets back
pseudonyms.py computes stable tokens from a secret key, the same in every document and process
index.py masks documents and questions for a search index, with the store of what tokens hide
integrations/haystack.py Haystack components that mask documents and questions and restore answers
gateway/ a gateway compatible with the OpenAI API, run on the operator's own machine
output_sweep.py flags invented tokens and personal data the answer introduced

Adding a detector

Subclass Guard, return spans in original-text coordinates and respect the enabled set. A type of your own goes in entity_types, which turns it on by default; it becomes the label of its tokens, so it is written in uppercase letters and underscores:

import re

from polanon import Anonymizer, Guard, Span, default_guards


class EmployeeIdGuard(Guard):
    name = "employee_id"
    entity_types = frozenset({"EMPLOYEE_ID"})

    def detect(self, text: str, enabled: set[str]) -> list[Span]:
        if "EMPLOYEE_ID" not in enabled:
            return []
        return [
            Span(m.start(), m.end(), "EMPLOYEE_ID", 1.0, self.name)
            for m in re.finditer(r"\bEMP-\d{5}\b", text)
        ]


anonymizer = Anonymizer([*default_guards(with_ner=False), EmployeeIdGuard()])
anonymizer.anonymize_sync("Wniosek EMP-11111").anonymized  # "Wniosek <EMPLOYEE_ID_1>"

The score decides who wins where detections overlap: 1.0 for a value a checksum, a label or an official format vouches for, 0.85 for a statistical guess; the judge's findings score 0.5. A Polish name with its surname scores 0.9 and a first name alone 0.8, so a guess of a place wins over it. A secret is masked whole, with whatever was detected inside it. A span outside the text, a type spelled any other way, or one spelled like a built-in token label (OSOBA, DATA) raises ValueError naming your guard.

Before you rely on it

Masking is pseudonymisation, not anonymisation in the legal sense, and no detector catches everything. The threat model says what is covered, what is not, and what stays your application's job, starting with keeping the mapping safe.

polanon was built as the masking engine of PolGate, a self-hosted hub that lets a team use Claude or GPT on real documents without sending personal data to the provider. The library knows nothing about it — no HTTP, no database, no settings — and a test fails if it ever starts to.

Used by

PolGate is a self-hosted company chat for Claude and GPT built on polanon: every message and uploaded document is masked on the server before it reaches the model, and the answer streams back with the originals in place. It adds what a team needs around the library: an encrypted vault for the mapping, retention that deletes, an audit log of types and counts, and an admin panel for the policy, including the allowlist and denylist.

PolCV writes CVs for Polish job postings and can mask the candidate's name, contact details and company names with polanon before any text reaches the model, then put them back into the answer. It adds a guard of its own for what the profile already knows, such as a LinkedIn link with the name in it, as Adding a detector describes. polanon stays optional there: the app installs it on request, talks to it through a small JSON-lines bridge, and sends nothing at all while masking is on but polanon is not working.

Contributing

Bug reports, missed values and new detectors are welcome: see CONTRIBUTING.md. A way to get an original value past the masking is a security report, made privately as SECURITY.md describes.

MIT licensed.