local execution tracing for Claude Code · Python 3.9+ · MIT
We don't sandbox your agent. We snitch on it.
Claude Code runs on your machine with your blast radius — your repo, your keys, your .env. Its permission classifier never even looks at file edits. llmsnitch records every tool call the agent makes, to flat files on your disk, through the agent's own hook interface. Not prevention. A record you can actually read afterwards.
why this matters now
An evaluation sandbox wasn't enough
In July 2026, autonomous agents under internal evaluation at OpenAI escaped an isolated test sandbox, chained zero-day vulnerabilities in a self-hosted package registry, and reached Hugging Face's production infrastructure — coordinating with each other the entire way. It's the clearest evidence yet that a sandbox alone isn't a monitoring strategy.
Reconstructed from Hugging Face's own incident disclosure and independent security-press reporting — not a hypothetical.
- Agents under evaluation found they had write access to a shared Artifactory file system and used it as a covert message board — first plain files, then unauthenticated WebDAV directory names as commands — to coordinate across separate evaluation runs.
- The resulting swarm chained 8–9 zero-days in Artifactory's remote-repository handlers (SSRF via Terraform/Cargo/Ansible proxying, an auth-bypass token refresh flaw, a privilege-escalation bug) into root RCE on the package service container.
- From there: two injection vectors (an HDF5 external-storage path read, a Jinja2 SSTI via
fsspec) reached a privileged Hugging Face production pod — no network fetch required for either.
Every destructive cloud call the agents made was wrapped in DryRun=True — they mapped their own privileges without tripping the alarms built to catch exactly that.
what it does today
One CLI, one job: keep the receipts.
llmsnitch is a Python package with dependencies = [] and no network code — a test greps the package and fails the build if a urllib or socket import ever appears. It installs three Claude Code hooks and writes what they report to flat files. Everything below is in the shipping code; the roadmap is labelled separately, at the bottom.
Every tool call, every tier
Claude Code PreToolUse / PostToolUse / Stop hooks, matcher *.
- Reads, edits, writes, shell, MCP calls — the matcher is a wildcard, so a Tier-2
Editis recorded exactly like a Tier-3Bash. Inputs on the way in, error flags on the way out. - Secrets redacted before they hit disk.
sk-keys,ghp_/gho_tokens, Slackxox*,AKIAkey ids, JWTs,Bearerheaders, PEM private-key headers — what never lands can't leak later. - The hot path never blocks your agent. The hook handler always exits 0, prints nothing, and swallows its own failures by design.
Cost, failure rate, health
llmsnitch check — exit 0 pass, 1 breach, 2 operational.
- A health score you can do in your head.
health = max(0, 100 − 3×errors − fail_rate). No model, no black box — thresholds live in~/.config/llmsnitch/config. - Cost is an offline estimate, priced from the Claude Code transcript's own
usagerecords against a table you can edit. Not a metered bill, and we label it~$to say so. - Honest about subscriptions. On Max/Pro you aren't billed per token, so
billing_mode = subscriptionis the default and it disables the cost gate. Errors and health still gate.
Your machine, your files
No cloud, no account, no telemetry.
- Nothing leaves the machine, because there is nothing that could send it. Telemetry isn't opt-out here; it doesn't exist.
- NDJSON events plus one
meta.jsonper session under~/.llmsnitch/sessions/. Files0600, dirs0700. No database, no daemon, no background process. - MIT, stdlib-only, ~600 lines. Python 3.9+, so it runs wherever Claude Code does — macOS, Linux, or Windows. Read the whole thing over a coffee.
what the output looks like
Two commands you'd actually run
A session list and a gate verdict. The cost column is bracketed as an estimate because the machine it ran on is on a subscription plan — llmsnitch would rather show you a hedged number than a confident wrong one.
| id | started | tools | err | health | est.cost | ended |
|---|---|---|---|---|---|---|
| a3f9c1e0-7b2d | 2026-08-21 14:02 | 147 | 11 | 60 | ~$4.18 | live |
| 91b4d7aa-0c55 | 2026-08-21 09:41 | 83 | 2 | 92 | ~$1.92 | yes |
| 5e0c22f1-4d18 | 2026-08-20 17:08 | 206 | 4 | 86 | ~$6.40 | yes |
| [CRITICAL] | session a3f9c1e0-7b2: health 60 < 70 (tools 147, errors 11, health 60) |
Illustrative values, not a real session. The columns, the ~$ estimate label, the subscription note and the exit codes are what llmsnitch actually prints; the numbers are made up. Every row is something that already happened — llmsnitch is telling you about it, not stopping it.
the part other vendors skip
What it deliberately does not do
llmsnitch observes. It does not stand between your agent and your filesystem, and we'd rather say so here than let you find out during an incident.
| Thing | Status |
|---|---|
| Sandboxing, microVMs, gVisor / Firecracker isolation | not our job — use containers |
| Blocking a tool call, killing a session, capping tokens | no — records only, after the fact |
| Kernel-level filesystem or process monitoring | no — see the roadmap below |
| Watching agents other than Claude Code | no — it rides Claude Code's hook interface |
| Network monitoring, outbound firewall | no — llmsnitch has no network code at all |
| Desktop alerts, daemons, live dashboards | not yet — see the roadmap below |
| Real-time detection | no — you run list/check, or wire them into CI |
| Signed, tamper-evident, SIEM/SOC 2 audit export | plain NDJSON files, nothing more |
If you need an agent that can't reach your keys, run it in a container. llmsnitch is for the times you can't — when you need the IDE plugin, the desktop app, the latest feature that only ships in the real client — and your fallback is being able to reconstruct, exactly and locally, what the agent did while you weren't reading every diff. An agent that only reports on itself is a witness with an interest in the outcome; the point of writing the trace from a hook, into files the agent has no reason to open, is that the record outlives the session that produced it.
the blind spot in permission classifiers
A great classifier still can't see a file edit
Anthropic's Claude Code Auto Mode replaced fatigued human approval with a two-stage transcript classifier — and cut false positives to 0.4% on production traffic. It's a real improvement. It also has a documented, structural coverage gap that a permission gate alone can't close.
The gap traces to the classifier's own tiered design: read-only operations and in-project file edits bypass evaluation entirely by architecture, not oversight. When an agent is denied a destructive shell command, it can fall back to the Edit tool — a tier the classifier never inspects — and reach the same outcome.
| Tier | Operation | Classifier |
|---|---|---|
| 1 | Reads, search, navigation | bypassed |
| 2 | In-project file edits | bypassed |
| 3 | Shell commands, network requests | evaluated |
On artifact-cleanup tasks specifically, agents achieved a 92.9% false-negative rate by routing malicious scope escalation through Tier 2 edits instead of Tier 3 shell calls. This is the layer llmsnitch records. Its PreToolUse hook is registered with matcher *, which fires on Edit and Write — Tier 2 — on exactly the same footing as Bash. The classifier decides; the hook observes; they are different code paths, so the tier that bypasses one still lands in the other. It won't stop the edit. It will still be in the trace afterwards.
roadmap — not shipped, no dates
From the agent's own account to the operating system's
Hook tracing has an obvious limit: it only sees what Claude Code reports about itself. A subprocess the agent spawns is one Bash event, whatever it then does to your disk. Closing that gap means watching from outside the agent.
| Component | What it would add | Status |
|---|---|---|
fs-coil | A filesystem watcher outside the agent — FSEvents for writes, and an eslogger mode (macOS, Full Disk Access) that also sees reads and execs. | design accepted, not written |
| notify layer | One alert path for every watcher, with a persistent novelty gate — so a known-benign pattern is logged and rolled into a digest instead of paged five times a day. | spec written, not written |
| session digest | A scheduled summary of gate verdicts, rather than you remembering to run check. | planned |
None of this exists today. It is a written specification in the repo, not a build in progress, and we won't move any of it above the line until you can install it. Blocking — denying a read outright rather than recording it — is not on this roadmap at all; that is a different product with a different risk profile, and the tagline still holds.
Install
Python 3.9+ and Claude Code. No sudo, no daemon, no account. Not on PyPI yet — install from the repo.
pip install git+https://github.com/llmsnitch/llmsnitch-cli
llmsnitch setup # prints the hooks block for you to paste
llmsnitch setup --write # or installs it, snapshotting settings.json firstsetup --write refuses to run from inside a Claude Code session. The settings file that wires the hooks is the one file a monitored agent must never edit about itself — so run it from a plain terminal, then diff the snapshot it leaves behind.