Loading ToolBench
Agent firewall daemon: inspects Claude Code tool calls via native hooks. Detection engine with injection/PII/secret detectors, risk scoring, and policy enforcement.
soup-wall is a specialized agent firewall with 12 tools covering sandbox execution, policy approval, and MCP proxying. Strengths: all tools have descriptions (10-200 chars, well-targeted); input schemas are present with types and descriptions for most parameters; tool names are action-verb based (Serve, Replay, Approve, GuardedShell). Weaknesses: output schemas are not documented anywhere in the source; error handling guidance is absent (no recovery hints, no error categorization); no tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite high-risk operations; descriptions are functional but minimal (avg ~80 chars, below the 194-char baseline for A+ tools); no pagination or result-limiting patterns visible; security-critical tools like GuardedShell and Sandbox lack explicit permission gates or audit trail documentation in the schema itself.
Approve one exact tool call that policy would otherwise only ask about. The approval is signed, expires, is single-use, and is bound to the exact arguments given here.
Inspect a typed file deletion and apply it only inside the workspace.
Inspect a typed WebFetch and retrieve it only through an egress proxy.
Inspect a typed file rename and apply it only inside the workspace.
Inspect a Bash command and execute it only after an explicit allow inside the Linux bubblewrap sandbox.
Inspect a typed file write and apply it only inside the workspace.
Print the settings.json hook block and setup instructions.
Proxy an MCP server, pinning its tool manifest at handshake.
Output schemas not documented. No tool returns a documented response structure. LLMs cannot plan downstream calls or extract required fields (e.g., what does Replay return? What fields does Approve return for the signed token?).
No tool annotations despite high-risk operations. GuardedDelete, GuardedShell, Sandbox are destructive/irreversible but lack destructiveHint=true. Approve and Preflight are read-only but lack readOnlyHint=true. Annotations guide LLM caution and planning.
No error handling guidance. Tools like GuardedShell and Sandbox can fail (timeout, sandbox violation, command error) but provide no recovery hints. LLMs cannot distinguish retryable vs fatal errors or know what to do next.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Check that the daemon is up, and report its enforcement posture. Exits non-zero when it is not.
Summarize an audit log: what would this policy have done?
Run one high-risk command inside the Linux bubblewrap sandbox.
Run the daemon.
Serve and Install have no input schema (empty or missing). Serve is a daemon startup with no parameters; Install is a read-only info tool. Both should have explicit empty schemas or be documented as taking no input.
Descriptions are minimal (avg 70 chars vs 194-char baseline). E.g., 'Run the daemon' (Serve), 'Print the settings.json hook block' (Install). These lack context on WHEN to call, WHAT happens, and WHY. Descriptions should be 50-200 chars and answer: what, when, prerequisites.