Loading ToolBench
A runtime gate for autonomous coding agents. It blocks destructive tool calls (force-push main, rm -rf, secret exfil) while letting real work through. Its git-branch sub-policy is machine-checked by z3; the rest is high-precision heuristics.
qedra defines 6 tools with clear security-focused purposes. Three execution tools (run_shell, write_file, git) have concise descriptions and basic input schemas with type and description fields. Three audit/receipt tools (audit_log, policy_info, session_receipt) are read-only and well-described. However, parameter descriptions lack actionable constraints (e.g., no format specs, no examples of valid git args), and output schemas are entirely undocumented. Error handling is implicit (tool descriptions mention blocking but don't guide recovery). The tool set is well-composed (each does one thing) and naming is clear (verb_noun pattern), but lacks the depth expected for production-grade agent tools.
Return the tamper-evident audit chain of every guardrail decision so far.
Run a git command, e.g. 'add -A', 'commit -m fix'. A force-push, hard-reset, or rebase of a protected branch is blocked; normal git is executed.
Report the exact ruleset being enforced: the committed policy id and root (content hash), the protected branches, and any opt-in presets/extra patterns. A relying party can hold the same spec and confirm the receipt's policy root matches, so 'the gate enforces this policy' is checkable, not a promise.
Run a shell command in the workspace. Blocked if it is a destructive action (rm -rf the repo, secret exfiltration, force-push); otherwise executed.
Export a signed, independently-verifiable RECEIPT for this session: the agent id, the committed policy, every gated action with its verdict, and an Ed25519 signature over the tamper-evident chain head. A third party (an auditor, a bank, an insurer) can verify it with the PUBLIC KEY ALONE via qedra.control_plane.verify_receipt, proving the agent stayed within policy, without trusting this server or its operator, and catching any forged ALLOW.
Output schemas completely undocumented. No tool documents what fields it returns, their types, or structure. LLMs cannot plan downstream calls or extract data reliably.
Parameter descriptions lack actionable constraints. 'git arguments, without the leading git' does not specify format, valid subcommands, or examples. 'the shell command to run' does not hint at what commands are blocked or how to structure safe commands.
No error recovery guidance. Descriptions state what is blocked (e.g., 'force-push is blocked') but do not tell the LLM what to do instead or how to reformulate the request.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Write a file in the workspace. Blocked if it writes a secret or empties a CI config; otherwise executed.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The MCP server does not signal which tools are read-only (audit_log, policy_info, session_receipt) or destructive (run_shell, write_file, git), forcing LLMs to infer intent from descriptions alone.
STDIO transport only. Server is not remotely accessible and cannot be used by hosted MCP clients (Claude Desktop, Cursor, etc.). Limits deployment flexibility.