Loading ToolBench
Penetration testing for AI agents: finds and PROVES cross-server confused-deputy exfiltration chains in an MCP tool mesh, measures whether a real LLM falls for them, gates them at runtime, and runs as an MCP server any agent can call.
Renfield defines 4 security-focused tools with clear, domain-specific names (renfield_audit, renfield_scan, renfield_verify, renfield_remediate). All tools have descriptions (100 - 200 chars, within baseline 194 avg). Input schemas are present with typed parameters and descriptions. However, output schemas are not documented, LLMs cannot predict what these tools return, limiting composability. Error handling is minimal; no recovery guidance or actionable error messages. Tool descriptions lack WHEN/WHY context (e.g., when to call scan vs audit). Parameters lack constraints (e.g., max is unbounded integer). No tool annotations (readOnlyHint, destructiveHint). Security-critical tools (verify, remediate) lack explicit permission gates or audit trail declarations.
One-shot agent pentest: enumerate the target agent's MCP mesh, find cross-server confused-deputy exfiltration chains, PROVE each by a real side effect (canary egress), and compute the minimal config fix. Returns structured findings.
Compute the smallest set of capabilities to remove that breaks EVERY critical chain, and confirm 0 remain.
Map the target MCP mesh and list CANDIDATE cross-server toxic chains (capability analysis only — does not execute).
PROVE critical chains by planting a payload and observing a real side effect (canary secret reaching an external sink).
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Breaks tool chaining and forces agents to guess what data is available.
Parameter 'max' is unbounded integer with no min/max constraints. Description lacks format guidance (e.g., 'integer 1 - 100'). LLMs may pass absurd values.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Renfield tools are read-only security audits, but LLMs cannot infer safety from names alone.
Descriptions lack WHEN/WHY context. 'Map the target MCP mesh' (scan) vs 'enumerate the target agent's MCP mesh' (audit), unclear when to call each. No dependency hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No error handling guidance. If config_path is invalid or unreachable, no recovery hint (e.g., 'Verify path exists and is readable'). Agents cannot self-correct.