Loading ToolBench
Black-box security scanner for MCP servers — fires curated single-call probes and classifies findings. No LLM in the loop.
jakk is a security scanner, not a production MCP server. Its 5 tools are intentionally vulnerable (BOLA, cross-tenant auth bypass, secret exposure) to demonstrate attack patterns. Tool definitions exist but are designed to showcase flaws, not follow best practices. Naming is clear (fetch_project, list_projects, update_project, get_project_settings, echo), but descriptions explicitly document authorization failures rather than guiding safe usage. Schemas are present but minimal. The server's purpose is red-team education, not production deployment, scoring reflects this context.
Return a benign echo response with no secrets or directives.
Read a CRM project by ID. AUTHENTICATED but NOT AUTHORIZED — any valid api_key can read any project regardless of tenant.
Read project settings including notification config. AUTHENTICATED but NOT AUTHORIZED — webhook secrets returned without tenant check.
List CRM projects. AUTHENTICATED but NOT AUTHORIZED — returns ALL projects across ALL tenants, not just the caller's.
Modify a CRM project's fields. AUTHENTICATED but NOT AUTHORIZED — any valid api_key can modify any project regardless of tenant. This is the BOLA-write variant that the depth-of-exposure methodology uses to demonstrate escalation from read-only to read-write impact.
Credentials exposed as tool parameters. All four CRM tools require 'api_key' as a parameter, which will be logged in agent traces and prompt history. Violates pattern:secret-injection.
Descriptions document authorization flaws rather than guiding correct usage. E.g., 'AUTHENTICATED but NOT AUTHORIZED, any valid api_key can read any project' is a security warning, not a usage guide. LLMs cannot infer when to call these tools safely.
No error handling guidance. Tools return raw strings ('Project not found', 'Invalid API key') without actionable recovery steps. LLMs cannot determine if errors are retryable or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No permission gates or scope declarations. Tools accept any valid API key without checking tenant ownership. Destructive tool (update_project) has no confirmation or dry-run option.
Output schemas not documented. Tools return free-text JSON strings; LLMs must parse unstructured output. No pagination, no field descriptions, no chaining IDs.