Loading ToolBench
zeromcp exhibits severe definition quality issues. Tool definitions are minimally documented: most descriptions are under 50 characters (well below the 194-char baseline for production tools). Parameter descriptions are sparse or missing entirely (e.g., 'fetch_evil', 'tokenstore_check', 'hang', 'leak_memory' have empty input objects with no guidance). Duplicate tool names ('hello' appears 4 times, 'slow' appears 2 times) indicate poor composition and naming discipline. Tools like 'stdout_corrupt', 'leak_memory', and 'hang' appear to be test/chaos fixtures, not production tools. Input schemas are present but minimal, most lack meaningful parameter descriptions. No output schemas are documented. Error handling is absent: tools like 'throw_error' provide no recovery guidance. The server lacks tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite declaring all tools as READ_ONLY. Overall, this reads as a test harness rather than a production-grade MCP server.
Add two numbers together
Check if credentials were injected
Echo a message back
Fetch an allowed domain
Fetch a blocked domain
Tool that tries a domain NOT in allowlist
Tool with network disabled
Tool with no network restrictions
Duplicate tool names ('hello' ×4, 'slow' ×2) violate single-responsibility principle and confuse LLM tool selection. Each duplicate should be renamed to reflect its distinct purpose or removed.
12 tools have empty input schemas (no parameters documented): fetch_evil, tokenstore_check, throw_error, hang, leak_memory, stdout_corrupt, crm_check_creds, nocreds_check_creds, fetch_allowed, fetch_blocked, fetch_no_network, fetch_unrestricted, fetch_wildcard.
Most tool descriptions are under 50 characters (e.g., 'Say hello', 'Tool that hangs forever', 'Tool that leaks memory'). Baseline is 194 chars. Descriptions lack context on WHEN to use the tool, WHAT it returns, and any prerequisites.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | <=2025-11-25 | v2 |
Tool with wildcard network permission
Greet a person by name
Tool that hangs forever
Say hello
Say hello to someone
Fast tool
Say hello to someone
Tool that leaks memory
Check credentials in unconfigured namespace
Tool that takes 3 seconds
Tool that takes 3 seconds
Tool that writes to stdout
Tool that throws
Return the current token from credentials
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing what each tool returns. Production tools must document return types.
Tools like 'throw_error', 'hang', 'leak_memory', 'stdout_corrupt' appear to be test/chaos fixtures, not production tools. They should be removed from the main tool registry or clearly marked as internal/testing-only to avoid accidental agent invocation.