Loading ToolBench
ZeroMCP exhibits significant definition quality gaps. While 20 tools are registered with basic descriptions and input schemas, most descriptions are extremely terse (10-30 chars), falling well below the 50-200 char baseline for LLM-optimized descriptions. Parameter descriptions are largely absent, most tools have parameters with no explanation of what they control or accept. Schema completeness varies: simple tools like 'hello' and 'add' have minimal but present schemas, while chaos/sandbox test tools lack meaningful parameter documentation. No tool declares output schemas, error handling guidance, or recovery paths. The codebase shows DSL-based registration (ToolBuilder pattern) with schema generation, but the actual tool definitions lack depth. Naming is inconsistent: 'fetch_url', 'fetch_allowed', 'fetch_blocked', 'fetch_evil' are confusingly similar and suggest multiple responsibilities bundled together. Test-oriented tools (throw_error, hang, leak_memory, stdout_corrupt) are not production-grade and should not be exposed in a real MCP server.
Add two numbers together
Check if credentials were injected
Echo a message back
Fetch an allowed domain
Fetch a blocked domain
Tool that tries a domain NOT in allowlist
Tool with network disabled
Tool with no network restrictions
Descriptions critically short: 19 of 20 tools have descriptions under 40 characters, far below the 50-200 char baseline. Examples: 'Tool that throws' (16 chars), 'Tool that hangs forever' (23 chars).
Parameter descriptions almost entirely absent. Tools like 'fetch_url' have a 'url' parameter with description 'The URL to fetch', but most sandbox/test tools (fetch_allowed, fetch_blocked, etc.) have empty input schemas with no parameter guidance. LLMs cannot infer what parameters control without descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Fetch a URL (with permission)
Tool with wildcard network permission
Greet a person by name
Tool that hangs forever
Say hello to someone
Tool that leaks memory
Check credentials in unconfigured namespace
Tool that takes 3 seconds
Tool that takes 3 seconds
Tool that writes to stdout
Tool that throws
Return the current token from credentials
No output schemas documented. The rubric requires 'Document the output schema. LLMs need to know what fields to expect.' None of the 20 tools declare what they return, forcing LLMs to guess and breaking tool chaining.
Confusing naming: 'fetch_allowed', 'fetch_blocked', 'fetch_no_network', 'fetch_unrestricted', 'fetch_wildcard', 'fetch_evil' are all variants of the same operation with unclear distinctions. They appear to be test fixtures, not production tools.
Test/chaos tools exposed in production API: 'throw_error', 'hang', 'leak_memory', 'stdout_corrupt' are clearly test fixtures (from ChaosTest.kt, CredentialTest.kt). These should not be registered in a production MCP server. They violate the single-responsibility principle and confuse agents.