Loading ToolBench
zeromcp is a Java MCP framework with 23 tools, mostly test/example fixtures. Critical issues: (1) Duplicate tool names ('hello' appears 4 times, 'slow' twice), LLMs cannot disambiguate. (2) Most descriptions are under 20 chars ('Say hello', 'Echo a message back', 'Tool that hangs forever'), below the 34-char p10 baseline and insufficient for LLM selection. (3) Many tools lack input descriptions entirely (fetch_evil, tokenstore_check, throw_error, hang, slow, leak_memory, stdout_corrupt, crm_check_creds, nocreds_check_creds, fetch_allowed, fetch_blocked, fetch_no_network, fetch_unrestricted, fetch_wildcard). (4) No output schemas documented anywhere. (5) No error handling guidance, tools like 'throw_error' and 'hang' have no recovery hints. (6) Tools are test fixtures (ChaosTest, BypassTest, CredentialTest, SandboxTest) not production-grade. Positive: Input schemas ARE present and typed for most tools; permissions declared as READ_ONLY.
Add two numbers together
Check if credentials were injected
Echo a message back
Fetch an allowed domain
Fetch a blocked domain
Tool that tries a domain NOT in allowlist
Tool with network disabled
Tool with no network restrictions
Duplicate tool names ('hello' appears 4 times, 'slow' twice), LLMs cannot disambiguate which variant to call. MCP spec requires unique tool names.
14 of 23 tools have NO input parameter descriptions (fetch_evil, tokenstore_check, throw_error, hang, leak_memory, stdout_corrupt, crm_check_creds, nocreds_check_creds, fetch_allowed, fetch_blocked, fetch_no_network, fetch_unrestricted, fetch_wildcard). LLMs cannot infer parameter meaning from names alone.
Most tool descriptions are under 20 characters ('Say hello', 'Tool that hangs forever', 'Echo a message back'), below the 34-char p10 baseline. Descriptions lack context for LLM selection and do not explain WHEN to use the tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Fetch a URL (with permission)
Tool with wildcard network permission
Greet a person by name
Tool that hangs forever
Say hello
Fast tool
Say hello to someone
Say hello
Tool that leaks memory
Check credentials in unconfigured namespace
Tool that takes 3 seconds
Tool that takes 3 seconds
Tool that writes to stdout
Tool that throws
Return the current token from credentials
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields without knowing what each tool returns.
No error handling guidance. Tools like 'throw_error' and 'hang' have no recovery hints, retry guidance, or actionable error messages. Agents cannot self-correct.