Loading ToolBench
create-mcp is a CLI scaffolding tool that generates MCP server templates. The evaluated server ('create-mcp' itself) is minimally functional as an MCP server, it contains only one tool (getJokes) in the template it generates. The tool has a basic description and input schema, but falls well short of production-grade quality. The description is terse (12 chars: 'Gets n random...'), lacks depth about when/why to use the tool, and provides no guidance on error recovery or result structure. The input schema defines 'n' as a number but lacks range constraints, defaults, or validation guidance. The output schema is never documented, callers cannot know what fields to expect. No error handling guidance is provided.
Gets n random programming jokes using the JokeAPI.
Tool description is incomplete and does not meet minimum quality standards. 'Gets n random programming jokes using the JokeAPI.' is 12 characters, well under the 34-char p10 baseline and fails to explain when/why to use the tool or what it returns.
Input parameter 'n' has no range constraints, defaults, or validation guidance. LLMs cannot determine safe values, should describe bounds (e.g. '1-20') and default.
Output schema is not documented. The tool returns a structured response with 'content' array containing 'text' field, but this is visible only in implementation, not exposed to callers. LLMs have no way to know what fields to expect.
No error handling or recovery guidance. The tool calls an external API (JokeAPI) with no timeout, retry logic, or error message guidance. If the API fails, the LLM receives a generic error with no actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 22 | - | v1 |
Tool naming does not clearly indicate side effects or idempotency. While getJokes is read-only, the pattern should be explicit in metadata (risk: READ_ONLY is present but not integrated into schema annotations).