Loading ToolBench
MCP server that allows users to dynamically add custom tools/functions at runtime
DIY Tools MCP exhibits mixed quality across its 5 tools. Naming conventions are mostly verb-driven and clear (add_tool, remove_tool, list_tools, update_tool, get_tool), following the verb_noun pattern expected by production tools. However, descriptions vary significantly in depth and actionability. Input schemas are present for all tools with typed parameters and enums where appropriate (language enum in add_tool), but descriptions for individual parameters are inconsistent, some are detailed ('Unique identifier for the tool'), while others are generic ('Optional entry point function name'). The biggest issue is that output schemas are NEVER documented: none of the tool descriptions explain what the response structure will be, preventing LLMs from planning downstream operations or extracting data confidently. Error handling documentation is absent, there's no guidance on what errors are retryable, user-fixable, or fatal, and no recovery suggestions. The tool descriptions themselves (10 - 64 characters) fall below the baseline 194-char average and lack context about when to use each tool or prerequisites. For a server managing dynamic tool execution, the lack of security guardrails in the tool definitions (no mention of sandboxing, permission checks, or audit trails for add_tool/update_tool/remove_tool) is a significant gap given the destructive nature of these operations.
Add a custom tool/function at runtime
Get details about a specific tool
List all available custom tools
Remove a previously added tool
Update an existing tool's code or metadata
Output schemas entirely undocumented. No tool description explains what the response structure will be (e.g., what fields are in a tool object, what list_tools returns). This prevents LLMs from confidently parsing responses or planning chained calls.
Tool descriptions are too short (10 - 64 chars vs. baseline 194 chars) and lack actionability. Descriptions do not explain WHEN to use each tool (e.g., 'Use add_tool to dynamically register custom functions at runtime'), prerequisites, or relationships between tools. LLMs cannot reliably select the right tool.
No error handling guidance in tool descriptions. LLMs do not know which errors are retryable (e.g., timeout on code execution) vs. user-fixable (e.g., invalid language enum) vs. fatal (e.g., permission denied). Error responses in code map to ErrorCode but do not provide recovery suggestions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 45 | - | v1 |
Destructive operations (remove_tool, update_tool) lack confirmation or dry-run capability. An LLM could accidentally remove a critical tool or corrupt its definition with no safeguard. No mention of audit logging or permission gates in tool definitions.
Parameter descriptions are inconsistent and sometimes absent. 'entryPoint' has a description, but 'language' enum description is generic. Parameters like 'code' and 'codePath' (mutually exclusive) lack documentation of this constraint. LLMs will not know when to pass code vs. codePath.
No indication of tool chaining support. If add_tool returns a tool_id, does update_tool and remove_tool accept it? Are there any response fields that downstream tools require? The lack of documented IDs/references breaks potential tool chains.