Loading ToolBench
A FastAPI-based MCP server that loads and executes bash scripts and markdown agent files from an agents-library directory, providing tools for cloud resource management, deployment, monitoring, and health checks.
This server has severe definition quality issues. All 7 tools have identical, generic schemas and descriptions that fail to meet production standards. The schema uses 'additionalProperties: true' and lacks required field definitions, making it permissive rather than constraining. Descriptions are boilerplate and do not explain WHEN or WHY to use each tool. Parameters lack typed descriptions. No output schemas are documented. Tools wrapping bash scripts expose execution details rather than abstracting them. Risk annotations (IRREVERSIBLE, DESTRUCTIVE) are present but not coupled to confirmation patterns.
Executes the cost_optimizer.sh script and returns its output.
Executes the deploy_app.sh script and returns its output.
Executes the health_check.sh script and returns its output.
Executes the list_markdown_files.sh script and returns its output.
Executes the manage_resource.sh script and returns its output.
Executes the monitor_logs.sh script and returns its output.
Executes the uptime.sh script and returns its output.
Generic, identical descriptions across all tools. Each description is just 'Executes the [tool_name].sh script and returns its output.' This provides zero guidance for LLM tool selection. Per the rubric, descriptions must answer WHAT the tool does, WHEN to use it, and what it returns. These fail on all three counts.
Parameter descriptions are missing entirely. The schema shows three parameters (project_name, mcp_server_url, script_timeout) but provides almost no semantic guidance. For example, 'project_name' is labeled 'The name of the new project' but this description is confusing for tools like 'monitor_logs' or 'health_check' that do not create projects. LLMs cannot infer parameter meaning from names alone.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 24 | - | v1 |
Input schema uses 'additionalProperties: true' and declares no required fields. This is a permissive anti-pattern that invites invalid LLM calls. The schema should be strict: declare which parameters are required, their types, and any constraints (enums, ranges, patterns). The absence of 'required' array means LLMs might call tools with zero parameters.
No output schemas documented. Tools return 'TextContent' (per the code line 'mcp.types.TextContent') but the structure of that response is not described. LLMs need to know what fields to expect in the response to plan downstream calls and extract relevant data.
Destructive tools (deploy_app, manage_resource) are marked IRREVERSIBLE/DESTRUCTIVE but have no confirmation pattern. Per the rubric, irreversible operations should support a dry-run or confirmation step to prevent catastrophic agent errors. The risk annotation alone is insufficient.
Parameter naming is inconsistent and overloaded. All tools accept 'project_name' (but health_check and monitor_logs do not create projects), 'mcp_server_url' (not all tools interact with an MCP server), and 'script_timeout' (opaque implementation detail leaking to LLM). The schema does not match the semantic intent of each tool.
Tool names do not clearly distinguish intent. 'manage_resource' is vague, does it create, update, delete, or list resources? Ambiguous names cause LLMs to misselect tools. Compare to 'delete_resource', 'update_resource', 'list_resources' which are unambiguous.
No error handling guidance documented. Tools can fail (e.g. script timeout, missing file) but there is no indication of what errors are retryable, what the LLM should do next, or how to interpret error responses. Per the rubric, errors must tell the LLM what to do: 'User not found. Try search_users() with a partial name.'
Implementation exposes bash script execution details (timeout in seconds, script file paths) in the tool interface. This violates the abstraction principle, users say 'Check the health of my app', not 'Execute health_check.sh with a 60-second timeout'. The schema should abstract these details and expose only semantic parameters.