Loading ToolBench
MCP server that dynamically registers and executes shell scripts from a commands directory as tools
Both tools lack input schemas entirely (empty {} objects), descriptions are minimal (under 20 chars for ping.sh), and there is no parameter documentation. Tool names do not follow verb_noun convention (hello.py, ping.sh are filenames, not action verbs). No output schema is documented. Error handling is absent, subprocess failures return JSON but with no recovery guidance. The server dynamically registers tools by scanning a directory, making explicit schema definitions invisible in the codebase.
A simple command that says hello.
A simple ping / pong command.
No input schemas defined. Both tools declare Input: {} with no parameters, types, or constraints. Per HARD SCORING RULE, schema score must be 0.
Tool names are filenames (hello.py, ping.sh), not action verbs. LLMs cannot infer intent from 'hello.py', should be 'run_hello' or 'greet_user'. Violates verb_noun naming convention.
ping.sh description is 'A simple ping / pong command.' (34 chars), under 20-char threshold. Per HARD SCORING RULE, description score capped at 0-20. No explanation of when/why to call it.
No output schema documented. exec_command returns JSON with stdout, stderr, exit_code, but LLMs have no formal schema to parse or plan downstream calls. Violates response-shaper pattern.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 21 | 2026-07-28+ | v2 |
Error handling is minimal. subprocess failures return JSON errors but provide no recovery guidance (e.g., 'Failed to set execute permission' with no next steps). Violates recovery-guide pattern.