Loading ToolBench
MCPExec defines a single tool 'exec' with minimal quality standards. The tool has a basic description and a single string parameter, but lacks essential safety features, error handling guidance, and parameter constraints. The tool name 'exec' is generic and does not follow verb_noun conventions (should be 'run_command' or 'execute_command'). No input validation, no output schema documentation, no error recovery guidance. This is a destructive tool that accepts arbitrary shell commands with no safety constraints, parameter validation, or confirmation mechanism. The implementation uses shlex.split() for parsing, which provides some protection against injection, but the tool itself offers no guards against misuse by an LLM.
Run local cli commands
Generic tool name 'exec' does not follow verb_noun convention. Should be 'run_command', 'execute_command', or 'run_shell_command' to signal intent clearly.
Description (24 chars) is far too brief and lacks context for LLM decision-making. Does not explain WHEN to use this tool, what it returns, or dependencies. Should be 50-200 chars with examples of appropriate use cases.
Parameter 'cmd' lacks type specification in the exposed schema (only visible as Python type hint, not in JSON Schema). No description of expected format, constraints, or examples. LLMs cannot infer that this accepts shell commands vs. Python code vs. API endpoints.
No output schema documented. Tool returns 'str' (stdout) but does not specify: is there stderr? What about exit code for successful commands? What fields downstream tools should expect?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 29 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Destructive and arbitrary command execution with NO safety guardrails. No confirmation/dry-run mechanism, no rate limiting, no logging of executed commands, no permission checks. An LLM can execute 'rm -rf /' or exfiltrate data without warning.
Error handling returns RuntimeError but does not guide LLM recovery. A failed command returns 'Command failed with exit code X: <stderr>', the LLM cannot distinguish between retryable transient errors (network timeout) vs. permanent failures (command not found).
No input validation or sanitization. Tool accepts any string and passes it to shlex.split(). While shlex provides some injection protection, the tool lacks validation of whether commands are allowed (blacklist/whitelist of commands), timeouts for hanging commands, or resource limits.
No audit logging. Irreversible operations (deletes, data exfiltration) leave no trace of who called what command, when, or with what outcome. Required for compliance and incident response.