Execute static commands (ls pwd cd cat echo grep git ...) and interactive programs (vi/vim/nano top/htop less tmux mc ...) in WSL
Single tool 'exec' with moderate schema definition but critical naming and description deficiencies. Tool name 'exec' is generic and fails verb_noun pattern. Description is 131 chars (within range) but lacks critical guidance on WHEN to use this dangerous tool and what its WRITE risk means. Parameters have types and descriptions, but parameter descriptions are minimal (14-37 chars) and lack format/constraint details. No output schema documented. Error handling is present but minimal, timeout errors are shown, but no guidance on recovery or parameter constraints. The tool exposes dangerous command execution with weak safeguards (forbidden-word filtering via substring match is bypassable). This is a WRITE-risk tool that should require explicit user confirmation patterns, defensive prompting, or guardrails, none present.
Executes a static command (ls pwd cat echo ps mkdir cp grep find git sed ...) and returns its result
Tool name 'exec' is generic and does not follow verb_noun pattern. 'exec' is ambiguous, does it execute commands, scripts, code, or something else? Should be 'execute_command', 'run_shell_command', or similar to signal the specific action.
Tool description (131 chars) omits critical context: WHEN to use this tool vs alternatives, what WRITE risk means in practice, that it executes with security restrictions, and that dangerous commands are blocked. Description states WHAT but not WHY or WHEN, forcing LLMs to guess.
Parameter 'input' description ('Command to execute in the process') is only 37 chars and lacks format/constraint details. Should document: accepted command types (ls, pwd, cat, etc., list explicitly), forbidden patterns (rm -rf, shutdown, reboot documented in code but NOT in schema), command format requirements, and examples. LLM cannot infer these from current text.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Parameter 'timeout' description (14 chars: 'Timeout before termination (seconds, optional)') lacks numeric constraints. No min/max documented. Should state: range (e.g. 1 - 3600 seconds), default behavior (60 sec per code), and consequences of very large/small values.
No output schema documented. Tool returns text with structure 'return code: X / STDOUT: ... / STDERR: ...' but this is not specified in any schema definition. LLMs cannot plan downstream parsing or extract structured fields reliably.
Tool is marked WRITE risk but lacks confirmation/dry-run pattern. Dangerous commands (rm -rf, shutdown, reboot) are rejected with a warning asking user to 'explicitly confirm', but no confirmation mechanism or dry-run mode is implemented. This is a command execution tool, irreversible actions need explicit gating.
Security: Forbidden-word filtering uses substring matching (any(cmd in command for cmd in config['forbidden_words'])), which is bypassable. Command 'rm -rfv /tmp' matches 'rm -rf', but 'rm-rf' or 'rm -rf' (extra spaces) does not. Should use word boundary or regex-based validation. Also, filtering in code but not documented in tool description, LLMs do not know what is forbidden.
Error handling is minimal. Timeout errors and generic exceptions are returned as text but lack actionable recovery guidance. When a command fails, LLM cannot distinguish retryable (e.g. transient network) from permanent (e.g. command not found) errors. Should categorize errors as per pattern:error-classification.