Multi-provider AI coding agent CLI — 45 tools, 40 commands, autonomous mode, verification agent, multi-agent coordination
Critical failures across naming, descriptions, and schemas. 20 tools with minimal documentation: most descriptions are 1-2 sentences (avg ~40 chars, well below 194-char baseline). Duplicate tool names (Read/FileRead, Edit/FileEdit, Write/FileWrite, WebFetch/Web_Fetch, WebSearch/Web_Search, NotebookEdit/Notebook_Edit) indicate poor composition and confuse LLM selection. Parameter schemas present but descriptions are generic ('Path to the file to read'). No output schemas documented. No error handling guidance. TodoWrite has EMPTY input schema. Tools lack verb-noun clarity (e.g., 'Read' vs 'FileRead', which should the LLM choose?). No pagination, no result limits, no chaining IDs in responses. Security: Bash/Shell/PowerShell execute arbitrary commands with no validation, sanitization, or permission gates.
Execute bash shell commands
Compare two files and show differences
Edit file contents
Edit file contents
Read file contents from the filesystem
Write content to a file
Find files matching a glob pattern
Search for patterns in files
Duplicate tool names with no clear distinction. Read/FileRead, Edit/FileEdit, Write/FileWrite, WebFetch/Web_Fetch, WebSearch/Web_Search, NotebookEdit/Notebook_Edit. LLMs cannot disambiguate and will pick arbitrarily, causing unpredictable behavior.
TodoWrite has empty input schema ({}). No parameters defined. LLM cannot understand what data to provide or how to invoke the tool.
Bash, Shell, PowerShell execute arbitrary commands with no input validation, sanitization, or permission gates. Vulnerable to command injection and privilege escalation. No dry-run or confirmation step for destructive operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | 2026-07-28+ | v2 |
List directory contents
Edit notebook files
Edit notebook files
Execute PowerShell commands
Read file contents from the filesystem
Execute shell commands
Write to a todo list
Fetch content from a web URL
Search the web for information
Fetch content from a web URL
Search the web for information
Write content to a file
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Responses are unstructured and waste tokens.
Descriptions are generic and under 50 characters (e.g., 'Read file contents from the filesystem'). No guidance on when to use each tool, prerequisites, or error recovery. Baseline is 194 chars.