An AI agent framework with support for multiple LLM providers, tool execution, skill management, and various interfaces (CLI, HTTP, ACP protocol)
Coddy Agent exposes 30 tools with minimal schema documentation and sparse descriptions. Most tools lack detailed parameter descriptions, input validation constraints, and output schema documentation. Tool names are inconsistent (verb_noun vs bare nouns). Critical security issues: run_command and ssh_run_command expose shell execution without dry-run or confirmation patterns. No evidence of error handling guidance, recovery paths, or actionable error messages. Descriptions average ~30 chars, well below the 194-char baseline for production tools. Only 5 tools have descriptions exceeding 20 chars. Parameter schemas are inferred from file references rather than explicitly visible in source code, preventing verification of type definitions and constraints.
Apply a unified diff patch to a file
Read Coddy documentation pages
Search Coddy documentation
Create a scheduled job
Delete a scheduled job
Update a scheduled job
Update a todo item status
Get configuration values
Irreversible operations (run_command, ssh_run_command, rm, rmdir, spawn_agent, load_skill) lack dry-run, confirmation, or recovery patterns. Agents can execute destructive commands without safeguards.
Parameter descriptions are minimal (15-30 chars) and lack actionable constraints. No enum values, format specs, or validation rules visible. E.g., 'command' param has no guidance on shell syntax, escaping, or allowed operations.
Output schemas are not documented. Tools return unspecified structures; LLMs cannot plan downstream calls or extract required fields. No evidence of pagination, result limits, or field documentation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Revert configuration to defaults
Set configuration values
Edit file contents with line-based modifications
Find files matching a glob pattern
Search for text patterns in files
Make HTTP requests
Load and execute a skill
Create a directory
Move or rename a file or directory
Print directory tree structure
Ask the user a question with options
Read file contents
Remove a file or directory
Remove an empty directory
Execute shell commands
Describe or rename a session
Spawn a child agent
Execute commands over SSH
Create or update a file
Fetch and parse web content
Search the web
Write content to a file
No error handling guidance. Tools do not document retryable vs fatal errors, recovery steps, or actionable error messages. Agents cannot self-correct on failures.
Tool names lack consistency. Scheduler tools use 'coddy_scheduler_job_*' prefix while file tools use bare verbs. No clear verb_noun pattern. Names like 'question' and 'session_describe' are ambiguous.