Monorepo containing multiple packages for ACP (Agent Client Protocol) and MCP (Model Context Protocol) integration, including MCP server management, AI SDK providers, and agent capability probing
Critical gaps across all dimensions. Tool descriptions are minimal (10-30 chars), far below the 194-char baseline for production tools. Input schemas present but lack parameter descriptions, the rubric requires descriptions for EVERY parameter. No output schemas documented. No error handling guidance. Tools like 'bash' and 'write' are high-risk (IRREVERSIBLE, WRITE) but lack safety patterns (confirmation, dry-run, permission gates). Naming is acceptable (verb-noun style) but descriptions do not explain WHEN to use each tool or what it returns. No evidence of idempotency, pagination, or composition patterns.
Execute a shell command.
Edit a file by replacing text.
Find files matching a glob pattern.
Search file contents with a regex pattern.
Read file contents.
Search the web.
Write content to a file.
All tool descriptions are under 20 characters (e.g., 'Read file contents.', 'Write content to a file.'). Descriptions lack WHEN to use, prerequisites, and return value guidance.
Input parameters lack descriptions. Schema shows 'file_path' and 'content' but no descriptions explaining what values are valid, expected format, or constraints. Rubric requires descriptions for EVERY parameter.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. Rubric baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
High-risk tools (bash=IRREVERSIBLE, write/edit=WRITE) lack safety patterns. No dry-run, confirmation step, or permission gate. Agents can execute arbitrary shell commands or overwrite files without safeguards.
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery hints (e.g., 'file not found, check path' or 'regex invalid, verify pattern syntax').