A small native coding-agent harness with bounded tools and OpenAI Responses support
Critical gaps across all dimensions. Tool definitions are minimal: descriptions are extremely brief (5-30 chars, well below the 34-char p10 baseline), input schemas are either missing or incomplete, and parameter descriptions are absent. The 'shell' tool with DESTRUCTIVE risk has no input schema visible and no error handling guidance. 'apply_patch' and 'scheduled_task' lack documented output schemas. No tool demonstrates idempotency hints, permission gates, or recovery guidance. Overall, this reads as a proof-of-concept harness rather than production-grade tooling.
Apply a patch to a file
Echo a JSON value from an external provider
Read file contents
Read and process image files
Schedule and manage tasks
Execute shell commands
Fetch content from web URLs
Input schemas missing or not visible for 6 of 7 tools (apply_patch, shell, read_image, web_fetch, scheduled_task). Cannot verify parameter types, constraints, or descriptions.
Descriptions are extremely brief (5-30 chars, far below 34-char p10 baseline). 'Apply a patch to a file' (26 chars) and 'Execute shell commands' (22 chars) lack context on WHEN to use, WHAT happens, and prerequisites.
Destructive tools (shell, apply_patch, scheduled_task) lack error handling guidance, confirmation steps, or recovery paths. No dry-run support visible. Agents cannot distinguish retryable vs fatal errors or know what to do on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 33 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in source. Risk metadata is inferred from tool names, not declared. Agents cannot reliably determine safety profile.
Output schemas not documented for any tool. LLMs cannot plan downstream calls or extract required fields. No pagination, result limits, or chaining IDs visible.