Zero-dependency production gate, resource budget, and capacity model for AI-assisted code
ShipProof exposes 3 tools with complete input schemas and descriptions, but quality is uneven. All tools are READ_ONLY and well-scoped (scan, budget, capacity). Descriptions are present (60-90 chars) but lack actionable context for LLM selection. Parameters have types and descriptions, meeting baseline requirements. However, output schemas are not documented, error handling guidance is absent, and parameter constraints (e.g., severity enums, confidence levels) lack formal validation hints. The tools follow verb_noun naming (scan, budget, capacity) but descriptions do not explain WHEN to use each tool or what distinguishes them. No tool annotations (readOnlyHint, idempotentHint) are present despite all being read-only operations.
Check resource budget and capacity constraints for AI-assisted code
Generate capacity model and load test stages for production readiness
Scan a repository for security findings and production readiness issues
Output schemas not documented. LLMs cannot plan downstream tool calls or extract structured data without knowing what fields to expect.
Descriptions lack WHEN/WHY context. 'Scan a repository for security findings' does not explain when to call scan vs budget vs capacity, or what distinguishes them for LLM selection.
Parameter constraints not formalized. 'fail_on' accepts 'high|medium|low' but lacks enum declaration; 'min_confidence' similarly lacks enum. LLMs may hallucinate invalid values.
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal, or how to recover from failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
Tool annotations missing. All tools are read-only but lack readOnlyHint annotation, and scan is idempotent but lacks idempotentHint. Agents cannot infer safety properties.