Tiny stdlib-only honest-evaluation substrate for the cap-evolve skills library with Harbor integration, telemetry, and dashboard support
cap-evolve exhibits severe definition quality issues. Tool definitions are inferred from fixture files rather than explicitly registered in the codebase. Only 1 unique tool exists (calculate); others are duplicates across test fixtures. Descriptions are present but generic (10-50 chars). Input schemas lack proper type definitions for complex parameters (flights, passengers, payment_methods are arrays with no item schema). No output schemas documented. No error handling guidance. No parameter constraints or enums. The project appears to be a benchmarking/evaluation framework rather than a production MCP server.
Book a reservation.
Book a reservation.
Calculate the result of a mathematical expression.
Calculate the result of a mathematical expression.
Cancel the whole reservation.
Cancel the whole reservation.
Update the flight information of a reservation.
Tool definitions inferred from test fixtures, not explicitly registered in MCP server code. No evidence of actual MCP tool registration or server implementation.
Complex array parameters (flights, passengers, payment_methods) lack item schemas. No definition of what fields each array element must contain.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields.
Descriptions are generic and under 50 characters. 'Book a reservation' and 'Cancel the whole reservation' lack context for when/why to use them.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal.