Evaluates software engineering progress based on token-time metrics. Generates duration and cost estimations for tasks based on token consumption, model throughput, and pricing data.
Single tool 'calculate_plan' has a basic description but critical gaps in schema completeness and parameter documentation. The description is 234 chars (within baseline 194 avg), but the input schema is minimal: only 'plan_path' string with a short description. No output schema is documented, the tool returns a Report object with a single 'report' string field, but this is not formally declared in the tool definition. No error handling guidance, no parameter constraints (e.g., file path validation, format requirements), and no indication of what the tool actually computes or when to use it. The description mentions reading from 'ttp://spec/plan' but does not explain what a valid plan.yaml structure looks like or what metrics are returned. No enum constraints, no range limits, no validation rules. The tool name 'calculate_plan' is generic and does not clearly convey that it estimates token costs and execution time.
Evaluates software engineering progress based on token-time metrics. You MUST read the specification from "ttp://spec/plan" before generating the plan.yaml. Pass the path of your generated plan.yaml to this tool for precise duration and cost estimation.
No output schema documented. Tool returns a Report object with a 'report' string field, but the MCP tool definition does not declare the output structure. LLMs cannot plan downstream operations or extract structured data.
Input parameter 'plan_path' lacks validation constraints. No description of expected file format, required YAML structure, or what fields plan.yaml must contain. LLMs cannot validate input before calling the tool.
No error handling guidance. Tool description does not explain what happens if plan.yaml is malformed, missing, or contains invalid model IDs. No recovery hints for LLMs.
Tool name 'calculate_plan' is generic and does not clearly convey the action. Better names: 'estimate_token_cost', 'analyze_plan_metrics', or 'compute_plan_duration'. Current name does not start with a clear action verb.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Description does not explain WHEN to use this tool or what prerequisites exist. It mentions reading from 'ttp://spec/plan' but does not clarify whether the LLM must call a resource first or if the user provides plan.yaml externally.