Two tools with basic verb-noun naming (get_alerts, get_forecast). Both have descriptions and input schemas with typed parameters. However, descriptions are minimal (under 100 chars) and lack context on when to use each tool or what to do with results. No output schema documentation. Parameters lack constraints (e.g., state code validation, lat/lon bounds). Error handling returns plain strings with no recovery guidance. No tool annotations. Schemas are present but sparse, missing examples, constraints, and output structure documentation.
Get weather alerts for a US state. Args: state: Two-letter US state code (e.g. CA, NY)
Get weather forecast for a location. Args: latitude: Latitude of the location longitude: Longitude of the location
No output schema documentation. Tools return plain strings; LLMs cannot parse structured results or chain to downstream tools. Baseline: 100% of A+ tools document return types.
Parameter descriptions lack constraints. 'state' accepts any string; should specify '2-letter US state code (uppercase, e.g. CA, NY)' with enum or pattern. Latitude/longitude lack bounds (±90, ±180). Invites invalid LLM input.
Error handling returns generic strings ('Unable to fetch alerts', 'No alerts found') with no recovery guidance. Per pattern:recovery-guide, errors should tell LLM what to do next (e.g., 'No alerts for CA. Try a different state or check NWS website.').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Descriptions are too brief (45 chars avg). Baseline: 194 chars. Missing context on when to use each tool, what data is returned, and dependencies. 'Get weather alerts for a US state' does not explain when to call this vs get_forecast.
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent, declaring this helps agents reason about retry safety and side effects.