A weather server that provides weather alerts and forecasts using the National Weather Service API
Two tools with basic verb-noun naming (get_alerts, get_forecast). Both have short descriptions (34-45 chars, below the 194-char baseline). Input schemas present with proper JSON Schema structure and typed parameters. However, parameter descriptions are minimal (8-40 chars vs 72-char baseline), and output schemas are not documented. No error handling guidance, no pagination support despite potential for large result sets, and no tool annotations. Naming is acceptable but descriptions and parameter documentation fall significantly short of production baselines.
Get weather alerts for a US state.
Get weather forecast for a location.
Parameter descriptions are too brief (8-40 chars vs 72-char baseline). 'Two-letter US state code (e.g. CA, NY)' for state param is acceptable, but latitude/longitude descriptions lack context on coordinate system, precision, or valid ranges.
Tool descriptions are too short (34-45 chars vs 194-char baseline). 'Get weather alerts for a US state' lacks WHEN to use it, what it returns, and any prerequisites. LLMs cannot determine selection criteria from such minimal text.
Output schemas are not documented. LLMs cannot plan downstream calls or extract relevant data without knowing what fields to expect from alerts or forecast results.
No error handling guidance. If a state code is invalid or coordinates are out of bounds, the tools provide no recovery hints. LLMs cannot self-correct without actionable error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
No pagination or result limits documented. Weather alerts or forecasts could return large datasets; without limit/offset parameters and a documented cap, results may blow the context window.