Weather information server providing alerts and forecasts via National Weather Service API
Two tools with basic verb-noun naming (get_alerts, get_forecast). Both have short descriptions (40-50 chars) and documented input schemas with type info. However, descriptions lack context on WHEN to use each tool, what they return, and any prerequisites. No output schema documentation. Parameters have types and brief descriptions but lack constraints (e.g., state code format, lat/lon ranges). Error handling returns plain strings with no recovery guidance. No pagination, no structured output, no tool annotations.
Get weather alerts for a US state.
Get weather forecast for a location.
Descriptions are too brief (40-50 chars) and lack context. 'Get weather alerts for a US state' does not explain when to use this vs get_forecast, what format the output is, or what happens if the state code is invalid.
No output schema documented. Tools return plain strings; LLMs cannot parse structured data or chain results to other tools. Should return JSON with fields like alerts[], forecast_periods[], timestamps, etc.
Input parameters lack constraints. 'state' should document format (2-letter uppercase, e.g. CA, NY, or list valid codes). Latitude/longitude should specify valid ranges (-90 to 90, -180 to 180) and precision.
Error handling returns generic strings ('Unable to fetch alerts', 'No alerts found') with no recovery guidance. Should tell LLM: is this retryable? Should it try a different state? Is the API down?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
get_forecast returns only 5 periods (hardcoded limit) with no pagination or total count. If user needs a 10-day forecast, tool silently truncates. Should document limit in description and offer pagination.