Weather information server providing alerts and forecasts via National Weather Service API
Two tools with basic verb-noun naming (get_alerts, get_forecast) but significant quality gaps. Both have minimal descriptions (under 100 chars) lacking context on when to use them or what they return. Input schemas present but incomplete: get_alerts lacks validation rules for state codes; get_forecast has no bounds on latitude/longitude. No output schemas documented. get_forecast implementation is incomplete (commented-out logic, hardcoded ['nice day'] response). No error handling guidance, no pagination support, no structured output documentation. Tools are READ_ONLY but lack explicit idempotent hints. Overall definition quality is below community baseline (median ~45-55).
Get weather alerts for a US state.
Get weather forecast for a location.
No output schema documented for either tool. LLMs cannot plan downstream calls or extract structured data. get_alerts returns formatted string; get_forecast returns hardcoded ['nice day']. Both lack field documentation.
Descriptions are too brief (under 100 chars) and lack context. 'Get weather alerts for a US state' does not explain when to use this vs other weather tools, what data is returned, or prerequisites. Baseline for A+ tools is 194 chars average.
Input parameters lack validation constraints. get_alerts 'state' parameter has no enum of valid US state codes (CA, NY, TX, etc.). get_forecast latitude/longitude have no bounds (±90, ±180). LLMs will pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
get_forecast implementation is incomplete and non-functional. Logic for fetching actual forecast data is commented out; function returns hardcoded ['nice day']. This tool cannot be used in production.
No error handling guidance. make_nws_request() silently returns None on any exception. Tools return generic strings like 'Unable to fetch alerts' without actionable recovery steps (e.g., 'Try a different state code' or 'Service temporarily unavailable, retry in 30s').