A FastAPI-based MCP server that provides weather alerts and forecast tools via the National Weather Service API
Two tools with basic verb-noun naming (get_alerts, get_forecast) and minimal descriptions. Both tools have input schemas with types and descriptions, but output schemas are undocumented (return plain strings). Descriptions are 50-60 chars, meeting minimum length but lacking actionable context for LLM selection. No error handling guidance, no parameter constraints (e.g., latitude/longitude bounds), no documentation of what the string output contains. Tools are READ_ONLY but lack explicit idempotent hints. No pagination, no structured output, no recovery guidance.
Get weather alerts for a US state.
Get weather forecast for a location.
Output schemas undocumented. Both tools return plain strings with no structure. LLMs cannot parse results reliably or chain to downstream tools. Should return structured JSON with typed fields (alerts array with event/severity/description, forecast array with temperature/wind/description).
Descriptions lack actionable context. 'Get weather alerts for a US state' does not explain when to use this vs get_forecast, what format alerts are returned in, or what to do if no alerts exist. Should include: 'Returns active weather alerts (severe storms, floods, etc.) for a US state. Use this to check for urgent weather warnings. Returns empty string if no alerts active.'
No parameter constraints or validation hints. Latitude/longitude parameters lack bounds (±90, ±180). State parameter lacks enum or pattern. LLMs may pass invalid values (lat=999, state='XX'). Should document: 'latitude: float between -90 and 90', 'state: two-letter US state code (e.g., CA, NY, TX)'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
No error handling guidance. If NWS API is down, returns 'Unable to fetch alerts' with no recovery path. Should guide LLM: 'NWS API unavailable. Retry in 30 seconds or ask user for cached forecast.' Missing error classification (retryable vs fatal).
No tool annotations. Both tools are read-only and idempotent but lack explicit idempotentHint or readOnlyHint in schema. Agents cannot infer safety properties and may avoid retrying on transient failures.