A Model Context Protocol (MCP) server providing weather alerts and forecasts using the National Weather Service API
PyWeatherMCP has basic tool definitions with descriptions and input schemas, but falls short of production quality. All 6 tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization) and visible input schemas with type declarations. However, parameter descriptions are minimal or absent, most params lack guidance on format, constraints, or valid ranges. Output schemas are not documented; tools return free-text strings rather than structured JSON. Error handling is generic ('Unable to fetch alerts') with no recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk profiles. The server uses file-based memory (weather_memory.json) for state, which is fragile and not production-grade. Tool composition is reasonable (6 focused tools), but descriptions are too terse to guide LLM selection effectively.
Clear all search history (keeps favorites)
Get weather alerts for a US state.
Get all saved favorite locations
Get weather forecast for a location.
Get recent search history.
Save a favorite location for quick access.
Tool descriptions are too short (10-50 chars vs. 50-200 char baseline). LLMs cannot determine when to select a tool or distinguish it from similar tools. E.g., 'Get weather alerts for a US state' lacks context on when to use this vs. get_forecast.
Parameter descriptions are missing or trivial. 'state' param in get_alerts has description 'Two-letter US state code (e.g. CA, NY)' but includes example values (CA, NY) which LLMs may reuse literally. No enum constraint declared. Similar issues in all tools.
Output schemas not documented. All tools return free-text strings (e.g., 'Unable to fetch alerts', formatted forecast text). LLMs cannot parse structured data or chain tool calls. No documented return type or field structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Error handling is generic and non-actionable. 'Unable to fetch alerts' and 'Unable to fetch forecast' give no recovery guidance. No error classification (retryable vs. user-fixable vs. fatal). LLM cannot determine next step.
No tool annotations despite clear risk profiles. save_favorite and clear_history are WRITE/DESTRUCTIVE but lack readOnlyHint/destructiveHint annotations. Agents cannot distinguish safe reads from dangerous mutations.