Weather data server providing alerts and forecasts from the National Weather Service API
Two tools with basic verb-noun naming (get_alerts, get_forecast) and READ_ONLY risk classification. Both have input schemas with typed parameters and descriptions. However, descriptions are minimal (under 50 chars each), lack context on when to use each tool, and do not explain output structure. No output schemas documented. Error handling returns plain strings with no recovery guidance. Parameters lack constraints (e.g., state code validation, lat/lon bounds). Tool composition is reasonable but descriptions fail to distinguish use cases or guide LLM selection.
Get weather data for a US state. Args: state: Two-letter US state code (e.g. CA, NY)
Get weather forecast for a location. Args: latitude: Latitude of the location longitude: Longitude of the location
Descriptions are under 50 characters and lack actionable context. 'Get weather data for a US state' does not explain when to use get_alerts vs get_forecast, what the output contains, or what state codes are valid.
No output schema documented. LLMs cannot plan downstream calls or extract structured data from string responses. Responses are plain text concatenations with no typed fields.
Parameters lack validation constraints. 'state' accepts any string; no enum of valid US state codes. Latitude/longitude have no bounds (±90/±180). LLMs will pass invalid values.
Error handling returns plain strings ('Unable to fetch alerts') with no recovery guidance. LLMs cannot determine if the error is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
No distinction between tools. Both fetch weather data; descriptions do not explain that get_alerts returns active warnings while get_forecast returns temperature/wind forecasts. LLM may select wrong tool.