A weather server providing access to US National Weather Service alerts and forecasts
Two tools with basic verb-noun naming (get_alerts, get_forecast). Both have descriptions and input schemas with typed parameters. However, descriptions are minimal (under 100 chars each), lack context on when to use each tool, and do not explain output structure. Parameters have type definitions but lack constraints (e.g., state format, latitude/longitude bounds). No output schema documented. Error handling returns plain strings without recovery guidance. No tool annotations (readOnlyHint, etc.). Schemas are present but sparse.
Get weather alerts for a US state. Args: state (str): The two-letter state abbreviation (e.g., "CA" for California).
Get the weather forecast for a specific latitude and longitude. Args: latitude (float): Latitude of the location. longitude (float): Longitude of the location.
Output schema not documented. Tools return plain strings; LLMs cannot parse structured data or chain downstream calls. Forecast and alert responses should be JSON objects with typed fields.
Parameter constraints missing. 'state' accepts any string; should validate 2-letter US state codes (enum or regex). Latitude/longitude lack bounds (±90, ±180). LLMs will pass invalid values.
Descriptions too brief (45 chars each). Do not explain when to use each tool, what data is returned, or dependencies. Baseline for A+ tools is 50-200 chars with context.
Error handling returns generic strings ('Unable to fetch alerts') without recovery guidance. Should tell LLM: is this retryable? Should it try a different location? What went wrong?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
No tool annotations. Tools are read-only but lack readOnlyHint. Helps LLMs understand safety and retry semantics.