Two tools with basic structure but significant gaps in schema completeness, parameter documentation, and output definition. Naming is clear (verb-noun pattern), but descriptions lack the detail required for effective LLM tool selection. No documented output schemas, no pagination support, no error recovery guidance. Tool implementations use proper async patterns and include basic error handling (try/except), but this is not surfaced to the LLM. The server returns plain strings instead of structured objects, forcing LLMs to parse unstructured text. Average per-tool score: 52 (get_alerts: 50, get_forecast: 54).
Get weather alerts for a US state. Args: state: Two-letter US state code (e.g. CA, NY)
Get weather forecast for a location. Args: latitude: Latitude of the location longitude: Longitude of the location
No documented output schema. Tools return plain strings instead of structured objects with typed fields. LLMs cannot reliably parse unstructured text responses or plan downstream tool chains.
Parameter 'state' in get_alerts lacks format constraints. Description says 'Two-letter US state code (e.g. CA, NY)' but uses example values instead of declaring an enum. LLMs may hallucinate invalid state codes like 'California' or 'CALIFORNIA'.
get_forecast accepts latitude/longitude as free floats with no bounds documentation. Valid ranges (-90 to 90 for lat, -180 to 180 for lon) are not stated. LLMs could pass invalid coordinates without guidance.
Tool descriptions are under 100 characters and lack 'when to use' context. get_alerts description (78 chars) and get_forecast description (81 chars) do not explain when to call each tool instead of alternatives, or what the response contains.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Error handling returns generic fallback strings ('Unable to fetch alerts or no alerts found') without recovery guidance. No indication whether the failure is transient (retry), user-caused (bad coords), or fatal.
get_forecast returns only the next 5 forecast periods hardcoded (periods[:5]), with no pagination parameters or limit explanation. Large forecasts are silently truncated; the LLM does not know results are capped.
get_alerts response combines multiple unrelated alert fields (Event, Area, Severity, Description, Instructions) into a single formatted string. No structure enables downstream tools to filter by severity, extract only descriptions, or act on specific alert types.