Server has 2 unique tools (get_alerts, get_forecast) with basic descriptions and parameter schemas, but significant quality gaps. Descriptions are present but minimal (34-50 chars, below the 50-200 char LLM-optimized baseline). Parameter descriptions exist but lack format/constraint details. No output schemas documented. Error handling returns plain strings without recovery guidance. Tool definitions are visible in source code but lack depth expected for production use.
Retrieves the weather alerts for a US state.
Get weather alerts for a US state.
Retrieves the weather forecast for a location.
Get weather forecast for a location.
Descriptions too brief (34-50 chars vs 50-200 char baseline). Missing WHEN to use, prerequisites, and return value details. LLMs cannot reliably select between tools with minimal descriptions.
No output schema documented. Tools return plain strings; LLMs cannot parse structured fields or plan downstream calls. Should return JSON objects with typed fields (event, severity, temperature, etc.).
Parameter descriptions lack format/constraint details. 'state' should specify '2-letter uppercase US state code (e.g., CA, NY)' with enum or pattern. 'latitude/longitude' should specify valid ranges (-90 to 90, -180 to 180).
Error handling returns generic strings ('Unable to fetch alerts') without recovery guidance. Should tell LLM what to do next: retry, check input, or escalate.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Duplicate tool definitions in tool list (get_alerts and get_forecast each appear twice with identical schemas). Indicates either data collection error or actual duplicate registration in code.