Loading ToolBench
Weather MCP Server that provides weather alerts and forecasts using the National Weather Service API
Two tools with basic but incomplete definitions. Both have clear verb-noun naming (get-alerts, get-forecast) and descriptions present. Input schemas are properly defined with Zod constraints (state length, latitude/longitude bounds). However, descriptions are minimal (10-15 chars baseline), lack context on WHEN to use each tool, and output schemas are not documented. No error recovery guidance. Tool composition is reasonable but lacks pagination/limits for potentially large result sets.
Get weather alerts for a state
Get weather forecast for a location
Descriptions are too brief (10-15 chars). 'Get weather alerts for a state' lacks context on when to use this vs other weather tools, what data is returned, and prerequisites (e.g., state code format). Baseline for A+ tools is 50-200 chars with WHEN/WHAT/RETURN guidance.
Output schemas are not documented. Callers cannot predict the structure of returned forecast periods, alert fields, or error responses. LLMs must infer structure from unstructured text responses, increasing hallucination risk.
No pagination or result limits documented. get-alerts could return hundreds of active alerts for a large state; get-forecast returns all forecast periods without a cap. Large responses blow context windows. Baseline: cap at 20-50 items and offer pagination.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Error responses are generic text ('Failed to retrieve alerts data'). No recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal. Should include actionable next steps.
Parameter descriptions lack format/constraint details. 'Two-letter state code (e.g. CA, NY)' uses example values instead of formal enum or pattern. LLMs may hallucinate invalid codes. Latitude/longitude descriptions omit that only US locations are supported (discovered only in error message).