Loading ToolBench
Weather MCP Server providing access to US National Weather Service (NWS) data including alerts and forecasts
The server defines two weather tools with basic structure. Both tools have action-verb names (get-*) and descriptions, which is good foundational work. However, the descriptions are brief (41-46 chars), parameter descriptions are minimal (27-29 chars), and there is no output schema documentation. The tools accept reasonable constraints (latitude/longitude bounds, 2-letter state code), but lack comprehensive error guidance and recovery hints. The server does not document what fields the output contains or how agents should structure follow-up calls. Parameter descriptions meet the minimum bar but lack the depth recommended for LLM decision-making (typical A-grade param descriptions are 50-150 chars).
Get weather alerts for a state
Get weather forecast for a location
Tool descriptions are too brief (41-46 chars; baseline is 194 chars median, 34-392 range for A+ tools). Descriptions lack WHEN guidance: when should an agent call get-alerts vs get-forecast? What does each return? 'Get weather alerts for a state' alone does not explain prerequisites, output structure, or relationship to other tools.
Parameter descriptions are too short and lack format/constraint hints. 'Two-letter state code (e.g. CA, NY)' is good for state, but latitude/longitude descriptions ('Latitude of the location', 29 chars) do not mention units, acceptable range, or coordinate system (WGS84 assumed). Baseline param descriptions average 72 chars; these are ~27-29 chars.
No output schema is documented. Agents cannot know what fields to expect from the response. Code shows the tool returns formatted text in a content array, but there is no schema explaining the structure of alerts (event, areaDesc, severity, status, headline) or forecast periods (name, temperature, windSpeed, shortForecast, etc.). This violates the pattern requirement to 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Error handling returns generic text responses but does not categorize errors or guide recovery. When makeNWSRequest fails, the tool returns 'Failed to retrieve alerts data' (generic) or 'This location may not be supported by the NWS API' (good guidance). However, there is no systematic approach: no error codes, no retry hints, no actionable next steps. Code catches errors silently with console.error() rather than returning structured error details.
No pagination or result limits documented. get-alerts returns all active alerts for a state without limit. If a state has hundreds of alerts, the response could exceed token budgets. The code formatAlert() appends '---' between items and joins without a length cap. No mention of pagination in the description, and no limit on forecast periods (could be 14+ periods).