Loading ToolBench
A Model Context Protocol (MCP) server that provides real-time weather data and alerts using the US National Weather Service API
This server defines 2 tools with explicit schemas and descriptions, but has significant gaps in parameter documentation, error handling guidance, and output schema documentation. Tool names follow verb-noun conventions (get-alerts, get-forecast), which is positive. However, parameter descriptions are minimal, error responses do not guide recovery, and output structures are not formally documented. The implementation uses Zod for validation but does not expose those schemas to the LLM, limiting the agent's ability to understand valid inputs and outputs.
Get weather alerts for a US state
Get weather forecast for a location in the US
Parameter descriptions are too minimal. 'state' parameter is described as 'Two-letter US state code (e.g. CA, NY)', includes example values that LLMs may reuse literally, violating pattern:constrained-input. Should use enum constraint instead.
Output schemas are not documented. Callers cannot determine what fields to expect from either tool, no documentation of the structure returned by get-alerts (properties like event, areaDesc, severity, etc.) or get-forecast (name, temperature, windSpeed, etc.). This forces agents to guess field names and risks incorrect downstream processing.
Error handling does not guide recovery. When API calls fail, the tool returns generic text like 'Failed to retrieve alerts data' or 'This location may not be supported by the NWS API' without suggesting next steps (e.g., 'Try a different state code' or 'Only US locations are supported'). Per pattern:recovery-guide, errors must tell the LLM what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Parameter 'state' in get-alerts lacks enum constraint. Accepting free-form strings invites hallucinated values (e.g., 'California' instead of 'CA'). Should be constrained to a fixed set of 50 US state codes via enum in the schema.
Latitude/longitude parameters lack range documentation in descriptions. Schema defines min/max constraints (-90 to 90 for latitude, -180 to 180 for longitude) but the parameter descriptions do not state these bounds explicitly. Per pattern:tool-description, constraints must be documented in plain language.
No pagination support. The get-alerts tool caps results at 20 items via slice(0, 20) but returns no indication of truncation, total count, or how to fetch additional alerts. Large alert datasets will be silently truncated without the agent knowing.