Loading ToolBench
An MCP server that provides weather alerts and forecasts using the National Weather Service (NWS) API
Two tools with explicit Zod schemas and descriptions, but several critical gaps. Both tools have descriptions (15-35 chars) and input parameters with type constraints. However, descriptions are below the 50-200 char production baseline, parameter descriptions lack implementation details and usage guidance, and output schemas are entirely undocumented. Error responses return generic text rather than actionable recovery guidance. No tool annotations, no pagination support despite potentially large datasets (alerts can span multiple states, forecast periods could exceed reasonable context windows).
Get weather alerts for a state
Get weather forecast for a location
Descriptions are extremely brief (15-35 chars) and lack LLM guidance. 'Get weather alerts for a state' does not explain WHEN to use this tool, what data structure is returned, or what to expect when no alerts exist. Production baseline is 50-200 chars with clear WHAT/WHEN/RETURN structure.
Parameter descriptions are absent or minimal. The 'state' parameter has a description 'Two-letter state code (e.g. CA, NY)' but includes an example value in the description, which LLMs tend to reuse literally. The 'latitude' and 'longitude' descriptions state range constraints but do not explain when to use relative coordinates vs. address lookup, or what happens if coordinates are outside US bounds.
Output schemas are completely undocumented. The code returns generic {content: [{type: 'text', text: '...'}]} responses, but the LLM has no declared schema describing the structure, field types, or what to expect (e.g., does get-alerts return individual alert objects or concatenated text? What fields are in each alert?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
Error handling is generic and non-actionable. Responses like 'Failed to retrieve alerts data' do not tell the LLM what to do next: is this retryable? Should the user provide a different state code? The error does not include the constraint violated or suggest recovery steps (e.g., 'Invalid state code. Must be 2 uppercase letters.').
No pagination or result limits. The get-alerts tool can return many features for a single state (potentially 50-500+ active alerts). Without pagination or a stated limit, large responses blow the context window. The tool description does not warn about potential large result sets or offer limit/offset parameters.
Tool names are verb_noun (good), but 'get-alerts' is ambiguous: get alerts FOR A STATE, or get ALL alerts, or get alerts FOR A LOCATION? The description clarifies, but the name alone does not. By the naming baseline, 90% of A+ tools have unambiguous names that require no description parsing.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are READ_ONLY and idempotent, but the code does not declare this via annotations, forcing the LLM to infer safety from descriptions alone.