Loading ToolBench
MCP server that provides weather alerts for US states using the National Weather Service API
Single tool 'get_alerts' has a verb-noun name (good), but the description is only 34 characters, below the 50-char minimum for LLM clarity. Input schema is present with proper type and description for the 'state' parameter. However, output is unstructured plain text (string), not a documented schema. No error handling guidance (e.g., 'If state is invalid, try a two-letter code like CA, NY'). The tool lacks pagination, result limits, and structured response fields. Resource 'echo://{message}' is present but not a tool and not evaluated here.
get Weather alerts for a US state
Tool description is only 34 characters ('get Weather alerts for a US state'). Below 50-char minimum for LLM clarity. Lacks WHEN to use, prerequisites, or recovery guidance.
Output is unstructured plain text (string). No documented schema. LLM cannot extract structured fields (event, severity, area) for downstream processing or chaining.
No pagination or result limit. If a state has 100+ active alerts, the response will be massive and waste tokens. No 'limit' or 'offset' parameter.
Error handling is generic ('Error: Could not retrieve alerts'). Does not guide LLM on recovery: is the state code invalid? Is the API down? Should it retry?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
No input validation. Bare-except clause in make_nws_request() swallows all errors. Invalid state codes (e.g., 'XX') will silently return None without actionable feedback to the LLM.