Loading ToolBench
Provides weather context and alerts for supported US cities via MCP protocol
Single tool 'get_alerts' has a verb-noun name (good), but description is only 31 chars, below the 50-char minimum for LLM clarity. Input schema is present with 'state' param, but param description is only 15 chars ('US state code'), violating the 20-char floor. No output schema documented. Tool returns unstructured string concatenation instead of structured JSON. No error handling guidance, bare 'No active alerts found' string offers no recovery path. Resource 'echo://{message}' is present but not an MCP tool and not evaluated here.
Get weather alerts for a US state
Tool description is 31 characters ('Get weather alerts for a US state'). Below 50-char minimum for LLM selection clarity. Lacks WHEN to use, prerequisites, or return type.
Parameter 'state' has description 'US state code' (15 chars). Below 20-char floor. Does not specify format (2-letter code?), valid range, or examples. LLM cannot infer constraints.
Output schema not documented. Tool returns raw string (alerts joined by '---'). LLMs cannot parse structure or extract fields for downstream calls. Should return JSON with typed fields: {alerts: [{event, area, severity, description, instruction}], count, timestamp}.
No error handling guidance. 'No active alerts found' is a bare string. Does not tell LLM whether to retry, ask user, or treat as success. Should return structured error: {status: 'no_alerts', message: '...', next_steps: []}.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
No input validation or constraint documentation. 'state' param accepts any string. Should validate against 50 US state codes (enum or regex pattern). Invalid input (e.g. 'XX') returns API error with no guidance.