Loading ToolBench
Weather MCP Server that provides weather alerts for US states
Single tool 'get-alerts' has a clear verb-noun name and basic schema with type constraints (2-letter state code). However, the description is minimal (11 words), lacks context on when to use it, and provides no guidance on output structure or error recovery. Parameter description is adequate but generic. No output schema documented. No error handling guidance. Tool lacks idempotency hints and does not explain what happens on API failure beyond 'Failed to retrieve alerts data'. Naming is good; everything else is below production baseline.
Get weather alerts for a state
Tool description is too brief (11 words). Does not explain WHEN to use this tool, what the output contains, or how to handle failures. LLMs cannot determine selection criteria from 'Get weather alerts for a state' alone.
No output schema documented. Callers cannot know what fields to expect (event, areaDesc, severity, status, headline). LLMs must infer structure from unstructured text response, risking parsing errors and wasted tokens.
Error handling returns generic 'Failed to retrieve alerts data' with no recovery guidance. Does not tell LLM whether to retry, ask user, or try a different state code. Violates recovery-guide pattern.
No idempotency or side-effect declaration. Tool is read-only (correct), but description does not state this. LLMs should know this is safe to retry without side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
State parameter accepts only 2-letter codes but description does not explain valid format or provide examples of valid states (CA, NY, TX). LLMs may pass invalid codes like 'California' or 'US-CA'.