Loading ToolBench
MCP server that provides weather alerts and forecasts using the National Weather Service API
Two tools with basic descriptions and schemas, but significant gaps in parameter documentation and output structure. get_alerts and get_forecast both have descriptions (10-20 words each) and input schemas with types, but lack detailed parameter descriptions beyond the bare minimum. No output schema documentation. Error handling returns plain strings without guidance for LLM recovery. Parameter descriptions are minimal: 'Two-letter US state code' and coordinate descriptions lack format constraints, ranges, or validation hints. No tool annotations (readOnlyHint, idempotentHint). Naming is clear (verb_noun pattern), but composition and error recovery are weak.
Get weather alerts for a US state.
Get weather forecast for a location.
Parameter descriptions lack format constraints and validation hints. 'state' parameter has no enum constraint (should be 2-letter US state codes); latitude/longitude lack range bounds (±90, ±180) or format specification.
No output schema documented. Tools return plain strings; LLMs cannot parse structured results or extract IDs for chaining. Should return JSON objects with typed fields (e.g., {alerts: [{event, area, severity, description}], count: N}).
Error handling returns generic strings ('Unable to fetch alerts', 'No active alerts') without recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal. Should return structured errors with actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Tool descriptions are too brief (10-20 words). Lack context on WHEN to use each tool, what data they return, and prerequisites. Should be 50-200 characters with use-case guidance.
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared in the schema. LLMs cannot infer safety properties without explicit hints.