Loading ToolBench
MCP server providing weather data connectors with tools for fetching weather alerts and forecasts from the National Weather Service API
Two tools with basic but incomplete schemas and minimal descriptions. Both tools have proper naming (verb_noun pattern with 'get_' prefix) and adequate input schemas with type definitions, but descriptions are too generic and lack context for when/why to use each tool. No output schema documentation visible. Error handling is minimal, tools return plain strings on failure with no recovery guidance. No tool annotations (readOnlyHint, etc.) despite both being read-only operations. Parameter descriptions are present but terse. The server represents a typical community MCP tool with functional definitions but missing production-grade polish.
Get weather alerts for a US state.
Get weather forecast for a location.
Descriptions lack context for tool selection. Both tools have ~20-char descriptions that do not explain when to use them, what they return, or whether they are stateful. 'Get weather alerts for a US state' does not tell an LLM whether to call this tool if the user says 'show me hazard warnings' or if it covers marine/flood alerts specifically.
No output schema documentation. The code shows tools return plain strings, but there is no formal schema describing the structure or fields of the response. LLMs cannot plan downstream operations or extract specific data (e.g., extracting alert severity) without knowing response structure.
No tool annotations despite read-only semantics. Both tools are safe to retry and have no side effects, but lack readOnlyHint annotation. This forces LLMs to treat them conservatively and may skip retries on transient failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 51 | - | v1 |
Error responses do not guide recovery. Tools return plain strings like 'Unable to fetch alerts or no alerts found' and 'No active alerts for this state' with no actionable next steps. If get_forecast fails due to invalid coordinates, the LLM has no guidance on how to correct the input.
Parameter 'state' in get_alerts accepts arbitrary strings but should be constrained to a known enum of US state codes. The description says 'Two-letter US state code (e.g. CA, NY)' but does not enforce it, allowing LLMs to hallucinate invalid codes like 'XX' or 'Cal'.
Parameter descriptions are minimal. 'Latitude of the location' and 'Longitude of the location' do not specify ranges (valid: -90 to 90 and -180 to 180), precision, or error handling for out-of-range values. An LLM may pass latitude=500 and get a cryptic API error with no self-correction hint.