Loading ToolBench
MCP server providing weather forecast and alert information via the National Weather Service API
The server has two tools with clear, verb-based names (get-alerts, get-forecast) and documented Zod schemas. Descriptions are concise and appropriate (10 - 40 chars range). Input parameters have types and constraints (latitude/longitude bounds, state length). However, the tools lack output schema documentation, the response structure is only visible in the code, not formally declared in the tool definitions. Error handling is minimal: the service returns fallback text strings but does not guide LLMs on recovery paths. No parameter descriptions explain WHY state must be 2 letters or WHAT latitude/longitude coordinates represent beyond the technical constraint. The tools are read-only weather queries, so they are low-risk, but descriptions could be more prompt-optimized (e.g., 'Get active weather alerts for a US state by two-letter code, e.g. CA, NY, TX').
Get weather alerts for a state
Get weather forecast for a location
Output schema not formally documented. Tool handlers return a fixed shape {content: [{type: 'text', text: string}]}, but there is no tool description or schema annotation declaring this structure to the LLM. LLMs must infer from response text format that the output is a text block, not a structured object.
Parameter descriptions lack context and use case guidance. 'Two-letter state code (e.g. CA, NY)' tells the LLM the format but not WHY (e.g., 'US state abbreviations for NWS alert lookup'). Similarly, 'Latitude of the location' lacks mention that only US coordinates are supported, forcing LLMs to discover this via error.
Error responses do not guide recovery. When coordinates are out of bounds or unsupported, the service returns 'This location may not be supported by the NWS API (only US locations are supported)' as plain text. No suggestion for next steps or alternative tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
No output pagination or result limiting. The forecast tool could return dozens of periods; alerts could return many features. No limit is enforced or documented. Large responses risk context window exhaustion.