Loading ToolBench
MCP server for providing weather data from National Weather Service API
Two tools with clear action verbs and basic schemas. Both have descriptions (10-20 chars baseline met), but parameter descriptions are minimal or missing. get_forecast has proper numeric constraints (min/max), but get_alerts lacks validation guidance. No output schemas documented. Error handling returns text but lacks recovery guidance. Zod validation present in code but not reflected in schema descriptions. Missing parameter descriptions for 'state' in get_alerts (only says 'Two-letter US state code', no guidance on what happens if invalid). No tool annotations (readOnlyHint, idempotentHint). Resources feature present but underutilized.
Get weather alerts for a US state
Get weather forecast for a location
Missing output schema documentation. Tools return formatted text or JSON but no schema is documented for LLM to plan downstream calls or extract structured data.
Parameter descriptions lack actionable constraints. 'state' param in get_alerts only says 'Two-letter US state code', no guidance on valid format, error handling, or examples of valid values (CA, NY, TX).
Error responses return plain text without recovery guidance. 'Error fetching alerts: ...' tells LLM nothing about whether to retry, ask user, or try alternative tool.
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent but this is not declared in schema, forcing LLM to infer safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Resource templates present but underutilized. weather://{state}/alerts resource exists but tool-based get_alerts duplicates functionality without clear separation of concerns.