Loading ToolBench
A weather service server using the Model Context Protocol (MCP) that provides access to National Weather Service (NWS) API data
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Both tools have action-verb names (get-alerts, get-forecast) and descriptions present, but descriptions are minimal (under 50 chars). Input schemas use Zod with type constraints (latitude/longitude bounds, state length), but lack detailed parameter descriptions beyond the constraint hints. Output schemas are not documented, responses are free-text strings wrapped in content arrays, making it hard for LLMs to extract structured data for chaining. Error handling returns text messages but lacks actionable recovery guidance. No tool annotations (readOnlyHint, idempotentHint) despite both being read-only operations.
Get weather alerts for a state
Get weather forecast for a location
Output schemas not documented. Both tools return free-text strings in content arrays. LLMs cannot extract structured forecast periods, alert details, or chaining IDs (e.g., state code for follow-up calls). Responses should document field structure so agents can parse and chain results.
Tool descriptions are too brief (under 50 chars). 'Get weather alerts for a state' and 'Get weather forecast for a location' lack context on when to use each tool, what data is returned, or prerequisites. Descriptions should be 50-200 chars and answer: what does it do, when to call it, what does it return?
Parameter descriptions are minimal. 'state' param has only 'Two-letter state code (e.g. CA, NY)', no guidance on case sensitivity (code shows toUpperCase() but description doesn't say it's case-insensitive). Latitude/longitude descriptions lack context on coordinate system or supported regions (US-only per code comment, but not in description).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
No tool annotations. Both tools are read-only and idempotent, but lack readOnlyHint and idempotentHint annotations. This prevents agents from reasoning about retry safety and side effects.
Error messages lack recovery guidance. When NWS API fails or coordinates are unsupported, responses return plain text ('Failed to retrieve alerts data', 'This location may not be supported by the NWS API'). Should guide LLM: 'Try a different state code' or 'Only US coordinates supported, provide latitude/longitude within ±90/±180 bounds.'