Loading ToolBench
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output structure. Both tools have verb-noun naming (get-alerts, get-forecast) which is correct. Descriptions exist but are minimal (18-28 chars, below the 50-200 char LLM-optimized baseline). Input schemas use Zod with type constraints (latitude/longitude bounds, state length), but lack detailed parameter descriptions beyond the constraint itself. No output schema documentation. Error handling returns plain text without recovery guidance. No tool annotations (readOnlyHint, idempotentHint). The server makes external API calls but provides no timeout handling, rate limiting, or structured error responses.
Get weather alerts for a state
Get weather forecast for a location
Minimal tool descriptions (18-28 chars). LLM-optimized descriptions should be 50-200 chars and answer: what does it do, when to use it, what does it return. Current descriptions lack context for tool selection.
No output schema documentation. Callers cannot predict response structure. get-alerts returns formatted text; get-forecast returns multi-period forecast. Both should document field names, types, and structure for downstream tool chaining.
Error responses are plain text without recovery guidance. 'Failed to retrieve alerts data' tells the LLM nothing about retry eligibility or next steps. Should follow pattern:recovery-guide with actionable error messages.
No tool annotations. Both tools are read-only (risk: READ_ONLY declared in metadata but not in tool definition). Should use readOnlyHint annotation in schema to signal safety to agents.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
External API calls lack timeout handling and rate limiting. makeNWSRequest() has no timeout; a hung API blocks the entire agent. No backoff or rate-limit awareness for NWS API quotas.