Loading ToolBench
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Both tools have action-verb names (get-alerts, get-forecast) and descriptions present, but descriptions are minimal (18-34 chars, well below the 50-200 char LLM-optimized baseline). Input schemas use Zod with type constraints (latitude/longitude bounds, state length), but parameter descriptions lack context about expected formats, error cases, and recovery paths. Output schemas are not documented, LLMs cannot predict response structure. Error handling returns plain text without actionable guidance. No tool annotations (readOnlyHint, idempotentHint) despite both being read-only operations.
Get weather alerts for State
Get weather forecast for a location
Minimal tool descriptions (18-34 chars). 'Get weather alerts for State' and 'Get weather forecast for a location' lack context about when to use each tool, what data is returned, and prerequisites (e.g., 'get-forecast only supports US coordinates via NWS API'). LLM-optimized descriptions should be 50-200 chars and answer: what, when, and what's returned.
Output schemas not documented. Tool implementations return {content: [{type: 'text', text: string}]} but this structure is not declared in the tool registration. LLMs cannot predict response fields, forcing them to guess at downstream data extraction and chaining.
Error responses lack actionable recovery guidance. When API calls fail (e.g., 'Failed to retrieve alerts data'), the response is a plain string with no hint about what the LLM should do next (retry, try a different state code, check coordinates). Per pattern:recovery-guide, errors must tell the agent what to do.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Parameter descriptions are minimal and lack format/constraint details. 'Two-letter state code (e.g. CA, NY)' includes an example (anti-pattern, LLMs reuse examples literally) instead of a formal constraint. Latitude/longitude descriptions lack mention of US-only support or precision expectations.
No tool annotations despite both tools being read-only. Adding readOnlyHint: true to both tool definitions would signal to agents that these calls are safe to retry and have no side effects, improving agent planning.