Loading ToolBench
Two tools with acceptable naming and basic descriptions, but missing critical schema documentation, parameter descriptions, and error guidance. Both tools use action verbs (get_*) appropriately. Descriptions are present but minimal (14-31 chars, well below the 50-200 char LLM-optimized baseline). Input schemas exist but lack parameter-level descriptions. Output is free-form text with no documented schema. No error handling guidance. No pagination despite potential for multiple alerts. Tool definitions are explicitly visible in code (source/core.py), so not capped at 50 on that basis.
Get weather alerts for a US state.
Get weather forecast for a location.
Parameter descriptions missing. Input schema defines 'state' and 'latitude'/'longitude' with types, but lacks descriptions explaining accepted format, constraints, or usage context. Rubric requires every parameter to have a non-empty description so LLMs understand what values are valid and why.
Tool descriptions are too short (14 and 18 chars respectively). Baseline for LLM-optimized descriptions is 50 - 200 chars. Current descriptions lack WHEN to use this tool, WHAT it returns, and any prerequisites. Agents cannot reliably select between similar tools with minimal descriptions.
Output schema not documented. Both tools return free-form strings with no structured schema defined. LLMs cannot plan downstream operations or extract specific fields. Rubric requires 100% of A+ tools to have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
No pagination support for get_alerts, despite potential for returning multiple alerts per state. Rubric states tools returning lists should accept limit/offset and return a total count or next_cursor. Current implementation caps output to formatted string but does not expose pagination to caller.
No error recovery guidance. Error handling (make_nws_requests) returns None and generic error strings ('Unable to fetch alerts'). Rubric requires error responses to tell the LLM what to do next and categorize errors as retryable, user-fixable, or fatal. Current errors provide no actionable guidance.
Parameter naming not fully compliant with best practice. 'state' is acceptable but not suffixed with a type (e.g., 'state_code' would be clearer). 'latitude' and 'longitude' are clear but no validation constraints (e.g., range ±90, ±180) are documented in descriptions.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only and idempotent, but these properties are not declared in the schema or via annotations, forcing LLMs to infer them from descriptions.