Loading ToolBench
Single tool with minimal definition quality. Tool name 'get_forecast' is appropriately verb-prefixed, but description is extremely brief (34 chars, below the 50-char minimum for clarity). Input schema present with proper types (float) and basic descriptions, but lacks critical metadata: no output schema documentation, no error handling guidance, no parameter constraints (latitude/longitude ranges), and no indication of what the response structure contains. The tool returns raw API JSON as a string, which violates structured output patterns. No pagination, no rate limiting info, no timeout documentation despite calling external APIs.
Get weather forecast for a location.
Output schema not documented. Tool returns raw dict/string with no description of response structure (current, daily fields, units, etc.). LLMs cannot plan downstream operations or extract relevant data.
Tool description is 34 characters ('Get weather forecast for a location.'), below the 50-char minimum for LLM clarity. Does not explain WHEN to use it, what it returns, or prerequisites. Baseline for A+ tools is 194 chars (p10=34, p90=392).
No parameter constraints. Latitude and longitude lack min/max bounds, format hints, or valid range documentation. LLMs may pass invalid coordinates (e.g., latitude > 90) without validation.
Error handling returns None or generic string ('Unable to fetch detailed forecast.') with no recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
No timeout or rate-limit documentation. Tool calls external API with 30s timeout but does not communicate this to LLM. No guidance on retry behavior or rate limits.