Loading ToolBench
Single tool with minimal definition quality. Tool name follows verb_noun convention (get_weather), but description is extremely sparse (24 chars, below 34-char p10 baseline). No output schema documented. No error handling, no recovery guidance. Description under 20 chars in context of quality expectations scores 0.
Returns weather of the city
Parameter 'city' has no description. LLM cannot infer what format is expected (city name vs code vs coordinates).
Tool description is 24 characters: 'Returns weather of the city'. Baseline p10=34 chars. Below minimum threshold for actionable LLM guidance. Missing WHAT (what fields returned?), WHEN (when call instead of similar tools?), and any prerequisites.
No output schema documented. LLM has no way to know what get_weather returns (string? object? fields?). LLMs need to know what fields to expect so they can plan downstream tool calls and extract the right data.' All 100% of A+ tools document return types.
No error handling or recovery guidance. If city is invalid or service fails, LLM gets no actionable next step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Tool returns a free-form string ('Savannah weather is good'). No structured output. Free-text responses require the LLM to parse unstructured output, which is error-prone and wastes tokens.'
Parameter schema lacks description field entirely. JSON input schema shows type but no 'description' key.