Loading ToolBench
Single tool with minimal definition quality. Tool name 'fetch weather' lacks action verb prefix (should be 'get_weather' or 'fetch_weather_data'). Description is present but generic (18 chars, below 34-char p10 baseline). Input schema exists with basic type but parameter description is minimal ('City name', 9 chars, well below 72-char average). No output schema documented. No error handling guidance for common failures (city not found, API timeout, invalid input). Tool returns unstructured JSON dump rather than shaped response. No pagination, limits, or result constraints despite calling external API.
Tool to fetch the weather of a city
Tool name does not start with action verb. 'fetch weather' is ambiguous; should be 'get_weather' or 'fetch_weather_data' to signal intent clearly.
Parameter description is trivial (9 chars). 'City name' provides no format guidance, constraints, or examples. Should specify: expected format, whether partial names work, max length, and what happens if ambiguous.
No output schema documented. Tool returns raw JSON from API without shaping. LLM cannot predict response structure, forcing unstructured parsing. Should document: fields returned, data types, and which fields are safe to chain to other tools.
Error handling lacks recovery guidance. Returns generic text messages ('Failed to fetch data', 'City not found') without suggesting next steps. Should guide LLM: retry with partial name, try alternative city, or check API status.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
Tool description is too short (18 chars, below p10=34 baseline). 'Tool to fetch the weather of a city' lacks context on when to use it, what data is returned, and any prerequisites. Should explain: returns location metadata (not current weather), useful for geocoding, requires valid city name.