Loading ToolBench
Single tool 'get_weather' has a basic description but lacks critical depth. Description is only 34 chars ('Get the current weather for a city.'), below the 50-char minimum for actionable LLM guidance. Input schema is present with proper type (string) but parameter 'city' has no description, violates the rule that every parameter needs one. No output schema documented. No error handling guidance (what happens for invalid cities? what fields are returned?). Tool is read-only but lacks explicit idempotentHint annotation. No pagination, no rate-limit guidance, no examples of valid inputs or expected output structure.
Get the current weather for a city.
Parameter 'city' lacks description. LLMs cannot infer whether to pass 'Austin', 'austin', 'Austin, TX', or a city code.
Tool description is only 34 characters. Below 50-char minimum for actionable LLM guidance. Should explain WHEN to call it, WHAT it returns, and any prerequisites (e.g., 'Returns current temperature in Fahrenheit and weather condition for a given city. Supports: austin, seattle, new york. Returns {temp_f, condition}.').
Output schema not documented. LLMs do not know what fields to expect (temp_f, condition, etc.). Cannot plan downstream reasoning or extract data reliably.
No error handling guidance. Tool returns {temp_f: 75, condition: 'unknown'} for unknown cities, LLM cannot distinguish between a real 'unknown' condition and a lookup failure. Should return explicit error: {'error': 'City not found. Supported cities: austin, seattle, new york.'}
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Tool lacks idempotentHint annotation. Although read-only and clearly idempotent, explicit annotation helps agents optimize retry logic and caching.