Loading ToolBench
Single tool with basic structure but significant quality gaps. Tool name 'weather_get' lacks action verb clarity (should be 'get_weather'). Description is adequate (67 chars) but parameter lacks type specification in schema, only 'string' type visible, no format/pattern constraints. No output schema documented. Error handling returns custom objects but lacks recovery guidance. Mock geocoding implementation severely limits real-world utility. No pagination, no rate limiting, no input validation beyond null checks.
Fetches weather data from Open-Meteo API based on zipcode
Tool name 'weather_get' violates verb_noun convention. Should be 'get_weather' for clarity and LLM parsing.
No output schema documented. LLMs cannot plan downstream calls or extract structured data from weather response.
Zipcode parameter lacks format constraint (e.g., regex '^\d{5}$'). Description says 'US zipcode' but does not specify 5-digit format or validation rules.
Mock geocoding hardcoded for 5 zipcodes; all others default to New York. Real implementation requires actual geocoding API (Google Maps, OpenCage) or zipcode database.
Error responses lack recovery guidance. 'Could not find coordinates for zipcode' does not tell LLM what to do next (retry, ask user, try different input).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |