Loading ToolBench
Single tool with minimal definition quality. Tool name 'fetch-weather' uses a verb prefix (fetch) which is good, but the description is sparse (31 chars, below the 50-200 char LLM-optimized range). Input schema is present with basic type but lacks detail in parameter descriptions. Output schema is entirely undocumented, the tool returns structured JSON but the agent has no visibility into what fields to expect. Error handling is minimal; a missing city returns a Spanish error message in an English codebase with no clear recovery path. No input validation, no pagination despite potentially returning weather data. The tool is simple but cuts corners on every dimension except the most basic schema presence.
Tool to fetch the weather of a city
Output schema completely undocumented. Tool returns JSON structure (hourly, temperature_2m, etc.) with no declared schema. LLM cannot plan downstream calls or extract specific fields reliably.
Tool description too brief (31 chars, below 50 char minimum for LLM clarity). Does not explain WHEN to use it, WHAT it returns, or error handling. Current text 'Tool to fetch the weather of a city' lacks actionable context.
Parameter description trivial ('City name', 9 chars). Should specify format expectations: is 'New York' valid? 'new york'? 'NYC'? Spell out constraints to guide LLM input.
Error handling weak. Missing city returns Spanish message ('No se encontro informacion ara la ciudad') in English codebase, with no recovery guidance. Error should state 'City not found. Try a different spelling or country name (e.g., 'London, UK').'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
No input validation. Tool accepts any string for 'city' without sanitization or constraint. Should reject empty strings, enforce max length (e.g., 100 chars), and validate against geocoding API rate limits.
External API calls unguarded by timeout. If geocoding-api.open-meteo.com or api.open-meteo.com hangs, agent blocks indefinitely. Add explicit timeout (e.g., 10 seconds) and return retryable error.
console.log(data) in production code. Logs raw API responses to stderr, wasting tokens and potentially leaking data. Remove or use structured logging.