Loading ToolBench
MCP server providing weather information tools
Single tool with reasonable naming and acceptable description, but significant gaps in schema documentation, parameter description, and output structure. The tool definition uses Zod validation internally but does not expose a documented JSON Schema to clients. No error recovery guidance. Output is JSON-stringified rather than structured. Naming follows verb_noun convention (get-weather) which is correct. Description is 33 chars, which meets the minimum bar but lacks detail on when to use the tool, what it returns, or prerequisites. Parameter description is minimal ('City name'). No pagination, no rate-limit guidance, no retry semantics documented.
Get current weather for a location
Tool description lacks actionable context. '33 chars total, missing WHAT it returns, WHEN to use it, and any prerequisites or limits.'
Parameter description is too minimal. 'City name' does not explain format constraints, examples, or whether partial names are accepted.
Output schema is not documented. Function returns a JSON string with fields (temperature, feelsLike, humidity, windSpeed, windGust, conditions, location) but LLMs cannot see the structure, they receive a stringified blob. Must expose schema.
No error handling guidance. If location is not found, the tool throws 'Location not found' but LLMs are not told what to do next (e.g., 'Try a larger region' or 'Search available cities').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
No rate-limit or timeout guidance. Open-Meteo APIs have rate limits but the tool description does not warn LLMs about retry behavior or backoff.
Parameter validation and format constraints not documented. Accept 'London', 'London, UK', coordinates? Tool silently assumes city name format.
No tool annotations present. Missing readOnlyHint (tool is safe to call), which helps agents understand idempotency and side-effect safety.