Loading ToolBench
A weather information server that provides current weather conditions and multi-day forecasts for cities using the Open-Meteo API.
Two tools with clear verb-noun naming (get_current_weather, get_forecast) and basic descriptions. Input schemas are present with Zod validation and parameter descriptions. However, descriptions are minimal (10-20 chars range), output schemas are undocumented, and error handling lacks recovery guidance. No tool annotations. Composition is sound (single responsibility), but parameter constraints (days 1-7) are enforced in code, not declared in schema. Baseline: avg tool description ~194 chars; these are ~60 chars. No pagination, no structured output documentation.
Get current weather conditions for a city.
Get a multi-day weather forecast for a city (1-7 days, default 3).
Output schema undocumented. Tools return text-formatted strings; LLMs cannot parse structured fields for downstream chaining. No documented return type, field names, or structure.
Descriptions too brief (10-20 chars). 'Get current weather conditions for a city.' lacks WHEN to use, WHAT it returns, and prerequisites. Baseline: 194 chars avg for A+ tools. These are 60% below baseline.
Parameter constraint 'days 1-7' enforced in clampDays() but not declared in schema. Zod schema lacks .min(1).max(7). LLMs cannot see the constraint and may pass invalid values.
Error handling returns isError flag with text content, but no recovery guidance. 'Could not find location: Berlin' tells LLM nothing about next steps (retry? search? ask user?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent but this is not declared in the schema, forcing LLMs to infer safety.