Loading ToolBench
Two tools with complete input schemas and descriptions. Tool naming follows verb_noun pattern (get_current_weather, get_forecast). Descriptions are adequate (100-150 chars) but lack actionable context for LLM selection. Parameters are well-typed with constraints (lat/lon bounds, enum units, count limits). However, output schemas are undocumented, LLMs cannot predict response structure. Error handling returns text but lacks recovery guidance. Tool annotations present on get_forecast but missing on get_current_weather (inconsistent). No parameter descriptions for 'city' in get_current_weather explain the format (e.g., 'City name or city,country code').
Get current weather for a city name or latitude/longitude using OpenWeatherMap, including umbrella advice and outdoor checks
Get a 5-day / 3-hour weather forecast for a location using OpenWeatherMap, including umbrella advice and outdoor checks per period
Output schemas undocumented. LLMs cannot predict response structure (fields, types, nesting). Responses are formatted as plain text, not structured JSON, forces LLMs to parse unstructured output.
Tool annotations inconsistent: get_forecast has weatherToolAnnotations (readOnlyHint, idempotentHint, openWorldHint), but get_current_weather has empty annotations object. Both are read-only; both should declare the same hints.
Error handling returns plain text without recovery guidance. 'Failed to retrieve current weather data' tells LLM nothing about retry eligibility or next steps. Should indicate: retryable? user-fixable? fatal?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Parameter 'city' in get_current_weather lacks format guidance. Description says 'City name, e.g. "Lagos" or "London,GB"' but does not state whether country code is required, optional, or format (ISO 3166-1 alpha-2). LLMs may pass invalid formats.
No mutual exclusivity guidance for get_current_weather parameters. Description says 'Provide either a city name, or both latitude and longitude' but does not state what happens if both are provided or if neither is provided. Code validates but error message is generic.