Loading ToolBench
A server to retrieve and cache weather data from OpenWeather
The server defines 4 tools with basic functional descriptions and Zod-based input schemas. However, multiple critical gaps reduce quality: (1) descriptions are generic and lack actionable context for LLM tool selection; (2) parameter descriptions are minimal or missing entirely; (3) no output schema documentation; (4) error handling is minimal with no recovery guidance; (5) no tool annotations (readOnlyHint/destructiveHint) despite clear semantic differences between read and write tools. Naming is verb-noun correct but descriptions fall short of production standards. The code shows competent implementation (caching, Redis integration, proper formatting) but the tool interface lacks the detail required for reliable agentic tool use.
Clear cached weather data for a city
Get the current weather for a city
Get the current weather using latitude and longitude coordinates
Get a 5-day weather forecast for a city
Tool descriptions lack context for LLM selection and decision-making. Descriptions like 'Get the current weather for a city' (40 chars) do not explain when to use this tool vs. alternatives, what the output contains, or prerequisites. Production baseline is 50-200 chars with WHAT/WHEN/OUTPUT detail.
No output schema documentation. Tools return formatted text via content[].text, but LLMs cannot infer the structure or available fields. Responses should declare the JSON structure (e.g., {temperature, humidity, conditions, location}) so agents can extract and chain data.
Minimal error handling and no recovery guidance. Failure responses ('Failed to retrieve weather data...') do not tell the LLM what to do next. No distinction between retryable errors (API timeout) vs. user-fixable errors (invalid city name) vs. fatal errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Parameter 'city' in get-current-weather and get-weather-forecast lacks meaningful description. Description is 'Name of the city to get weather for' (39 chars), which repeats the tool name. Should clarify: expected format (e.g., 'London, UK' or just 'London'), whether partial matches are accepted, and what happens with ambiguous names.
Parameters 'lat' and 'lon' in get-weather-by-coordinates lack descriptions explaining acceptable ranges, precision, or what happens at invalid coordinates. Expected: 'Latitude in decimal degrees (-90 to 90)' and 'Longitude in decimal degrees (-180 to 180)'.
No tool annotations despite clear semantic differences. 'get-current-weather', 'get-weather-forecast', and 'get-weather-by-coordinates' are READ_ONLY and safe to retry; 'clear-weather-cache' is WRITE and has side effects. Should use readOnlyHint: true / destructiveHint: true to guide agent behavior.
'clear-weather-cache' has optional parameter 'city' (omitting it clears all cache). Description is vague: 'Name of the city to clear cache for. If not provided, clears all weather cache.' Should explicitly warn that clearing all cache is a destructive operation and should be confirmed with the user.
Responses embed formatted text instead of structured JSON. formatWeatherData() and formatForecastData() return multi-line strings (40-60 lines). LLMs cannot easily extract temperature, humidity, or wind speed for downstream reasoning or extraction tasks. Should return structured {temperature, humidity, conditions, location, sunrise, sunset, wind} objects.