Loading ToolBench
Real-time weather and air quality data provider with health/travel advisories. Uses free APIs (Open-Meteo) requiring no API keys.
Server has 4 tools with complete schemas and reasonable descriptions (avg 120 chars). Naming follows verb_noun convention (get_*). However, descriptions lack actionable context for LLM selection, they state WHAT but not WHEN or WHY to use each tool. Parameter descriptions are minimal (e.g., 'City name' vs 'City name (e.g., Mumbai). Required for geocoding; improves accuracy with country_code'). No output schemas documented. Error handling is absent, no guidance on retryability, user-fixable errors, or recovery paths. Missing tool annotations (readOnlyHint, idempotentHint). get_health_advisory is well-scoped with enum constraints on 'profile', but other tools lack similar rigor.
Get a 5-day weather forecast for any city. Useful for travel planning and event scheduling.
Fetch current air quality index (AQI) and pollutant levels (PM2.5, PM10, CO, NO2, O3) for a city.
Fetch real-time weather for any city. Returns temperature, humidity, wind speed, precipitation, and weather condition.
Generate a health and outdoor activity advisory based on current weather + air quality. Great for runners, elderly, children, and people with respiratory conditions.
No output schemas documented. LLMs cannot plan downstream calls or extract fields without knowing response structure (e.g., does get_current_weather return {temp, humidity, wind_speed} or {current: {temp, ...}}?).
Parameter descriptions are too terse. 'City name' and 'ISO 2-letter country code. Optional.' lack context on why country_code improves accuracy or when it's necessary. Should state: 'City name (e.g., Mumbai, London). Required for geocoding.'
Tool descriptions lack WHEN/WHY context. 'Fetch real-time weather for any city' states WHAT but not when to call this vs get_5day_forecast or get_health_advisory. Should clarify: 'Use for current conditions (now). For future planning, use get_5day_forecast. For health guidance, use get_health_advisory.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
No error handling guidance. If geocoding fails (city not found), the code raises ValueError but provides no recovery path. Should return structured error: 'City not found. Try: (1) add country_code, (2) check spelling, (3) call list_cities() for alternatives.'
Missing tool annotations. All 4 tools are read-only and idempotent but lack readOnlyHint and idempotentHint annotations. LLMs cannot infer safety properties without explicit hints.