Loading ToolBench
Weather MCP Server using OpenWeatherMap API
This server has 4 tools with basic verb naming (get_*) and present descriptions, but significant gaps in schema quality, parameter documentation, error handling, and output specification. All tools are READ_ONLY, which is appropriate, but the implementation lacks the structure and detail expected for production use. Descriptions are minimal (15-105 chars) and lack guidance on when to use each tool vs. alternatives. No input validation is visible, no error recovery guidance is provided, and output schemas are not formally documented. The tools operate on external weather APIs but expose raw responses with minimal curation.
Get 5-day weather forecast for the next 24 hours by city name
Get air quality information for a city including AQI, PM2.5, PM10, CO, and NO2 levels
Get current weather information by city name
Get current weather information by geographic coordinates (latitude and longitude)
No output schema documented for any tool. LLMs cannot plan downstream operations or know what fields to expect. Currently, responses are raw dictionaries with inconsistent field naming across tools (e.g., 'city' vs 'location', 'weather' vs 'forecast_next_24h').
Error responses are minimal and non-actionable. All tools return {'error': message} with no guidance on recovery. Example: 'Failed to fetch weather' gives the LLM no hint to try different parameters or call a different tool.
Tool descriptions are too brief (15 - 105 chars, average ~54 chars) and lack context on when to use each tool vs. alternatives. 'Get current weather information by city name' does not explain when to use get_weather vs. get_weather_by_coords, or what fields are returned.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Parameter descriptions are present but minimal. 'City name to fetch weather for' does not specify format, length, case-sensitivity, or what happens if the city name is ambiguous (e.g., 'Springfield').
No input validation visible. Tools accept free-form city names and coordinate floats with no bounds checking, range validation, or format constraints. get_weather_by_coords accepts any float for latitude/longitude without checking ±180/±90 bounds.
Inconsistent response field naming across tools. get_weather returns 'city' and 'weather', while get_weather_by_coords returns 'location' and 'weather', and get_5day_forecast returns 'city' and 'forecast_next_24h'. This inconsistency confuses LLMs and requires manual field mapping.
No tool composition or chaining support. get_weather_by_coords and get_5day_forecast both require a city name or coordinates, but get_5day_forecast does not accept coordinates, forcing the LLM to call get_weather first to get city name before calling get_5day_forecast.
get_air_quality description misleadingly states it takes a city name, but internally makes two API calls: one to convert city→lat/lon, then one to fetch air quality. If the first call fails with 'invalid city', the error message is 'Invalid city name', but the second call has a different error structure. These intermediate failures are not clearly surfaced.
Forecast tool description says 'Get 5-day weather forecast for the next 24 hours', contradictory. Code shows it returns 8 items (~24 hours), not 5 days. Description should clarify the actual time window and number of data points returned.
No rate limiting or timeout configuration visible. Tools call external OpenWeatherMap API without explicit timeout, which could hang the MCP server if the API is slow or unreachable.