Loading ToolBench
Aviation weather MCP server — METARs, TAFs, PIREPs, SIGMETs, ICON-D2 NWP forecasts, and ensemble VFR outlook. No API key required.
Aviation Weather MCP has 8 well-scoped tools with consistent naming (wx_* prefix) and clear READ_ONLY risk classification. All tools have descriptions (10 - 50 chars) and input schemas with type definitions. However, descriptions are uniformly terse (e.g., 'current METAR, decoded' is 20 chars), falling below the 50 - 200 char baseline for LLM-optimized descriptions. Parameter descriptions are minimal ('ICAO airport code (e.g., KJFK)' is adequate but sparse). Output schemas are not documented in the visible code, the tool implementations return complex nested objects (MetarData, TafData, OpenMeteoResponse, EnsembleResponse) but these structures are not exposed in the tool registration. Error handling is present (timeouts, HTTP status checks, network diagnostics) but recovery guidance is limited. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are visible, despite all tools being read-only.
hourly ICON-D2 NWP forecast for any lat/lon
current METAR, decoded
3–15 day ensemble VFR confidence outlook: per-day P(No-Go)
recent PIREPs near an airport
composite VFR brief: METAR + TAF + ICON-D2
direct-route brief: METARs + 5-point ICON-D2 + SIGMETs
active SIGMETs/AIRMETs (global / europe / germany filter)
Tool descriptions are uniformly terse (20 - 50 chars), below the 50 - 200 char baseline. LLMs cannot determine when to select wx_metar vs wx_preflight vs wx_route without richer context. E.g., 'current METAR, decoded' does not explain that wx_metar returns only the latest observation, while wx_preflight includes TAF and forecast data.
Output schemas are not documented. The code defines MetarData, TafData, OpenMeteoResponse, EnsembleResponse interfaces, but these are not exposed in tool registration or descriptions. LLMs cannot plan downstream operations or extract required fields without knowing the response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
TAF forecast periods, decoded
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. All 8 tools are read-only and idempotent, but this is not declared in the tool definitions. Agents cannot infer safety properties without explicit annotations.
Parameter descriptions lack format/constraint details. E.g., 'ICAO airport code (e.g., KJFK)' does not specify length (4 chars), character set (uppercase letters), or validation rules. LLMs may pass invalid codes like 'kjfk' or 'KJFK1'.
Error recovery guidance is minimal. Network timeouts and HTTP errors are caught with clear messages (e.g., 'ensemble-api.open-meteo.com did not respond within 20s'), but no guidance on retry strategy, fallback tools, or user-facing recovery steps.