Loading ToolBench
A production-ready Model Context Protocol server exposing live weather tools.
Two tools with clear verb-noun naming (get_current_weather, get_forecast) and well-structured JSON schemas. Both have descriptions (194 and 56 chars respectively) and input parameters with type constraints (minLength, maxLength, min/max for days). Tool descriptions explain WHAT they do and WHEN to use them. However, descriptions lack explicit guidance on return structure, error cases, and when to prefer one tool over the other. Parameter descriptions are adequate but could be more prescriptive about format expectations. No output schema documentation visible in source. Error handling strategy not evident from code sample.
Get the current weather for a city. Returns temperature, humidity, wind and sky conditions.
Get a multi-day weather forecast for a city (1-14 days).
Output schemas not documented. LLMs cannot plan downstream operations without knowing what fields to expect (e.g., does get_current_weather return wind_speed_kmh or wind_speed_mph? Both?).
Tool descriptions lack error recovery guidance. No mention of what happens when a city is not found, API is down, or input is invalid. LLMs need explicit recovery paths.
get_forecast description (56 chars) is below the 100-char minimum for adequate context. Does not explain when to use forecast vs current weather, or what '1-14 days' means in practice.
No distinction between tools when both accept 'city' parameter. LLM must infer from names alone that get_current_weather returns now, get_forecast returns future. Descriptions should explicitly state this.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
City parameter accepts any string 1-100 chars. No guidance on format (e.g., 'Tokyo' vs 'Tokyo, Japan' vs 'JP-Tokyo'). LLMs may pass ambiguous city names causing failures.