Loading ToolBench
Single tool 'get_forecast' has a clear verb-noun name and proper JSON Schema with typed parameters (city: string, days: number with min/max constraints). Description is adequate (47 chars) but minimal. Parameter descriptions are present but terse. No output schema documented, LLMs cannot predict response structure. Resource handler exists but is hardcoded to San Francisco, limiting utility. Error handling returns structured responses but lacks recovery guidance.
Get weather forecast for a city
No output schema documented. Tool returns JSON array of forecast objects, but LLMs cannot predict field names (date, temperature, conditions) or types. Agents cannot plan downstream operations or extract data reliably.
Tool description is 47 characters, below the 50-200 char baseline for LLM-optimized descriptions. Lacks context on when to use (vs. current weather), what the response contains, or any prerequisites.
Parameter 'days' description '"Number of days (1-5)"' is terse. Does not explain what 'days' means (forecast horizon? number of data points?). Constraint is in schema but not in description text, LLMs may not parse JSON Schema constraints reliably.
Error handling in CallToolRequestSchema returns isError: true with plain text message, but does not guide recovery. E.g., 'Weather API error: city not found' does not suggest calling a search_cities tool or trying an alternative city name.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Resource handler hardcoded to San Francisco only. ListResources returns a single static resource; ReadResource rejects any URI other than 'weather://San Francisco/current'. Severely limits utility, agents cannot query weather for arbitrary cities via resources.