Loading ToolBench
Get weather forecast information using wttr.in with Open-Meteo fallback
Single tool with basic registration but significant quality gaps. Tool name follows verb_noun pattern (get_weather) which is positive, but descriptions lack specificity for LLM optimization. Input schema is present but minimal. No output schema documentation. No error handling guidance. No parameter enums or constraints despite clear valid options (cities). No tool annotations (readOnlyHint, idempotentHint). The implementation shows working weather logic but the interface definition is below production standards.
Get weather forecast information for a specified city using wttr.in (Open-Meteo fallback). Parameters: city: city name, e.g., "Beijing", "New York", "Tokyo", "武汉" If None, it will return the weather for the current city. Returns: str: weather forecast information in Markdown format.
Missing output schema documentation. Tool returns 'str: weather forecast information in Markdown format' but LLMs need to know the actual structure (is it always Markdown? Are there fallback formats?). No documented return contract prevents agents from planning downstream processing.
Parameter description contains example values ('Beijing', 'New York', 'Tokyo', '武汉') per the city parameter. LLMs tend to reuse example values literally rather than adapting to the actual context.' These should be removed or moved to a separate constraint specification.
No input validation or error recovery guidance. The tool accepts None or any string as city, but provides no enum of valid cities, no format constraints, and no error message guidance for invalid cities. If wttr.in returns 404 for a malformed city name, the LLM has no way to know what went wrong or how to recover.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 46 | - | v1 |
Missing tool annotations (readOnlyHint, idempotentHint). The tool is READ_ONLY and idempotent (fetching weather for the same city twice returns the same result) but these are not declared in the tool registration, forcing LLMs to infer safety properties.
No parameter type constraint for 'city'. It is declared optional (required=false) but accepts any string. No enum, no regex pattern, no length limits. The parameter description says 'If None, it will return the weather for the current city' but the implementation only shows how to handle None, unclear what happens with invalid city names, empty strings, or extremely long strings.