Loading ToolBench
MCP Server exposing three tools over stdio transport: get_weather (real weather via Open-Meteo), calculator (evaluates arithmetic expressions), and get_time (returns current time for IANA timezones)
Three tools with clear names and reasonable descriptions, but significant gaps in schema completeness, parameter documentation, and error handling guidance. All tools are read-only and have basic input validation. Descriptions are present but lack the depth needed for optimal LLM tool selection. No output schemas documented. Error messages are present but lack recovery guidance.
Evaluate a basic arithmetic expression and return the result. Supports + - * / ( ) and decimal numbers only.
Return the current time formatted for an IANA timezone (e.g. 'America/New_York'). Defaults to UTC.
Return real current weather for any city using Open-Meteo (no API key needed). Returns temperature, feels-like, humidity, wind speed, and conditions.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract data without knowing return structure.
Error messages lack recovery guidance. Examples: get_weather returns JSON with 'error' field but does not suggest next steps (e.g., 'Try a different spelling'). calculator returns validation errors but does not guide on valid format.
Parameter descriptions in input schemas are either missing or repetitive. 'calculator' parameter description duplicates the tool description. 'get_time' timezone_name description is adequate but lacks constraint detail (no regex pattern in schema).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
No pagination or result-limit documentation. get_weather returns one location (hardcoded count=1 in API call, which is good), but this limitation is not documented in the tool description. If Open-Meteo API fails or times out, no timeout or retry guidance is provided.
calculator uses eval() with a regex whitelist, which is a security risk. While the regex attempts to restrict input, eval() is inherently dangerous and could be exploited via edge cases (e.g., nested expressions, scientific notation). Better approach: use ast.literal_eval or a dedicated expression parser.
Tool descriptions lack dependency hints and prerequisites. No guidance on when to call get_time vs get_weather, or whether get_weather requires network access or will fail offline.