Loading ToolBench
A FastAPI-based MCP server that provides weather information via OpenWeatherMap API
Single tool with basic schema and description. Tool name follows verb_noun pattern (get_weather), but description is minimal (34 chars, below baseline 194). Input schema present with types and descriptions, but lacks output schema documentation. No error handling guidance, no parameter constraints (units should be enum), no security considerations for API key exposure. Missing idempotency hints, pagination, and recovery guidance.
Retrieves current weather information for a specified city
Description too short (34 chars vs baseline 194). Lacks WHEN to use, prerequisites, and return value documentation. LLM cannot determine when to select this tool over alternatives.
Output schema not documented. LLM cannot plan downstream operations or extract required fields. Should document: city (string), temperature (number), humidity (number), condition (string).
Parameter 'units' accepts free-form string but should be enum constraint (metric|imperial). Current implementation invites hallucinated values like 'celsius' or 'fahrenheit'.
No error handling guidance. Tool returns generic 'City not found or API failed' without recovery hints. Should guide LLM: 'Try search_cities() with partial name' or 'Check spelling'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
API key exposed in environment variable without validation. Tool should verify OPENWEATHER_API_KEY is set at startup and return clear error if missing, not generic 'API key not configured'.