Loading ToolBench
A Model Context Protocol server that provides weather information via OpenWeatherMap API. Supports multiple transport modes (stdio, SSE, streamable-http) with optional OAuth 2.0 Bearer Token authentication.
The server exposes 3 tools with mixed quality. The weather.get_weather tool has a reasonable schema and description (61 chars), but health_check and get_404_page are poorly scoped utility endpoints that should not be exposed as MCP tools. Only 1 of 3 tools has meaningful input parameters; the other two accept empty objects. Parameter descriptions are present but minimal (12-15 chars on average). No output schemas are documented. Error handling is not evident in the definitions. Tool naming is reasonable (verb_noun) but the overall composition suggests confusion between HTTP health checks and actual user-facing tools.
Get a custom 404 error page. Returns HTML content for a nice 404 error page.
Health check endpoint for Docker. Returns a status indicating if the service is running.
Get weather for a city and day offset (0=today/current, 1=+1 day, ... up to 15). Returns a dictionary with date, temperature (including min/max), and weather description.
health_check and get_404_page should not be exposed as MCP tools. They are HTTP infrastructure utilities (Docker health probe, error page). They do not represent user-facing capabilities and clutter the tool namespace. Remove them or gate them behind a separate internal interface.
health_check and get_404_page have empty input schemas ({}). Per the scoring rules, tools with no input schema must score 0 on schema. Additionally, their descriptions (35 and 30 chars) fall below the 50-char minimum and provide no context for when or why an LLM would invoke them.
No output schemas are documented for any tool. The tool descriptions do not specify what fields or structure callers should expect. LLMs cannot plan downstream operations or extract the right data without knowing the response format. health_check likely returns {status: string}, weather.get_weather returns {date, temperature_min, temperature_max, description}, but this is not formally declared.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Parameter descriptions for weather.get_weather are minimal. 'city' is described as 'City name, optionally with country code (e.g., "London,uk")', good example, but does not specify format constraints or expected fallback behavior if the city is not found. 'days' description lacks bounds and error handling guidance (what if days > 15? Truncate or error?).
No error handling guidance. What does weather.get_weather return if the city is invalid or the API is unreachable? The descriptions do not tell the LLM what to do next or how to recover. Per pattern:recovery-guide, error responses must guide the agent's next step.