Loading ToolBench
Collection of MCP servers for fetching weather data using the OpenWeather API
This repository contains three weather tools across two implementations (Python FastMCP and TypeScript SDK). Critical issues: Tool #2 (test_weather/get_weather) has zero input parameters but no description explaining why or what it returns beyond a hardcoded string. Tool #1 and #3 have basic schemas but lack parameter descriptions (null values visible in TypeScript schema). Naming is acceptable (verb_noun pattern), but descriptions are minimal (10-30 chars) and do not explain when to use each tool or their differences. No output schemas documented. Error handling is present in implementation but not surfaced in tool definitions. The repository appears to be a collection of examples rather than a cohesive, production-ready server.
Returns the current weather for a given zip code.
Returns a weather message.
Tool #2 (test_weather/get_weather) has no input parameters and no description. It returns a hardcoded string 'the weather is cool' regardless of input. This violates the tool-description pattern, LLMs cannot determine when or why to select this tool.
TypeScript tool (getWeather) has parameter descriptions set to null in the schema. The zip_code and country_code parameters lack descriptions explaining their format, constraints, or purpose. LLMs cannot infer parameter meaning from names alone.
No output schemas documented for any tool. LLMs do not know what fields to expect in responses, making it impossible to plan downstream tool calls or extract structured data. Baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Three tools with nearly identical names (two 'get_weather', one 'getWeather') create ambiguity. LLMs cannot distinguish between them without reading full descriptions. The test_weather variant is non-functional (hardcoded response), suggesting this is a development artifact that should not be exposed.
Tool descriptions are 10 - 30 characters, below the 50-character minimum for LLM clarity. 'Returns the current weather for a given zip code' (real_weather) is acceptable, but 'Returns a weather message' (test_weather) and missing descriptions (TypeScript) fail to guide tool selection.