Loading ToolBench
An MCP server that provides weather information tools for cities with get, set, and list operations. Integrates with LangChain agents and Ollama LLM for AI-powered weather queries.
This weather server demonstrates POOR definition quality with significant gaps in descriptions, schemas, and parameter documentation. Tool names follow basic conventions (get_, set_, list_) but descriptions are minimal (9-30 characters), all below the 50-200 character A+ baseline. Parameter schemas are present but lack descriptions for critical fields. No output schemas are documented. The server is designed for demo purposes, not production use. Evidence: get_weather_tool has a 30-char description with no output schema; set_weather_tool lacks parameter descriptions for 'temperature' and 'condition'; list_cities_tool has a 30-char description with no return structure defined. No error handling guidance is present.
Get current weather for a city
List all available cities
Set weather for a city
All tool descriptions are critically short (9-30 characters), below the 50-character minimum for LLM clarity. Descriptions lack context for tool selection and do not explain when to use each tool vs. alternatives.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields. For example, list_cities_tool's return structure is undefined, does it return an array of strings, objects with metadata, or a single string?
set_weather_tool parameters 'temperature' and 'condition' lack descriptions. LLMs cannot infer that 'condition' accepts strings like 'sunny' or 'rainy' without explicit guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 52 | - | v1 |
No parameter constraints or enums. 'condition' parameter accepts free-form strings, LLMs will hallucinate values like 'xyz weather' or 'purple skies'. Should declare: condition as enum with values [sunny, rainy, cloudy, snowy, etc.].
No error handling guidance. If get_weather_tool is called with a non-existent city (e.g., 'atlantis'), the error response should guide the LLM: 'City not found. Call list_cities_tool() to see available cities.' Currently, error behavior is undefined.
set_weather_tool modifies state but description does not explicitly state this. Agents need to know which calls are retryable and which have side effects. Description should read: 'Set weather for a city (modifies state). Use carefully as this updates the shared weather database.'
No input validation constraints documented. 'city' parameter has no format guidance (max length? required? case-sensitive?). Temperature has no range (is -100°F to 150°F valid? Just positive?). LLMs will pass invalid values.