Loading ToolBench
MCP server that provides current weather information for specified cities
Single-tool server with moderate quality. The get_current_weather tool has a valid name starting with 'get_' verb and basic schema, but descriptions lack depth and parameter documentation is minimal. Tool description is acceptable (54 chars) but parameter description is extremely sparse ('City name (e.g., "London", "New York", "Tokyo")', 60 chars, borderline). No documented output schema visible in source. No input validation, error handling guidance, or recovery patterns. The tool name is clear but the implementation relies on mock data with hardcoded fallbacks rather than robust error handling patterns expected in production tools.
Get current weather for a specified city
No documented output schema. The tool description states what it does but provides no schema for the response structure. LLMs cannot plan downstream tool calls or extract the right data without knowing what fields to expect.
Parameter description lacks actionable detail. The 'city' parameter description is example-based ('e.g., London, New York, Tokyo') rather than constraint-based. LLMs tend to reuse example values literally instead of adapting to context. Should state: 'City name as a string (e.g., London). The tool supports major world cities and returns mock data for cities not in the predefined list.'
No error handling guidance. The implementation silently falls back to mock data or default weather data without documenting this behavior in the tool description. If the API fails, the LLM receives fabricated data with no indication it's synthetic. Agents cannot distinguish real failures from fallbacks.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
No input validation or constraint documentation. The tool accepts any string as a city name with no length limits, character restrictions, or validation rules stated. LLMs could pass extremely long strings, special characters, or injection attempts without guidance on what constitutes valid input.
Mock data hardcoded in implementation. The tool returns hardcoded mock weather data for predefined cities (London, New York, Tokyo, Paris, Sydney) with no clear documentation that users may receive synthetic data. Real production tools should clearly separate live and fallback modes in descriptions and return metadata indicating data source.
Response bloat and token waste. The mock weather response object contains many fields (sunrise, sunset, cloudiness, wind_deg, pressure, icon URL) that may not be relevant to chat interactions. According to production baselines, responses should be stripped to only the fields the agent needs. This dilutes signal and increases context window consumption.