Loading ToolBench
An MCP server that integrates Ollama language models with MCP tools, providing weather forecasting and alert capabilities through both Python client and Java server implementations
The Ollama MCP Server exposes 2 weather tools with reasonable naming conventions. However, critical quality gaps emerge: (1) tool descriptions lack depth and context about when/why to use each tool, both are functional but minimalist; (2) parameter descriptions are incomplete, 'state' parameter has no description of format constraints (e.g., 'must be 2-letter US state code'), and latitude/longitude have only type info without range or use-case guidance; (3) no output schema documentation visible, callers cannot infer what fields the response contains or how to chain results to downstream tools; (4) no error handling guidance, if an invalid state code is passed or no forecast data exists, what does the tool return?; (5) tool composition is adequate (two distinct concerns: forecast vs alerts) but the interface does not guide the LLM on dependencies or natural sequencing. The server uses Spring AI framework, which handles tool registration via MethodToolCallbackProvider, but the actual tool implementations and output structures are not visible in the provided code excerpt.
Get weather alerts for a US state. Input is Two-letter US state code (e.g. CA, NY)
Get weather forecast for a specific latitude/longitude
Tool descriptions are too brief and lack context. 'Get weather forecast for a specific latitude/longitude' does not explain WHEN to use this tool, what format the response is in, or what coordinates are valid. LLMs cannot infer whether to call this before or after getAlerts, or how to interpret the result.
Parameter 'state' in getAlerts has no description. The tool description mentions '2-letter US state code' but this critical constraint is NOT in the parameter schema. LLMs cannot validate input before calling and may pass invalid values like 'California' or 'ca' (lowercase).
No output schemas documented. The source code does not show what fields getWeatherForecastByLocation or getAlerts return. LLMs cannot plan follow-up calls or extract the data they need. If forecast returns 'temp' and 'humidity', but the LLM expects 'temperature' and 'dew_point', it will fail to extract values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 40 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Numeric parameters (latitude, longitude) lack range constraints. The schema shows type='number' and format='double', but no min/max bounds. LLMs may pass latitude=900 or longitude=-200, which are invalid geographic coordinates. Descriptions should state '-90 to 90 for latitude, -180 to 180 for longitude'.
No error handling guidance. What does the tool return if coordinates are over the ocean, if the state code is invalid, or if the weather service is unreachable? Responses must tell LLMs what to do next (retry, ask user, try alternative tool).