Loading ToolBench
A cookiecutter template for building MCP servers in Python with support for tools, prompts, and resources
This is a template/cookiecutter server with acceptable tool definitions but significant gaps in parameter descriptions and schema completeness. Both tools (get_alerts, get_forecast) have clear naming (verb_noun structure) and reasonable descriptions (90-140 chars, within baseline range). However, parameter descriptions are minimal or missing, schemas lack comprehensive constraint declarations, and there is no documented error handling guidance. The template nature means some evaluation is conservative, actual instantiated servers would need review. Output schemas are not explicitly documented. No input validation constraints (e.g., state code format for get_alerts, lat/long ranges for get_forecast). Risk annotations are present but do not cover all error cases or recovery paths.
Get weather alerts for a US state. Fetches active weather alerts from the NWS API for a specified US state.
Get weather forecast for a location. Fetches the weather forecast from the NWS API for a specified location using latitude and longitude coordinates.
Parameter descriptions are minimal. 'state' param has only 'Two-letter US state code (e.g. CA, NY)', no guidance on edge cases (lowercase, invalid codes), no validation error message format documented.
Latitude/longitude parameters in get_forecast lack range constraints. No documentation of valid ranges (±90 for lat, ±180 for lon), precision requirements, or invalid-value error messages. LLMs may pass out-of-range coordinates without guidance on what went wrong.
No output schema documented. Tools return unstructured strings (formatted alert text, forecast text). Output structure should specify: for get_alerts, is it a single concatenated string or an array of alert objects? For get_forecast, what fields does the forecast contain, in what format? Current responses lack field-level typing.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 45 | - | v1 |
Error handling is implicit, not explicit. Code handles errors by returning generic messages ('Unable to fetch alerts or no alerts found', 'No active alerts for this state'). Valid codes are 2-letter US abbreviations: CA, NY, etc.' Current approach leaves agent guessing.
No input validation constraints documented. get_alerts accepts 'state' as a string with no length, format, or enum constraint. Should declare: 'state must be a 2-letter uppercase US state code (e.g., CA, NY, TX)' or use an enum. Without this, LLM may pass invalid values (3-letter codes, lowercase, invalid abbreviations).
Tool descriptions lack WHEN-to-use guidance. Both tools state WHAT they do but not WHEN to call them vs. alternatives or prerequisites. State WHAT the tool does, WHEN to use it, and any prerequisites.' get_alerts could add: 'Use when user asks about active weather hazards for a state.' get_forecast could add: 'Use when user provides a location (city/coordinates) and wants weather predictions.'
No idempotence declaration. Both tools are read-only and thus idempotent, but the tool descriptions do not state this explicitly.
No pagination or result-limiting guidance. For get_alerts, response is: 'No active alerts for this state' or a joined string of all active alerts. If a state has 100 active alerts, response could be extremely long.