Loading ToolBench
A quick-start MCP server that provides weather forecast and alert tools using the National Weather Service API
Two tools with basic schemas and minimal descriptions. Both tools have proper names starting with action verbs (get_alerts, get_forecast), but descriptions are extremely brief (under 50 chars) and lack context about when to use them, expected return formats, or error cases. Input schemas are present and valid JSON Schema, but parameter descriptions are generic. No output schema documentation. No error handling guidance. The code shows proper async implementation but the tool definitions themselves are underdeveloped for LLM use.
Get weather alerts for a US state.
Get weather forecast for a location.
Tool descriptions are under 20 characters and provide no actionable context. 'Get weather alerts for a US state' and 'Get weather forecast for a location' are too terse. LLMs need to know: when to use this tool, what format is returned, what errors are possible, and whether the call has side effects.
No output schema documentation. Tools return plain text strings (e.g., 'No active alerts for this state') but LLMs do not know this in advance. Return types should be declared so agents can plan downstream operations and extract structured data reliably.
Parameter descriptions are bare minimum. 'Two-letter US state code (e.g. CA, NY)' includes example values instead of constraint descriptions. The rubric states 'Do not put example values in the description. LLMs tend to reuse example values literally rather than adapting to the actual context. Use parameter constraints (enums, patterns) instead.' For get_alerts state parameter, provide an enum of valid US state abbreviations or a regex pattern, not examples.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 44 | - | v1 |
No error handling guidance. The implementation catches exceptions and returns None or 'Unable to fetch...' strings, but there is no structured error response, no categorization (retryable vs fatal), and no guidance for LLMs on what to do next. Per pattern:recovery-guide, errors should tell the agent what to try: 'Location not found. Use valid US coordinates (latitude -90 to 90, longitude -180 to 180)'.
No input validation rules documented. The latitude/longitude parameters accept any float, but NWS API requires valid ranges (-90 to 90 for lat, -180 to 180 for lon). Describe these constraints in parameter descriptions so LLMs understand valid input ranges.
State parameter in get_alerts is not constrained to a valid enum. LLMs will guess invalid state codes (e.g., 'California' instead of 'CA', or typos like 'XX'). Add an enum property listing all 50 US state abbreviations, or document a strict pattern in the description.