Loading ToolBench
A weather MCP server that provides weather forecasts and alerts using the National Weather Service API
Server has 4 tools with significant quality gaps. Two tools (get-alerts, get-forecast in index.ts) have proper Zod schemas with constraints (latitude -90..90, longitude -180..180, state length=2), but descriptions are minimal (10-40 chars). Two duplicate get_forecast tools in low-level-server.ts lack coordinate constraints entirely. No parameter descriptions beyond the schema. Output schemas are undocumented, LLMs cannot predict response structure. Error handling returns plain text without recovery guidance. No tool annotations (readOnlyHint, etc.). Naming is verb-noun compliant but lacks clarity on when to use get-alerts vs get-forecast.
Get weather alerts for a state
Get weather forecast for a location
Get weather forecast for a location
Get weather forecast for a location
Duplicate tool definitions: get_forecast appears twice in low-level-server.ts with identical schemas. This creates ambiguity and wastes agent reasoning cycles. Consolidate into a single canonical tool.
Input parameter descriptions missing. Latitude/longitude in low-level-server.ts lack descriptions; state in get-alerts has no description. LLMs cannot infer parameter meaning from names alone (e.g., is 'state' a US state code or a status field?).
Tool descriptions are too short (10-40 chars). 'Get weather alerts for a state' and 'Get weather forecast for a location' lack context on when to use each tool, what data is returned, or prerequisites. Baseline is 194 chars; these are 5x too brief.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | <=2025-11-25 | v2 |
Output schemas undocumented. Callers cannot predict response structure (e.g., does forecast return an array of periods or a single object?). LLMs must guess, risking parsing errors and wasted tokens.
Error handling lacks recovery guidance. Errors return plain text ('Failed to retrieve alerts data') without suggesting next steps. Per pattern:recovery-guide, errors should tell LLMs what to do: 'Failed to retrieve alerts. Verify state code is 2 uppercase letters (e.g., CA, NY).'