Loading ToolBench
Bidirectional conversion service between ANP (Agent Negotiation Protocol) and MCP (Model Context Protocol), with integrated weather server capabilities
The server defines 4 weather tools (2 TypeScript, 2 Python duplicates) with basic schema definitions and descriptions. However, there are critical issues: (1) The Python implementation tools (get_alerts, get_forecast) are not directly visible in the provided source code, only inferred from the file listing. (2) Parameter descriptions are minimal and lack actionable detail. (3) No output schemas are documented despite returning structured text responses. (4) Error responses are generic text strings with no recovery guidance. (5) Duplicate tool definitions across implementations create confusion. (6) No tool annotations (readOnlyHint, idempotentHint, etc.) despite all tools being read-only. The TypeScript tools have slightly better Zod schemas with min/max constraints on numeric parameters, but descriptions still lack the 'WHEN to use' and 'WHAT to expect' guidance required for LLM selection clarity.
Get weather alerts for a state
Get weather forecast for a location
Get weather alerts for a US state.
Get weather forecast for a location.
Python tools (get_alerts, get_forecast) have no visible schema definitions in source code, only file references provided. Cannot verify input/output structure or parameter validation.
Duplicate tool definitions across TypeScript and Python implementations (get-alerts vs get_alerts, get-forecast vs get_forecast). LLMs will experience ambiguity or tool collision if both are exposed simultaneously.
Tool descriptions are extremely brief (<40 chars on Python tools, <50 on TypeScript). Lack 'WHEN to use' guidance and prerequisites. E.g., get-alerts description 'Get weather alerts for a state' does not explain that it requires a valid US state code or that results may be empty.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
No output schemas documented. Tools return text-only responses with no structured field definitions. LLMs cannot extract data programmatically or plan downstream tool chains. E.g., get-forecast returns free-text forecast but should return structured periods array with typed fields (name, temperature, windSpeed, etc.).
Error responses are generic strings ('Failed to retrieve alerts data', 'No active alerts for <state>'). No recovery guidance. LLMs cannot distinguish retryable errors from user-fixable ones (e.g., invalid state code) or determine next steps.
No tool annotations despite all tools being read-only. Missing readOnlyHint tag would clarify to agents that these tools are safe to call without permission checks or confirmation steps.
State parameter on get-alerts has no enum constraint. Description says 'Two-letter state code (e.g. CA, NY)' but LLMs may pass invalid codes (e.g., 'California', 'ca', '12'). Should use enum: ['AL', 'AK', 'AZ', ...]
Parameter descriptions lack actionable format guidance. E.g., latitude/longitude descriptions state min/max in schema but do not explain NWS API coverage (US only) or coordinate precision expectations in the text description.