Loading ToolBench
Currency Converter MCP server
Single tool with basic schema and description. Tool is registered explicitly with Zod schema and description text. Parameters have type definitions and individual descriptions. However, the descriptions are minimal (under 100 chars for the main tool), lack context about when to use the tool, lack error recovery guidance, and the tool lacks idempotency and error handling sophistication. The schema is present and valid, but output is unstructured (generic text response). No parameter validation, no enum constraints despite domain allowing them (currency codes could be constrained), and error handling is generic. This server is functional but below production baseline for LLM agent integration.
Convert Amount from one currency to another currency
Tool description lacks actionable context: does not state WHEN to use this tool, does not mention prerequisites (valid currency codes), and does not describe the output structure or format. Description is 54 chars, meeting minimum threshold but providing minimal LLM guidance.
Currency code parameters (from, to) accept free-form strings with no enum constraints, despite currency codes being from a fixed, well-known domain (ISO 4217). This invites hallucinated invalid codes like 'XYZ' or 'DOLLAR'. Should declare valid codes as enum or validate strictly with actionable error messages.
Error handling is generic and non-actionable. Returns 'Error: ' + error.message, which could be a network timeout, invalid currency code, malformed API response, or API outage. LLM cannot determine if it should retry, suggest a different currency, or report failure to user. No error classification or recovery guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Output schema is undocumented and unstructured. Tool returns generic text content with no declared fields. LLMs cannot extract structured data (amount, from_currency, to_currency, rate, converted_amount) for downstream use. Should return a typed object with conversion details.
No input validation for the amount parameter. Accepts negative amounts, zero, NaN, Infinity, and enormous numbers without validation or bounds. Should specify min/max (e.g., amount > 0, amount <= 1e12) and validate with clear error messages.
No timeout, retry logic, or failure modes documented for external API call to open.er-api.com. If API is slow or down, agent waits indefinitely. Should set explicit timeout and return retryable error classification.
Tool is idempotent (read-only, no side effects), but this is not declared in tool registration. Current MCP spec (2026-07-28) supports tool annotations (idempotentHint) to signal this to agents. Should include idempotent: true in tool metadata.