Loading ToolBench
A Model Context Protocol (MCP) server for real-time and historical foreign exchange rates
The server implements 5 currency conversion tools with functional specifications. Tool naming follows verb_noun conventions (available_currencies, convert_currency, today_rates, historical_rates, time_series_rates), which is appropriate. All tools have descriptions ranging 50 - 142 characters. Input schemas are present for all tools with typed parameters and descriptions. However, descriptions are minimalist and lack context for when/why to use each tool. Output schemas are not formally documented, LLMs must infer return structure from the API response, which is a significant gap for composition and chaining. Error handling is basic (generic exception re-raises with logging) and provides no guidance to LLMs on recovery steps. No parameter constraints (enums) or validation rules are documented despite several parameters accepting only specific formats (e.g., date as YYYY-MM-DD, currency codes as ISO 4217). The tools are read-only (good for safety) but lack the descriptive depth and structure expected of A-grade tools.
Provides the list of available currencies from frankfurter dev API as of today
Converts the currency from one code to another code
Retrieves exchange rates for a specific date
Retrieves exchange rates over a time period
Retrieves the current exchange rates for the given currency code
Output schemas not documented. Tool return types (dict) are unspecified, LLMs cannot infer which fields to expect, breaking tool composition and downstream chaining. For example, if convert_currency returns 'converted_amount', but a hypothetical send_rate_alert tool expects 'result.rate', the mismatch forces extra discovery calls.
Descriptions lack composition context. Each description states WHAT the tool does but not WHEN to use it vs. related tools (e.g., when is convert_currency better than today_rates + manual math?). LLMs often select wrong tools when distinctions are unclear.
Parameter constraints not enforced or documented. 'date' and 'start_date'/'end_date' parameters accept YYYY-MM-DD format but this is only mentioned in descriptions, not as regex patterns or formal constraints. LLMs frequently pass malformed dates (e.g., '2024-1-15' or '01/15/2024'), causing silent API errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Currency codes not declared as enums. Parameters like 'from_code', 'to_code', 'code', 'base', and 'symbols' accept ISO 4217 currency codes or comma-separated lists, but no enum or pattern constraints are present. LLMs may pass invalid codes (e.g., 'US' instead of 'USD'), generating API errors with no recovery guidance.
Error handling is generic. All tools catch exceptions and re-raise, logging only the error message. No categorization (retryable vs. user-fixable vs. fatal), no invalid value echoing, no recovery guidance. If an LLM passes 'INVALID' as a currency, the error is a bare HTTP 400 or network error, the LLM cannot self-correct.
Default parameters may cause confusion. 'historical_rates' and 'time_series_rates' default base to 'EUR' without explanation. If an LLM forgets to specify base, it silently gets EUR rates instead of failing or asking the user, silent failures are harder to debug than explicit errors.