Loading ToolBench
MCP server that exposes public LatAm FX data and an auditable reconciliation engine to AI agents.
Six tools with complete schemas and descriptions. Naming follows verb_noun convention (list_, get_, convert, reconcile). Descriptions are clear and action-oriented (avg ~120 chars). All parameters have types and descriptions. Input schemas use proper JSON Schema with constraints (min/max, enums implicit via description). Error handling returns structured objects with 'error' field. Main gaps: no tool annotations (readOnlyHint/destructiveHint), output schemas not formally documented, no pagination for timeseries despite potentially large results, and reconcile tool is complex with underdocumented fuzzy matching semantics.
Convert an amount between currencies using a source's quote (e.g. USD<->ARS).
Get the latest buy/sell quote for an FX source (e.g. 'blue', 'oficial').
Summary statistics (min/max/mean/volatility) of a source's mid price.
Get the historical buy/sell series for a source (most recent ``last_n``).
List the supported FX sources (Argentine dollar variants).
Reconcile two ledgers with a multi-rule engine (exact ref, amount+date, fuzzy). Returns matched pairs (with the rule and a confidence score), unmatched ids on each side, and the overall match rate.
reconcile tool has complex nested schema (array of objects with optional fields) but fuzzy_threshold and enable_fuzzy semantics are underdocumented. What does fuzzy matching score mean? How are multiple rules ranked? LLM cannot reason about confidence thresholds without explicit guidance.
get_fx_timeseries returns potentially large result (up to 5000 observations) with no pagination or result limiting. Description says 'most recent last_n' but does not warn that returning 5000 items will exhaust context. No guidance on when to use smaller last_n.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All six tools are READ_ONLY per metadata, but this is not declared in tool definitions. LLM cannot infer safety properties from schema alone.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Output schemas not formally documented. Tools return dict[str, Any] with implicit structure (e.g., reconcile returns 'matched', 'unmatched_left', 'unmatched_right', 'match_rate'). LLM must infer field names and types from docstring prose.
Error handling returns {'error': str(exc)} for all failures. No error classification (retryable vs user-fixable vs fatal). LLM cannot determine whether to retry, ask user, or abort. E.g., 'Invalid source' should suggest calling list_fx_sources first.