Loading ToolBench
MCP server to interact with the Mercury API for banking operations
This server has visible tool definitions with reasonable naming and present descriptions, but several critical gaps prevent a higher score. All 9 tools follow verb_noun naming conventions (list_*, get_*), which is good. However, descriptions are minimal (15-50 chars), undershooting the 194-char baseline. Parameter descriptions exist but are sparse, most lack format constraints, ranges, or actionable guidance. Output schemas are not documented anywhere in the visible code. No error handling guidance, recovery hints, or field mappings shown. The tool definitions appear hardcoded in FastMCP decorators, so schemas ARE present in the framework registration, but the source code excerpt does not show them explicitly, this caps confidence. The server correctly avoids exposing secrets as parameters (API key injected via env), but no tool mentions what permissions are required (read:financial_data, etc.). Composition is sound, each tool does one thing, but tools like get_account_transactions could benefit from clearer pagination guidance.
Download a statement PDF
Get a specific Mercury bank account
Get cards for a specific account
Get transactions for a specific account
Get details for a specific recipient
Get statements for a specific account
Get details for a specific transaction
Descriptions are too short (15-50 chars) and lack context. Baseline is 194 chars. LLMs cannot determine when to select each tool vs. similar alternatives (e.g., when to call get_account vs. list_accounts). Descriptions should state WHAT the tool does, WHEN to use it, and WHAT it returns.
Parameter descriptions are generic and lack actionable constraints. Examples: 'The ID of the account' does not explain format (UUID? numeric? string prefix?), length, or case sensitivity. 'Maximum number of transactions' lacks range (1-10000? default 500 is a good start but needs bounds). Cannot see full schema in excerpt, assume they exist in FastMCP registration, but they must state format, pattern, minimum/maximum, and enum values where applicable.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
List all Mercury bank accounts
List all recipients
Output schemas are not documented in visible code. Tools return JSON (response.json()) but no documentation of structure, fields, types, or nested objects. LLMs need to know: does list_accounts return [{ id, name, balance, ... }] or { accounts: [...], total, nextCursor }? Without this, downstream tool calls fail because the LLM cannot map response fields to next tool's input.
No error handling guidance. If an account_id is invalid, does the tool return 404, 400, or 401? What should an LLM do? Should it retry, ask the user, or try a different lookup? No recovery hints, no suggestions for user-fixable vs. fatal errors. Example: 'Account not found. Try list_accounts() to see available accounts.' would be helpful but is missing.
Pagination is undocumented. get_account_transactions accepts limit and offset, but the response structure is unclear. Does it return { transactions: [...], total: 1500, hasMore: true }? Or { transactions: [...] }? Without knowing the total and whether more results exist, the LLM cannot implement pagination correctly. Common baseline: return total count and next_cursor for better UX.
No permission declarations. Tools do not state what scopes/permissions they require (e.g., read:accounts, read:transactions). This makes it impossible for an LLM framework to validate least-privilege access or explain to users what data exposure they are authorizing.
Response data not stripped. Code shows response.json() returned as-is from Mercury API (see MercuryClient methods). APIs often include verbose metadata, pagination internals, audit fields, etc. These waste tokens in the LLM context and dilute signal. Example: if Mercury returns { accounts: [...], meta: { version, timestamp, ... }, audit: { ... } }, strip to just { accounts: [...], total: X }.
Result limits not enforced. get_account_transactions accepts limit up to 500 default, but code does not cap or document an absolute maximum. If an LLM passes limit=10000, the tool should reject or cap it with a clear message. Current code just passes it to the API, risking timeouts, memory exhaustion, or context window overflow.