Loading ToolBench
MCP server providing read-only access to Quicken For Mac financial data via Claude
Well-structured MCP server with 8 read-only tools accessing Quicken financial data. All tools have descriptions, schemas, and proper type definitions. Tool names follow verb_noun pattern (list_, query_, search_). Schemas use Zod validation with clear parameter types. Main gaps: (1) output schemas not documented in descriptions; (2) no tool annotations (readOnlyHint, idempotentHint); (3) parameter descriptions lack constraint detail (format specs, enums, ranges); (4) no error categorization guidance or recovery instructions; (5) raw_query tool lacks safety barriers despite being powerful. Error handling is present but generic, does not guide LLM toward recovery steps.
List all accounts in the Quicken database, optionally filtered by account type.
List all spending categories in the Quicken database, with their hierarchy and type.
List investment holdings in the Quicken database, including security names, tickers, and current values.
Query transactions from Quicken, optionally filtered by date, amount, payee, and category.
Execute a raw SQL query against the Quicken database. Use only when other tools cannot answer the question.
Search for payees by name substring to find exact payee names for filtering.
Analyze spending by category over a date range, with optional account type filtering and grouping.
Analyze spending over time, grouped by day, week, month, or quarter.
No output schemas documented. Tool descriptions explain input parameters but not the JSON structure returned. LLMs cannot plan downstream field extraction or know what keys to expect (e.g., from query_transactions, do we get 'date' or 'transaction_date'? 'payee_name' or 'payee'?). Missing output documentation increases hallucination risk.
Parameter descriptions lack actionable constraints. 'period' in spending_over_time says 'Time period grouping: day, week, month, or quarter' but does not enforce enum in schema; 'group_by' in spending_by_category has same issue. Free-form strings invite hallucinated values ('year', 'daily', 'biweekly'). Should declare enums in schema and reference them in description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
raw_query tool provides direct SQL access with 'Use only when other tools cannot answer the question' guidance, but no input validation, query timeout, result size limit, or explicit error classification. Malicious or runaway queries could hang the server. Needs dry-run confirmation, max row limit (e.g., 1000), and explicit timeout (e.g., 30s).
No tool annotations (readOnlyHint, idempotentHint). All 8 tools are read-only and idempotent, but this is not signaled in the schema. LLMs cannot infer safety from tool names alone. Should add tool annotations via the MCP SDK's tool definition metadata.
Error handling returns helpful hints (e.g., 'Open Quicken with open -a Quicken') but does not categorize errors as retryable vs. user-fixable vs. fatal. LLMs cannot reliably decide: should I retry? Ask the user? Or give up? Error responses should include error_type: 'retryable'|'user_action_required'|'fatal'.
Date parameters accept 'optional' ISO 8601 strings but no validation of format. Descriptions say 'YYYY-MM-DD' but do not warn against '2024/01/15' or '01-15-2024'. Should validate input or document the exact regex pattern.
Numeric parameters (min_amount, max_amount) lack range constraints. No documented min/max values. Could LLM pass negative amounts? Amounts > 1 billion? Should specify realistic bounds (e.g., -999999.99 to 999999.99).