Loading ToolBench
MCP server for analyzing bank statements. Loads CSV and PDF bank statements, extracts transactions, categorizes them using AI, and provides natural language querying with hybrid retrieval (BM25 + semantic search).
StatementScope has 5 tools with complete input schemas and descriptions, but quality is uneven. Tool names follow verb_noun convention (load_statement, query_transactions, categorize_transactions, get_spending_summary, clear_statements). Descriptions are present and moderately detailed (100-250 chars), explaining WHAT and WHEN. However, parameter descriptions are sparse or missing context. Output schemas are not documented, LLMs cannot predict return structure. Error handling is minimal (mostly string returns with error prefixes). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. The categorize_transactions tool has no input parameters but is marked WRITE, creating ambiguity about idempotency. Overall: functional but below production baseline.
Automatically categorize all uncategorized transactions using AI. Assigns categories: Food, Shopping, Transport, Bills, Entertainment, Health, Travel, Income, Transfer, Other. Safe to call multiple times — skips transactions that are already categorized.
Remove all loaded statements and reset to a clean state. Clears all transactions from memory and the local database.
Get total spending figures, category breakdowns, and top merchants. Use this for questions like 'What's my total spending?', 'How much did I spend overall?', 'Give me a breakdown by category'. Period can be 'all', a month like 'January 2025', or 'last 30 days'.
Load a bank statement file (CSV or PDF) for analysis. Parses the file, extracts transactions, and indexes them for querying. Supports Chase and Bank of America CSV exports; other CSVs use a generic parser. PDF extraction and query answering send data to the Anthropic API. Provide the full absolute file path.
Answer a natural language question about specific transactions, merchants, or categories. Use this for questions like 'Show me Amazon purchases', 'What subscriptions am I paying for?', 'Did I go to Starbucks?', 'Show me refunds'. Do NOT use this for simple totals or summaries — use get_spending_summary instead.
Output schemas not documented. LLMs cannot predict return structure (string vs object vs array). Breaks downstream tool chaining and forces parsing of unstructured text.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). clear_statements is destructive but unmarked; categorize_transactions is idempotent but unmarked. Agents cannot reason about safety or retry behavior.
categorize_transactions has no input parameters but is marked WRITE. Unclear if calling it twice is safe (idempotent) or causes duplicate work. Description says 'safe to call multiple times' but no schema enforcement.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Error handling returns bare strings ('Error: ...', 'No statements loaded'). No recovery guidance. LLMs cannot distinguish retryable vs user-fixable vs fatal errors.
get_spending_summary period parameter lacks enum constraint. Description says 'all', 'January 2025', 'last 30 days' but no formal enum. LLMs may hallucinate invalid values like 'Q1 2025' or 'past week'.