Loading ToolBench
Import a bank CSV export, categorise it, summarise it per currency, find recurring charges and reconcile it against your expense ledger. Everything stays on your machine.
Bank Statement has well-structured tool definitions with complete input schemas and descriptions for all 8 tools. Naming follows verb_noun convention (bank_upload, bank_list, bank_categorize). Descriptions are substantive (100-200 chars typical), explaining WHAT the tool does and WHEN to use it. Parameters are typed with enums where appropriate (bank profile, group_by, action). However, output schemas are not documented, LLMs cannot see what fields to expect from responses. Error handling is present but generic (fail() returns plain text). No tool annotations (readOnlyHint/destructiveHint) despite clear READ_ONLY vs WRITE distinctions. Parameter descriptions could be more prescriptive about constraints (e.g., date format, max text length, free tier limits). Composition is sound, tools are single-responsibility and chainable.
List all accounts, or add a new one. An account is a named container for transactions from one bank export.
Assign a category to one or more transactions by ID, or bulk-categorize by matching a rule against the description and counterparty.
List all transactions, optionally filtered by date range, account, category, or uncategorized status. Free tier shows the last 12 months only.
Match bank transactions against an expense ledger (from the expense-tracker server). Shows unmatched bank transactions and unmatched expenses. Free tier allows up to 31 days of reconciliation.
Find recurring charges: transactions with the same counterparty, amount, and currency, spaced regularly. Free tier shows up to 5 recurring charges over the last 3 months.
List, add, or delete categorization rules. A rule matches a substring or regex pattern against the description and counterparty, and assigns a category. Free tier allows up to 5 rules.
Summarize transactions by category, month, account, or counterparty. Shows count, money in, money out, and net per group, per currency.
Output schemas not documented. LLMs cannot see what fields responses contain (e.g., does bank_list return transaction IDs, dates, amounts, categories?). This forces agents to guess field names for downstream operations.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). bank_list, bank_summarize, bank_recurring, bank_reconcile are clearly READ_ONLY; bank_upload, bank_categorize, bank_rules, bank_accounts are WRITE. Annotations help LLMs reason about safety and retry logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Import a bank CSV export. Detects the bank, parses dates, amounts and descriptions, deduplicates against prior imports, and stores everything. On the free tier, the last 12 months only.
Free tier limits (12 months, 5 rules, 31 days reconciliation, 5 recurring charges) are mentioned in descriptions but not enforced as parameter constraints. LLMs may request data outside these bounds without clear guidance on what to expect.
Error responses use generic fail() helper returning plain text. No error classification (retryable vs user-fixable vs fatal) or recovery guidance. E.g., 'Invalid regex' gives no hint to retry with a simpler pattern.
Parameter descriptions lack prescriptive constraints. E.g., 'match' in bank_categorize says 'substring or regex pattern' but does not state max length (MAX_MATCH_INPUT), safe regex rules (isSafeRegexSource), or what happens if regex is invalid.