Loading ToolBench
A Model Context Protocol server that provides real-time access to Beancount financial data. Enables querying transactions, running BQL queries, calculating metrics, and reading personal notes.
This Beancount MCP server has 11 tools with mostly well-structured definitions. Tool naming is consistently strong (verb_noun pattern: get_, list_, search_, add_, compare_). Descriptions are present for all tools and generally 150-250 characters, exceeding the 10-character minimum. However, schema quality is mixed: while JSON Schema type declarations are present for all parameters, some tools lack comprehensive parameter descriptions. The add_transaction tool stands out with excellent detail (postings array, double-entry validation, chronological ordering). Error handling is implicit rather than explicit, no recovery guidance in descriptions. Output schemas are largely undocumented (tools do not declare what fields they return). The server uses FastMCP (Python framework) with STDIO transport, which is a hard constraint limiting protocol readiness to 50 maximum. All tools are read-only except add_transaction (WRITE), and tool annotations are present. Composition is sound, tools chain well (search_transactions → add_transaction). Overall: solid tool definitions held back by STDIO transport, minimal error guidance, and undocumented output schemas.
Add a new transaction to your Beancount file with perfect syntax and chronological order. This tool creates properly formatted transactions with: - Correct indentation (2 spaces for postings) - Right-aligned amounts (at column 60) - Chronological insertion (finds correct date position) - Proper spacing between transactions - Validation of double-entry accounting (debits = credits)
Compare expenses or income between two periods. Useful for month-over-month or year-over-year analysis. Shows amounts for each period and calculates the difference and percentage change.
Get the current balance of a specific account. Returns the balance for an account, optionally filtered by currency. Works with any account type (Assets, Liabilities, Income, Expenses, Equity).
Generate an income statement (Profit & Loss) for a specific period. Shows total income, total expenses, and net income/loss.
Get expense breakdown by category for a specific month or entire year. Provides detailed breakdown of expenses by category, showing where money was spent.
Output schemas are not documented. Tools return data but descriptions do not specify what fields LLMs should expect (e.g., beancount_list_accounts returns 'hierarchical list' but structure is undefined). LLMs cannot plan downstream tool calls without knowing response shape.
Error handling provides no recovery guidance. Tool descriptions do not specify what errors are possible, when they occur, or what the LLM should do next (e.g., what if file_path does not exist? What if BQL query is invalid?). Descriptions like 'Execute a Beancount Query Language query' omit error scenarios.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Calculate current net worth (Assets - Liabilities). Net worth is the total value of all assets minus all liabilities, giving you a snapshot of your overall financial position.
Calculate savings rate for a specific year. Savings rate is calculated as (Income - Expenses) / Income * 100%.
List all accounts defined in your Beancount file. Returns a hierarchical list of all accounts (Assets, Liabilities, Income, Expenses, Equity) defined in your Beancount file.
Read personal notes and documentation about your financial setup. Access your personal notes, budget plans, financial goals, account documentation, and any other markdown files you've created to document your financial system.
Execute a Beancount Query Language (BQL) query against your financial data. BQL is SQL-like syntax for querying Beancount data. You can filter, aggregate, and analyze your financial transactions with powerful queries. Common query examples: - Monthly expenses: "SELECT year, month, sum(position) WHERE account ~ 'Expenses:' GROUP BY year, month" - Account balance: "SELECT sum(position) WHERE account = 'Assets:Bank:Checking'" - Tagged transactions: "SELECT date, narration, position WHERE 'vacation' IN tags"
Search for transactions matching specific criteria. Powerful search across all transactions with multiple filter options. Can filter by account, payee, narration text, tags, and date range.
Parameter descriptions vary in completeness. Some params lack actionable format/constraint details. E.g., beancount_compare_periods period1_month says '(1-12). If not provided, compares entire year', good. But beancount_read_personal_notes notes_dir says 'optional, uses BEANCOUNT_NOTES_DIR env variable' without clarifying what format is expected or what happens if directory does not exist.
beancount_run_query accepts free-form BQL strings with no syntax validation described. LLMs may pass invalid queries without guidance on format. No mention of BQL documentation, example error recovery, or constraints.
response_format parameter in beancount_run_query, beancount_get_monthly_expenses, beancount_search_transactions is defined with type string but no enum constraint. Parameter description says 'Output format' but does not list valid values ('markdown' vs 'json' implied by beancount_run_query but not explicit for others).
No idempotency guarantees documented. beancount_add_transaction is a WRITE tool that modifies a file. If an agent retries on ambiguous failure, will the transaction be added twice? No mention of idempotent behavior, deduplication, or confirmation steps.
Search and filter tools do not document pagination. beancount_search_transactions accepts 'limit' but no mention of offset, cursor, or total_count in response. Large result sets risk context window overflow.