Loading ToolBench
Server has 5 tools with basic MCP registration via FastMCP. Tool names follow verb_noun pattern (beancount_query, beancount_get_transaction, beancount_accounts, beancount_submit_transaction, beancount_replace_transaction). However, descriptions are inconsistent in quality and length. Input schemas are present but minimal, most parameters lack detailed type information or constraints. Output schemas are not documented anywhere in the visible source code. Error handling is present but generic. The incomplete transaction JSON shown in source suggests schema issues. Overall, this is a functional but below-average MCP server for definition quality.
Get all accounts from the ledger, and return it as a JSON string.
Get transaction details by ID, and return it as a JSON string. If transaction not found, return an error message.
Execute a BQL (Beancount Query Language) query against the ledger, and return it as a JSON string. You can call tool `beancount_accounts` to list accounts from the ledger if needed. Example: `SELECT sum(position), account WHERE date>=2024-01-01 GROUP BY account`
Replace an existing transaction by ID with new transaction data.
Submit a beancount transaction to the ledger. Please make sure the account exists in the ledger, and the transaction date appropriate. You SHOULD call tool `beancount_accounts` to list accounts before submitting.
Output schemas not documented. Tool descriptions mention 'return it as a JSON string' but the actual structure (field names, types, cardinality) is not documented anywhere. LLMs cannot plan downstream calls or extract fields when the response structure is opaque.
Input parameter 'query' in beancount_query has no format constraint, pattern, or example of valid BQL syntax. Description mentions 'Example: SELECT sum(position)...' but the rubric explicitly forbids example values in descriptions, they should be replaced with formal constraints or a separate examples field.
beancount_submit_transaction description is truncated/incomplete: 'You SHOULD call tool `beancount_accounts` to list', sentence ends mid-thought. This fails the 'complete description' requirement.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 11 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 46 | - | v1 |
beancount_accounts tool has a 1-sentence, 66-character description ('Get all accounts from the ledger, and return it as a JSON string.'). This is below the 10 - 1024 character recommendation and lacks context on when to call it, what structure to expect, or how to interpret the output.
Error handling in visible code (ValueError raises) provides only a message string. No error classification (retryable vs user-fixable), no recovery guidance, and no structured MCP error codes. Per pattern:recovery-guide, errors should tell the LLM what to do next.
No input validation visible for user-provided transaction strings or query strings. Agents could pass malformed Beancount syntax, leading to parse errors. Per pattern:tool-gateway, all agent input must be treated as untrusted and validated early with actionable error messages.
beancount_get_transaction and beancount_replace_transaction both use 'tx_id' as a parameter, but the ID format is opaque (hash_entry() output). No documentation on how to obtain a valid tx_id, forcing agents to discover this via trial-and-error or reading source code.
submit_transaction and replace_transaction are write operations, but their descriptions do not explicitly state 'This modifies the ledger' or 'This operation is irreversible'. Per pattern:command-tool, state-modifying tools must clearly declare side effects.