Loading ToolBench
The Beancount MCP Server is an experimental implementation that utilizes the Model Context Protocol (MCP) to enable AI assistants to query and analyze Beancount ledger files using Beancount Query Language (BQL) and the beanquery tool.
Beanquery MCP has 2 tools with basic definitions. Both tools have descriptions and input schemas present, but significant quality gaps limit production readiness. Naming is reasonable but not action-oriented. Descriptions lack depth about prerequisites, return values, and when to use each tool. Parameter descriptions are minimal. Output schemas are undocumented. Error handling is present but not recovery-focused. No tool annotations or structured output guidance. The resource implementations (get_tables, get_accounts) are internally dependent on run_query, creating composition risks.
Run a BQL query against the loaded Beancount file. The result will be returned as a string of bean-query's tabular text output.
Set the Beancount ledger file to use for queries.
No output schemas documented. Tools return strings with no structure guidance. LLMs cannot predict what fields or format to expect, forcing unstructured text parsing.
Descriptions lack WHEN/WHY context. 'Run a BQL query against the loaded Beancount file' does not explain when to use this vs set_ledger_file, what BQL is, or expected output format. LLMs cannot optimize tool selection.
Parameter descriptions are extremely sparse. 'filename' in set_ledger_file has only 'The path to the Beancount ledger file', no format, no examples, no validation hints. 'query' in run_query mentions BQL but provides no examples of valid queries, acceptable formats, or failure modes.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 47 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 51 | - | v1 |
run_query error handling is present (checks for missing ledger) but generic. Exception says 'No ledger file given or file could not be found. Use set_ledger_file to set a path.', this is recovery-focused, but does not categorize the error as user-fixable. LLM cannot distinguish between retryable and fatal conditions.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). set_ledger_file modifies state (sets a global ledger path) but is not marked destructive or write-intent. run_query is READ_ONLY but this hint is not encoded in the tool schema.
Resource implementations (get_tables, get_accounts) internally call run_query(). If run_query fails or requires a ledger to be set, these resources silently cascade the same constraint. No documentation of these dependencies.
No validation or constraint hints for parameters. 'filename' could accept anything; 'query' accepts any string. Parameter descriptions should specify formats (e.g. 'absolute path', 'valid BQL syntax').