Loading ToolBench
Open-source AI CFO for founders. React + FastAPI dashboard with Claude-powered copilot, KPI builder, Zoho Books MCP integration, scenario modelling, and audit ledger.
Four tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling guidance. All tools are READ_ONLY (good security posture), but descriptions lack actionable context for LLM selection. Parameter descriptions are present but generic. No documented output schemas, pagination support, or recovery guidance. Naming is clear (verb_noun pattern), but descriptions fall short of the 50-200 char LLM-optimized baseline.
Generate a financial report. Supports report types: summary, p&l, board_update, cash_flow. Returns dict with report sections and data.
Get current financial metrics for the user. Uses Supabase if credentials provided, otherwise falls back to SQLAlchemy. Returns: Dict with keys: mrr, arr, burn_rate, cash_balance, runway_months, gross_margin_pct, net_margin_pct, total_revenue, total_expenses
Get recent transactions, optionally filtered by category. Returns list of transaction dicts with id, date, description, amount, type, category, vendor, and status.
Run a what-if scenario on financial metrics. Projects forward based on adjustments to revenue and expenses over a specified number of months.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing what get_metrics, run_scenario, get_transactions, and generate_report return. Code shows return types in docstrings but no formal schema definitions visible in tool registration.
run_scenario accepts 'adjustments' as a free-form object with undocumented keys (revenue_increase, revenue_amount, expense_increase, expense_amount, months). No enum constraints, no validation rules, no description of which keys are required vs optional. LLMs will guess at valid keys.
get_metrics exposes Supabase credentials (supabase_url, supabase_key) as optional parameters. Credentials should never be tool parameters, they leak into logs and traces. Use server-side secret injection via environment variables.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
No error handling guidance. Tools lack recovery hints (e.g., 'If user_id not found, try search_users()'). No categorization of errors as retryable vs fatal. LLMs cannot self-correct on failures.
get_transactions accepts 'limit' with default 20 but no min/max bounds documented. No pagination support (offset, cursor, total_count). Large result sets will blow context window.