Loading ToolBench
41 tools with consistent naming (verb_noun pattern) and basic descriptions. However, critical gaps: (1) Most parameter descriptions are minimal (10-30 chars), violating the 50-200 char LLM-optimized baseline. (2) No output schemas documented, LLMs cannot infer what fields to expect from responses. (3) JSON-string parameters (accounts_create, transactions_create, etc.) force LLMs to construct raw JSON, error-prone and violates chat-data-model pattern. (4) No error recovery guidance, tools return raw API errors without actionable next steps. (5) Delta sync parameters (last_knowledge) underdocumented. (6) api_raw_request is a dangerous escape hatch that bypasses tool composition. Naming is solid (get_, list_, create_, update_, delete_ verbs), but parameter and output documentation is well below production baseline.
Create a new account
Get a single account by ID
List all accounts in a plan
Make a raw API request (any method, any path)
Set the budgeted amount for a category in a specific month
Get a single category by ID
List all categories grouped by category group
Get a category for a specific month
JSON-string parameters force LLMs to construct raw JSON payloads (accounts_create, transactions_create, transactions_update, categories_update, payees_update). This violates chat-data-model pattern and is error-prone. LLMs frequently produce malformed JSON.
No output schemas documented. LLMs cannot infer response structure, field names, or types. This prevents proper chaining (e.g., get_user returns what fields?) and forces agents to guess.
Parameter descriptions are minimal (10-30 chars), well below the 50-200 char LLM-optimized baseline. E.g., 'Plan ID (uses --plan-id if not specified)' lacks context on format, constraints, or when to use. Descriptions should explain WHAT, WHEN, and WHY.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Update a category (name, note, etc.)
List money movements for a specific month
List money movement groups
List money movement groups for a specific month
List all money movements
Get a single month by date
List all months
List payee locations for a specific payee
Get a single payee location by ID
List all payee locations
Get a single payee by ID
List all payees
Update a payee
Get a single plan by ID
List all plans (budgets)
Set the default plan ID (saved to config)
Get plan settings
Delete a scheduled transaction by ID
Get a single scheduled transaction by ID
List all scheduled/recurring transactions
List transactions for a specific account
List transactions for a specific category
List transactions for a specific month
List transactions for a specific payee
Create a transaction
Delete a transaction by ID
Get a single transaction by ID
Import transactions (bank-style deduplication)
List transactions (supports delta sync)
Search transactions by memo or payee name (client-side filtering)
Update a transaction by ID
Update multiple transactions at once
Get the authenticated user
Delta sync parameter 'last_knowledge' is underdocumented. No explanation of what it is, how to obtain it from previous responses, or when to use it. LLMs will not know to pass it.
api_raw_request is a dangerous escape hatch that bypasses tool composition and safety. It accepts arbitrary HTTP methods and paths, allowing agents to call any API endpoint. This violates tool-gateway pattern and should be removed or heavily gated.