Loading ToolBench
MCP server implementation for QuickBooks Online integration
Server has 10 tools with descriptions and input schemas present. Naming follows verb_noun pattern (create_*) consistently. However, schemas are incomplete: most tools accept generic 'object' types with minimal field-level documentation. Parameter descriptions in schemas are sparse or missing. Output schemas are defined but inconsistently applied. Error handling exists but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint/idempotentHint). Descriptions are verbose (500+ chars) with examples, which wastes tokens and risks LLM literal reuse of example values. Per-tool scores range 45 - 65; average ~58.
Create a bill (accounts payable) in QuickBooks Online to record money you owe to vendors for goods or services received.
Create a new Chart of Accounts entry in QuickBooks Online. Accounts are used to categorize and track financial transactions across all areas of the business.
Create a bill payment in QuickBooks Online to record payments made to vendors for outstanding bills. This tool creates a BillPayment entity that reduces bill balances and tracks payment details.
Create a new customer record in QuickBooks Online with contact information, billing details, and payment terms.
Create a new employee record in QuickBooks Online. This is essential for payroll processing, time tracking, and expense management.
Create a sales estimate (quote) in QuickBooks Online to provide pricing to potential customers. Estimates can later be converted to invoices when accepted.
Generic object schemas without field-level documentation. Tools like create_bill_payment, create_bill, create_customer accept 'object' type with only a brief description. LLMs cannot infer required fields, nested structure, or valid values.
Descriptions are 500 - 1000+ characters with embedded examples (e.g., 'e.g., Office Supplies Expense'). LLMs tend to reuse example values literally. Descriptions should be 50 - 200 chars; examples belong in enums or format constraints.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 10 tools are WRITE operations but lack destructiveHint annotation. Agents cannot distinguish safe from risky tools without reading descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2025-06-18+ | v2 |
Create an invoice (accounts receivable) in QuickBooks Online to bill customers for products or services provided.
Create a product or service item in QuickBooks Online that can be used on invoices, estimates, and sales receipts.
Create a journal entry in QuickBooks Online for manual accounting adjustments, accruals, deferrals, and other double-entry bookkeeping transactions. Journal entries must always balance (debits = credits).
Create a purchase transaction in QuickBooks Online to record expenses, checks, credit card charges, or cash purchases. Purchases track money going out of your business.
Error handling returns isError flag and error string but does not guide recovery. No categorization (retryable vs user-fixable vs fatal). Error messages like 'Unknown error occurred' provide no actionable next step for the agent.
Output schemas defined but inconsistently applied. create_account has outputSchema; others do not. Handlers return structuredContent conditionally. Inconsistency breaks agent parsing and planning.