Loading ToolBench
MCP server implementation for QuickBooks Online integration
18 tools with mixed quality. Naming is verb-based and clear (create_*). Descriptions present but generic, most are 40-80 chars, below the 194-char baseline for A+ tools. Critical gap: 6 tools (create_bill_payment, create_customer, create_employee, create_estimate, create_journal_entry, create_purchase) use z.any() for input schemas, exposing zero type safety. Remaining 12 tools have Zod schemas with field descriptions, but many lack parameter constraints (enums, min/max). Error handling returns text responses but no recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Output schemas undocumented, LLMs cannot predict response structure.
Create a bill in QuickBooks Online.
Create a chart‑of‑accounts entry in QuickBooks Online.
Create a new attachable (file attachment reference) in QuickBooks Online.
Create a bill payment in QuickBooks Online.
Create a new class in QuickBooks Online for categorizing transactions.
Create a credit memo (customer credit) in QuickBooks Online.
Create a customer in QuickBooks Online.
6 tools use z.any() for input schemas (create_bill_payment, create_customer, create_employee, create_estimate, create_journal_entry, create_purchase), providing zero type safety or LLM guidance on required fields.
Descriptions are generic and short (40-80 chars vs. 194-char baseline). Most lack WHEN to use, prerequisites, or recovery hints. E.g., 'Create a bill in QuickBooks Online' does not explain when to use vs. create_purchase or create_estimate.
No output schemas documented. LLMs cannot predict response structure (fields, types, pagination). Handlers return JSON.stringify(result) with no schema contract.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Create a new department in QuickBooks Online for location/department tracking.
Create a bank deposit in QuickBooks Online.
Create an employee in QuickBooks Online.
Create an estimate in QuickBooks Online.
Create an invoice in QuickBooks Online.
Create an item in QuickBooks Online.
Create a journal entry in QuickBooks Online.
Create a payment (receive money from customer) in QuickBooks Online.
Create a new payment method in QuickBooks Online (e.g., Cash, Check, Credit Card).
Create a purchase in QuickBooks Online.
Create a purchase order in QuickBooks Online.
Error handling returns plain text ('Error creating account: ...') with no recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 18 tools are WRITE operations but lack destructiveHint=true, preventing agents from reasoning about side effects.