Loading ToolBench
Model Context Protocol server for QuickBooks Online integration with OAuth authentication and OIDC user management
Scoring was not performed
Payload parameters lack structure documentation. qbo_create_entity, qbo_update_entity, qbo_operate_entity all accept a 'payload' object with description 'Entity data payload' or 'Operation payload', LLMs cannot infer required fields, nested structure, or valid values.
No output schemas documented. Tools return data but LLMs do not know what fields to expect, preventing downstream tool chaining and forcing exploratory calls. For example, after qbo_get_entity, does response include 'updated_at', 'SyncToken', or other metadata needed for qbo_update_entity?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 13 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
qbo_operate_entity uses free-form 'operation' parameter instead of enum. Valid operations (void, unsync, send, etc.) are not declared, inviting LLM hallucination of invalid operations and failed calls.
No error recovery guidance. Tools lack descriptions of failure modes, retryability, and recovery steps. E.g., qbo_create_entity may fail if customer does not exist, should LLM call qbo_search_entity first? Or qbo_create_entity to create the missing customer?
No tool annotations (readonly, destructive) in protocol layer. Read-only tools (qbo_query_company, qbo_get_entity, qbo_search_entity) and write tools (qbo_create_entity, qbo_update_entity, qbo_operate_entity, qbo_send_transaction) should be annotated to guide agent planning and confirmation flows.
Parameter descriptions are incomplete. Many parameters lack guidance on valid values, ranges, or formats. E.g., 'sandbox' boolean does not explain what environment it selects, or whether it defaults to production or sandbox.
Missing guidance on entity types. qbo_create_entity, qbo_get_entity, qbo_update_entity, qbo_search_entity accept an 'entity' parameter but do not list which entity types are supported. Are User, Vendor, Term supported? Only Invoice, Customer, Estimate?
qbo_operate_entity naming is vague. 'operate' does not convey intent, LLMs may confuse it with other operation tools. Better names: qbo_void_entity, qbo_unsync_entity, or qbo_execute_operation with operation as enum.
IQL/SQL syntax not explained. qbo_query_company and qbo_query_all_companies accept 'sql' parameter (IQL, Intuit Query Language) but provide no examples, syntax guide, or link to documentation. LLMs will guess syntax and fail.