Loading ToolBench
An MCP server that interacts with all QuickBooks public APIs.
This server has two explicitly defined tools with basic parameter schemas, but suffers from critical gaps in descriptions, parameter documentation, and output schema definition. Both tools lack LLM-optimized descriptions and parameter annotations. The dynamically generated API tools (via register_all_apis()) are not directly visible in source and therefore capped at 50. Tool names follow verb-noun convention (get_, query_) but descriptions are incomplete and descriptions for parameters are entirely absent. No structured output schema is documented. Error handling exists but does not guide LLM recovery. The server is STDIO-only, which is a hard transport constraint.
Fetches the schema for a given QuickBooks entity (e.g., 'Bill', 'Customer'). Use this tool to understand the available fields for an entity before constructing a query with the `query_quickbooks` tool.
Executes a SQL-like query on a QuickBooks entity. **IMPORTANT**: Before using this tool, you MUST first use the `get_quickbooks_entity_schema` tool to get the schema for the entity you want to query (e.g., 'Bill', 'Customer'). This will show you the available fields to use in your query's `select` and `where` clauses.
Parameter descriptions completely missing. The 'entity_name' parameter in get_quickbooks_entity_schema has no description explaining what entity names are valid, what format they take, or how to discover them. The 'query' parameter in query_quickbooks is similarly undocumented, LLMs cannot determine what SQL-like syntax is valid without a description.
No output schema documented for either tool. Both tools return types.TextContent, but the structure of the text (JSON, plain text, error format) is never documented. LLMs cannot plan downstream calls or extract fields without knowing the response schema.
Tool descriptions lack actionable context. 'Fetches the schema for a given QuickBooks entity' is 53 characters, below the 10-1024 char range baseline. Descriptions do not explain WHEN to use each tool, what prerequisites exist, or how they compose together. The query_quickbooks description mentions 'MUST first use get_quickbooks_entity_schema' but this is procedural guidance, not a constraint documented in parameter relationships.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 41 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Dynamically generated API tools via register_all_apis() are not directly visible in source code. Tool names, schemas, and descriptions are constructed via string interpolation in exec(). This makes verification impossible and means tools are effectively inferred. Per HARD SCORING RULES, these tools must be capped at 50 overall.
No parameter type constraints or validation. The 'query' parameter accepts any string, there is no documented SQL-like grammar, no examples, no format constraint. This invites hallucinated queries that will fail at runtime. The 'entity_name' parameter similarly lacks an enum of valid QuickBooks entities.
Error responses do not guide LLM recovery. When 'Error: Schema not found for entity' is returned, the response includes 'Available entities: [...]' which is helpful, but the format is free-text. No structured error category (retryable vs. user-fixable vs. fatal) is defined. Error messages from query_quickbooks exceptions are raw Python tracebacks, not LLM-actionable guidance.
QuickBooks credentials are loaded from environment variables and passed to QuickBooksSession(), but there is no documentation of which environment variables are required (.env config is assumed but not documented in tool descriptions). If initialization fails, error messages are printed to stderr only; the tools do not gracefully surface this to the LLM.
No pagination or result limiting documented. If query_quickbooks returns a large result set, the response is returned as a single string with no offset, limit, or next_cursor mechanism. This violates the pattern for handling large results that could overflow context windows.