Loading ToolBench
An MCP server that treats a large tool surface as a governance problem: tool-search discovery, budgets, human-in-the-loop confirmation, scoping, PII redaction and audit logging.
Strong foundation with clear naming conventions (verb_noun pattern), comprehensive parameter schemas using Zod, and explicit risk classifications. Tool descriptions are present and actionable. However, output schemas are not formally documented in the visible code, and several tools lack parameter descriptions. The search_tools and confirm_action pattern is innovative for governance but adds cognitive load. Tenant isolation is well-designed (defence in depth). Most tools follow single-responsibility principle.
Issue a single-use, time-limited confirmation token for a mutating tool. The token is bound to the exact arguments supplied here: they must match the later invoke_tool call.
Return the full description and JSON Schema for one tool discovered via search_tools.
Filters the catalogue by category name.
Exact-match SKU lookup.
Returns a single product record or null.
Paginated product catalogue for the tenant.
Surfaces products needing replenishment.
Output schemas not formally documented. While Zod input schemas are comprehensive, return types are inferred from handler implementations rather than explicitly declared in tool definitions. LLMs cannot plan downstream calls without knowing what fields to expect.
Mutating tools (orders.cancel_order, orders.refund_order) require a confirmation_token from confirm_action, but the error handling for missing/invalid tokens is not visible in the source. LLMs need clear guidance on what happens if they forget the token or provide an expired one.
invoke_tool description lacks detail on error cases. Does it return a structured error? Can the LLM retry? Is the failure retryable or user-fixable? Current description ('Run a domain tool by name...') does not guide recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Returns products that are completely unavailable.
Run a domain tool by name. Arguments are validated against the tool schema. Mutating tools additionally require confirmation_token.
Returns every order belonging to the given customer ID.
Sets an order status to cancelled. Cannot be undone through this API.
Aggregates the tenant order book into a status -> count map.
Returns the full order record including line items, or null when not found.
Returns the top N orders ranked by total value descending.
Expands an order into its constituent product lines.
Returns a paginated list of orders scoped to the calling tenant.
Returns operator notes. These are free text and may contain customer contact details, which the PII guardrail redacts.
Marks an order refunded and records the refunded amount.
Filters orders by status (pending, paid, shipped, cancelled, refunded).
Returns the gross value of the tenant order book.
Find domain tools by keyword. Returns name, domain, one-line summary and whether the tool mutates state. Use this instead of expecting every tool to be listed up front.
Empty input schemas (orders.count_by_status, orders.total_value, inventory.out_of_stock) lack descriptions explaining why no parameters are needed. LLMs may assume these are discovery tools or misunderstand their scope.
search_tools returns tool metadata but does not document the exact response schema (fields like 'name', 'domain', 'summary', 'mutating'). LLMs cannot reliably extract or chain results without knowing the structure.