Loading ToolBench
MCP server that connects Claude to a Shopify store: products, orders, customers, inventory, and sales analytics. Ships with a full demo store so it works out of the box.
Strong tool definitions with clear naming, comprehensive descriptions (avg 180 chars), and well-structured Zod schemas. All 8 tools have explicit registrations with inputSchema, descriptions, and type constraints. Error handling is present via BackendError and toolError(). Key gaps: no output schemas documented, no tool annotations (readOnlyHint/destructiveHint), and parameter descriptions could be more prescriptive about formats/constraints. Composition is clean, each tool has one responsibility. Naming follows verb_noun pattern consistently.
Fetch a customer profile by email address or customer id: contact details, lifetime order count and spend, plus a summary of their most recent orders. Provide exactly one of email/id.
Fetch one order in full: line items, totals breakdown, payment status, fulfillment status and shipment tracking numbers. Look up by internal order id or by the customer-facing order number (e.g. '1042' or '#1042'). Provide exactly one of id/order_number.
Fetch one product with full details: description, tags, and every variant with SKU, price and live inventory. Look up by numeric product id or by URL handle (e.g. 'summit-seamless-leggings'). Provide exactly one of id/handle.
List recent orders, newest first, with customer, total and payment/fulfillment status. Filter by order status and/or creation date range. Use get_order for line items and tracking details.
Aggregate sales analytics for a date range: order count, units sold, gross and net revenue, refunds, average order value, best day, and top products by revenue. Defaults to the last 90 days.
No output schemas documented. LLMs cannot predict response structure or plan downstream tool calls. Each tool should document returned fields (e.g., products return [id, title, price_range, status], orders return [id, order_number, customer, total, status]).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). update_inventory is marked WRITE in Risk but lacks destructiveHint annotation. Read-only tools lack readOnlyHint. Annotations help LLMs reason about safety and retry logic.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Search the store catalog by keywords. Matches product title, handle, type, vendor, tags and SKUs (case-insensitive, all words must match). Returns a compact list with ids, price range, stock and status.
Rank customers by net spend (order totals minus refunds) over a date range. Defaults to the last 90 days. Excludes cancelled orders and guest checkouts.
WRITE OPERATION. Set the available inventory quantity for a product variant (absolute value, not a delta). Get variant ids from get_product. Returns the previous and new quantity so the change can be verified or reversed.
Parameter descriptions lack prescriptive format guidance. E.g., created_after/created_before accept YYYY-MM-DD but description does not explicitly state 'must be YYYY-MM-DD format' or reference the DATE_PATTERN regex. LLMs may pass invalid formats.
get_product and get_order require 'exactly one of' two parameters (id/handle, id/order_number) but Zod schema does not enforce mutual exclusivity. LLMs may pass both or neither, causing silent failures or ambiguous errors.
Error messages are generic ('No product found', 'No orders matched'). Missing recovery guidance. E.g., 'No product found for id 8001. Use search_products() to find valid ids and handles.' is present for get_product but not consistently applied across all tools.