Loading ToolBench
MCP server for working across multiple Shopify stores (multi-client) via the Admin GraphQL API, with a production write-gate.
Server has 14 well-named tools with consistent verb_noun patterns (list_, get_, create_, update_, set_, add_, remove_). All tools have descriptions (avg ~120 chars, within baseline 34-392). Input schemas are present and typed with Zod. However, critical gaps exist: (1) object-type parameters (product, page, metafields) lack nested schema documentation, LLMs cannot infer valid field structures; (2) output schemas are not documented, agents don't know what fields to expect from responses; (3) error handling is basic (fail() returns text only, no recovery guidance); (4) pagination cursors are documented but total counts are missing; (5) write operations lack dry-run/preview capability despite high risk. Tool composition is strong, each tool has one responsibility, and read tools chain well to write tools via GID references.
Add tags to a resource (product, collection, customer, order, etc.). Pass the resource's GID as `id`.
Create a new Online Store page. Pass a page object with `title` and other fields.
Create a new product. Pass a product object with `title` and other fields.
Read metafields for any resource that has them (product, collection, customer, order, shop, etc.). Pass the resource's GID as `ownerId` (e.g. gid://shopify/Product/123).
List recent orders (newest first). Supports Shopify search syntax via `query` (e.g. "financial_status:paid", "fulfillment_status:unfulfilled"). Needs read_orders scope.
List/search Online Store pages (About, Contact, policies, etc.).
Object-type parameters (product, page, metafields) have no nested schema documentation. LLMs cannot infer which fields are required, their types, or valid values. E.g., create_product accepts {product: object} but doesn't specify title is required or what other fields exist.
Output schemas are not documented. Agents don't know what fields to expect from tool responses. E.g., get_products returns a GraphQL response structure, but the tool description doesn't specify the response shape (nodes array, pageInfo, etc.).
Error handling returns plain text with no recovery guidance. fail() function returns isError=true with a message, but doesn't categorize errors as retryable, user-fixable, or fatal. E.g., 'Request to store failed: 401' gives no hint that the agent should check credentials.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
List/search products. Supports Shopify search syntax via `query` (e.g. "status:active vendor:Acme", "updated_at:>2026-01-01"). Use `after` (endCursor) to paginate.
List all Shopify stores configured in this connector (alias, label, domain, env, API version). Use the returned `alias` as the `store` argument elsewhere.
Execute arbitrary GraphQL against a store. Pass a GraphQL query/mutation document and optional variables. Mutations against production stores require confirm=true.
Remove tags from a resource (product, collection, customer, order, etc.). Pass the resource's GID as `id`.
Set metafields on any resource (product, collection, customer, order, shop, etc.). Pass an array of metafield objects with `ownerId`, `namespace`, `key`, `type`, and `value`.
Fetch identity/plan info for a store. Handy connectivity check that the alias, domain and token are wired up correctly.
Update an existing Online Store page. Pass the page `id` and a page object with fields to update.
Update an existing product. Pass a product object with `id` and fields to update.
Write operations (create_product, update_product, set_metafields, etc.) lack dry-run or preview capability. The confirm flag gates production writes, but there's no way to preview changes before committing. High-risk mutations should support a dry-run mode.
Pagination responses include endCursor but no total count or hasNextPage clarity. Agents cannot estimate result set size or know when they've reached the end. Baseline pattern expects total count or explicit 'no more results' signal.