Loading ToolBench
A Model Context Protocol server that provides tools to interact with Shopify stores, enabling access to products, customers, orders, and inventory management through Shopify's REST API.
This server implements 4 tools for Shopify operations (read-only). Tool naming follows verb-noun convention (get_products, get_product_details, get_customers, get_customer_details), which is good. However, definitions lack depth across critical dimensions. All tools have descriptions, but they are minimal (10-42 chars), falling significantly below the baseline of 194 chars for A+ tools. Input parameters are defined with types and basic descriptions (e.g., 'Maximum number of products to return (default: 10)'), but lack constraints, ranges, or guidance on valid values. Output schemas are NOT documented in the provided source, the tool definitions show only input parameters, not return structures. Error handling is not visible in the code. The similar naming pattern between get_products/get_customers and get_product_details/get_customer_details creates some redundancy but is acceptable given the domain. Parameter descriptions meet minimum thresholds but do not provide format constraints, dependencies, or recovery hints. Overall, the server is functional for basic read-only queries but lacks the depth and rigor expected of production tools.
Get detailed information about a specific customer.
Get a list of customers from the Shopify store.
Get detailed information about a specific product.
Get a list of products from the Shopify store.
Output schemas not documented. The source shows only input parameter schemas; return structures are not visible. LLMs cannot determine what fields to expect or plan downstream tool calls.
Descriptions are extremely brief (10-42 chars, well below the 194-char baseline). 'Get a list of products from the Shopify store' does not explain when to call this instead of get_product_details, what structure is returned, or pagination behavior.
Parameter constraints missing. The 'limit' parameter lacks minimum/maximum bounds. Is it 1 - 100? 1 - 1000? Unbounded numbers can cause LLMs to pass absurd values (e.g., limit=99999) that stress APIs.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
No pagination guidance. Tools returning lists accept a 'limit' parameter but do not document whether results are offset/cursor-paginated, total count returned, or what happens when limit is exceeded.
No error handling guidance visible. No documentation of what happens if a product_id or customer_id is invalid, does not exist, or is inaccessible. LLM receives bare error with no recovery path.