Loading ToolBench
MCP server for querying and managing Shopify store data through the Shopify Admin API using GraphQL
Single tool with verbose description (1200+ chars) but critical gaps in schema clarity and parameter documentation. Tool name 'shopify_execute_graphql' lacks action verb clarity (execute vs query vs mutate). Input schema present but parameters lack type constraints (query/variables are free-form strings/objects with no validation guidance). Description includes helpful examples but violates best practice by embedding sample GraphQL queries, LLMs may reuse these literally. No output schema documented. Error handling returns generic JSON without actionable recovery guidance. Tool accepts arbitrary GraphQL, creating injection risk without input sanitization guidance.
Use this tool to query Shopify store data or perform store management operations through the Shopify Admin API. Perfect for retrieving product information, managing inventory, processing orders, or updating store content. **Before making a query or mutation, make sure run an introspection query to get the schema of the object types and fields.** This tool executes arbitrary GraphQL queries/mutations with the Shopify Admin API, providing access to all available operations permitted by your access token's scopes.
Tool name lacks clear action verb. 'shopify_execute_graphql' is ambiguous, does it query, mutate, or both? Should be 'query_shopify' or split into 'query_shopify' + 'mutate_shopify' for clarity.
No output schema documented. LLM cannot predict response structure (data vs errors, nested fields, pagination). Must document: returns {data: {...}, errors: [{message: string}]} with field descriptions.
Description embeds sample GraphQL queries (GetProducts, GetOrders, CreateProduct). LLMs reuse examples literally instead of adapting to context. Remove examples; replace with constraints: 'query must be valid GraphQL; use introspection to discover schema.'
Parameters 'query' and 'variables' lack type constraints and validation guidance. 'query' accepts any string, no mention of GraphQL syntax validation, max length, or injection prevention. 'variables' is untyped object with no schema.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Error responses are generic JSON without recovery guidance. 'HTTP Request Error connecting to Shopify' tells LLM nothing actionable. Should return: 'Shopify API unreachable (timeout). Retry in 30s or check token/store name in .env'.