Loading ToolBench
MCP server for building Shopify stores via the Admin GraphQL API
Server has 9 well-named tools with clear verb-noun patterns (shop_get_info, theme_list, page_create, etc.). All tools have descriptions (avg ~120 chars, within baseline 34-392 range). Input schemas are present and typed for all tools. However, output schemas are NOT documented, critical gap for LLM planning. Parameter descriptions are present but minimal (10-30 chars typical). Risk annotations (READ_ONLY, DESTRUCTIVE, WRITE) are excellent for security clarity. Missing: pagination guidance for list tools, error recovery hints, and output field documentation. No examples of parameter constraints (enums, ranges) despite several tools accepting constrained values (e.g., theme roles, publish status).
List the store's navigation menus with their GIDs, handles, and items. Themes reference menus by handle (e.g. main-menu, footer). Use the id and current items from here for menu_update.
Replace a menu's items in the Shopify store. Items not included are DELETED from the menu, so get the current items from menu_list first and include every item you want to keep, with their ids.
Create a new page in the Shopify store. Pages are created as unpublished drafts unless isPublished is true.
List the store's pages with their GIDs, handles, and publish status. Use the id from here for page_update.
Update an existing page's title, body, handle, or publish status. Only provided fields change. Get the page GID from page_list.
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., theme IDs from theme_list, page IDs from page_create). Forces agents to guess response structure.
List tools (theme_list, page_list, menu_list) lack pagination parameters (limit, offset, cursor) and result count documentation. Large store data could exceed context window.
Parameter descriptions are minimal (10-30 chars). E.g., 'filePath' lacks format guidance (is 'sections/header.liquid' the only valid pattern?). 'isPublished' boolean lacks default behavior clarity.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 72 | 2026-07-28+ | v2 |
Get basic info about the connected Shopify store: name, domain, currency, plan.
List the store's themes with their GIDs and roles. Role MAIN is the published live theme. Use the id from here for theme_read_file and theme_update_file.
Read the content of one file in a Shopify theme. Get the theme ID from theme_list first.
Overwrite one file in a Shopify theme with new content. Get the theme ID from theme_list first.
No error recovery guidance. If theme_update_file fails (e.g., invalid Liquid syntax), description does not hint at next steps (validate syntax, check theme role, etc.).
menu_update description warns 'Items not included are DELETED' but lacks confirmation/dry-run pattern. Destructive operations should support preview or confirmation step to prevent accidental data loss.