Loading ToolBench
Craft MCP demonstrates solid foundational quality with consistent naming conventions, reasonable descriptions, and proper schema definitions across all 11 tools. All tools follow verb_noun naming patterns (list_*, get_*, create_*). Descriptions are present and informative (avg ~100-150 chars), explaining what each tool does and when to use it. Input schemas are properly defined with types and descriptions for all parameters. However, there are notable gaps: (1) output schemas are not documented anywhere, LLMs cannot infer what fields to expect from tool results; (2) error handling is implicit rather than explicit, no clear guidance on recovery paths or error categories; (3) the create_backup tool lacks a confirmation/dry-run pattern despite being marked DESTRUCTIVE; (4) parameters could benefit from more detailed constraints and format specifications; (5) some descriptions lack dependency hints (e.g., list_assets should mention calling list_volumes first to get volume handles). The server demonstrates good SDL practices with tool annotations (McpToolMeta), conditional availability, and permission gating via attributes, but output documentation is the critical missing piece.
Create a new database backup. WARNING: This is a dangerous operation that creates files on the server.
Get a single asset by ID with full metadata
Get detailed information about a single Commerce order by ID or order number
Get detailed information about a single Commerce product by ID
List asset folders in a volume
List assets from Craft CMS. Filter by volume, folder, kind (image, video, pdf, etc.), filename.
List available database backups from storage/backups directory
Output schemas completely undocumented across all 11 tools. LLMs cannot infer what fields to expect from results, forcing them to guess or make wasteful follow-up calls. This is a critical blocker for tool chaining and context preservation.
create_backup is marked DESTRUCTIVE but lacks confirmation/dry-run pattern. No clear error recovery guidance. Agents cannot safely test the operation before executing it. Should support optional 'dry_run' parameter or require explicit confirmation.
get_order has two optional parameters (id and number) with no constraint stating at least one is required. Schema does not enforce mutual requirement. LLMs may call with both, neither, or only one without clear guidance on priority.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 69 | 2025-06-18+ | v2 |
List categories from Craft CMS. Filter by group handle.
List orders from Craft Commerce. Filter by status handle.
List products from Craft Commerce. Filter by product type handle.
List all asset volumes (storage locations) in Craft CMS
Discovery tools (list_volumes, list_asset_folders, list_categories, list_product_types, list_order_statuses) should include dependency hints in their descriptions. E.g., list_assets description should say 'Call list_volumes first to discover available volume handles.' This prevents unnecessary failed calls.
Parameter descriptions could be more precise about constraints. Example: 'limit' parameters lack explicit min/max guidance. 'kind' parameter values (image, video, pdf, word, excel, audio, compressed, text) should be listed as an enum constraint rather than prose description.
No explicit error handling guidance across any tool. No recovery paths documented (e.g., if create_backup times out, what should the agent do?). No categorization of errors as retryable vs. fatal.