Loading ToolBench
MCP server for building, decompiling, and managing Apple Shortcuts (iOS/macOS) with zero third-party dependencies. Supports content-length framed messages and newline-delimited JSON.
Server has 17 tools with mostly complete schemas and descriptions. Naming is verb-forward and clear (list_*, build_*, decompile_*, etc.). Descriptions are present and substantive (avg ~150 chars), exceeding the 10-char minimum. Input schemas use proper JSON Schema with types. However, output schemas are largely undocumented, most tools lack explicit outputSchema definitions. Error handling is minimal; tools do not guide recovery or categorize errors. Parameter descriptions are adequate but some lack format/constraint details (e.g., 'target_platform' lists values in description but not as enum). Tool composition is sound, each tool has one responsibility. Security annotations (readOnlyHint, destructiveHint) are present on most tools, which is good. Overall: solid foundation, but output documentation and error guidance are weak.
Compile a recipe into a signed .shortcut file. Returns path, size, and metadata. Validates recipe, compiles WF plist, signs with deterministic UUID namespace. Respects safe_mode.
Return statistics on the action catalog: total actions, categories, curated aliases, and coverage.
Compare two action recipes: return diff (added/removed/modified actions) and similarity score.
Reverse a .shortcut file (signed or raw plist) into a high-level recipe JSON. Supports AEA1 decryption for library exports. Returns recipe with action list and metadata.
Explain magic-variable syntax: ${action:N}, ${var:Name}, ${as:Alias}, ${input}. Returns parsed structure and usage examples.
Export a shortcut as a library (AEA1 signed package). Requires safe_mode=0. Returns path and metadata.
Output schemas missing for all 17 tools. LLMs cannot predict response structure, forcing them to guess at field names and types for downstream chaining.
Parameter constraints not formalized as enums. 'target_platform' lists 'ios | macos | watchos' in description but not as JSON Schema enum, inviting LLM hallucination of invalid values.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2025-06-18+ | v2 |
Extract dictation workflows from /System/Library/PrivateFrameworks/SpeechObjects.framework. Returns list of system dictation shortcuts.
Extract Apple Gallery shortcuts from /System/Library/PrivateFrameworks/WorkflowKit.framework. Returns list of built-in shortcuts.
Extract a system library shortcut from macOS. Requires safe_mode=0 and Full Disk Access. Returns plist and metadata.
Extract user shortcuts from ~/Library/Shortcuts/Shortcuts.sqlite. Requires Full Disk Access. Returns list of shortcuts with metadata.
Return the full action recipe for a built-in template (e.g. hello_world, screenshot_ocr, shell_echo).
Inspect a .shortcut file without decompiling: return name, action count, platform, size, and metadata. Fast read-only probe.
List Apple Shortcuts action types from the full harvested catalog (400+ is.workflow.actions.* identifiers) plus curated short aliases. Any is.workflow.actions.* id is also buildable even if unlisted. Call before build_shortcut when discovering actions.
List built-in shortcut recipe templates.
Resolve one action short name or full is.workflow.actions.* identifier to catalog metadata, aliases, category, and suggestions.
Suggest actions based on a query or context. Returns ranked list of relevant action types.
Dry-run compile an action recipe without writing files. Returns ok/errors/warnings/risks and compiled action count. Validates magic-variable refs, platform preflight, and flags generic (non-curated) serialization. Respects server safe_mode for dangerous actions.
Error handling absent. Tools do not return actionable error messages or recovery guidance. A failed build_shortcut call will not tell the LLM what to try next.
Safe mode gating (SAFE_MODE env var) blocks 'dangerous' tools but does not expose this as a tool capability or error message. LLMs will attempt blocked tools and receive cryptic failures.
Path sandbox (ALLOW_ROOTS) enforced at runtime but not documented in tool descriptions. LLMs may attempt to write outside allowed roots and receive PermissionError with no guidance.