Loading ToolBench
MCP server that provides access to Siri shortcuts
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error guidance. Tool names follow verb_noun convention (list_, open_, run_), which is good. However, descriptions are minimal (10-50 chars), parameters lack detailed constraints, and output schemas are not formally documented. No tool annotations (readOnlyHint/destructiveHint). Error handling exists but lacks recovery guidance. The server does not expose secrets as parameters (good), but lacks comprehensive validation and user-friendly error messages.
List all available Siri shortcuts
Open a shortcut in the Shortcuts app
Run a shortcut with optional input and output parameters
Descriptions are too brief (10-50 chars). LLMs cannot determine when to select each tool or distinguish run_shortcut from open_shortcut. Descriptions should be 50-200 chars and explain WHAT, WHEN, and WHAT IT RETURNS.
Output schemas are not documented. Callers cannot know what fields to expect (e.g., does list_shortcuts return {shortcuts: [{name, id}]} or {shortcuts: [string]}?). Document return types for all tools.
Missing tool annotations. open_shortcut and run_shortcut are destructive (WRITE risk) but lack destructiveHint annotation. list_shortcuts is read-only but lacks readOnlyHint. Annotations guide agent planning and safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Error messages lack recovery guidance. McpError responses (e.g., 'Failed to list shortcuts: ...') do not tell the LLM what to do next or whether the error is retryable. Add actionable recovery hints.
run_shortcut parameter 'input' lacks format constraints. Description says 'text, or a filepath' but does not specify: max length, allowed characters, or how to distinguish text from filepath. LLMs will guess and pass invalid values.