Loading ToolBench
MCP server for iOS Simulator development via xcrun simctl — device lifecycle, app management, screen interaction, testing helpers, logs, media, and keychain tools.
Scoring was not performed
Output schemas not documented. Tools return structured data but LLMs cannot see what fields to expect. Missing return type documentation forces agents to guess at response shape, leading to downstream mapping errors.
Error handling lacks recovery guidance. Error responses use generic callSimctl/callSimctlSilent patterns that return stderr or exit codes without actionable LLM guidance. E.g., 'simctl error' tells the agent nothing about what went wrong or what to try next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 16 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Destructive operations lack confirmation or dry-run support. Tools like erase_simulator and reset_keychain irreversibly modify simulator state but provide no confirmation step. Agents cannot validate intent before executing destructive actions.
Optional udid parameters lack clarity on resolution. Many tools state 'Optional if only one simulator is booted' but do not explain what happens if zero or multiple simulators are booted. This ambiguity forces agents to call resolveSimulator() which may throw or return undefined.
Some tool descriptions are vague on side effects. E.g., 'open_simulator' says it 'brings it to the foreground', does this fail if the app is not installed? Does it restart the Simulator? LLMs need to know exact behaviors to plan correctly.
No pagination or result limits on list_simulators, list_apps, etc. If a user has 100+ simulators or apps, the tool may return massive output. No mention of limits, pagination tokens, or result caps in descriptions.
Tool annotations missing. No readOnlyHint, destructiveHint, or idempotentHint annotations in tool definitions. This prevents MCP clients from rendering warnings, gating user confirmation, or tracking side effects.