Loading ToolBench
Swift-based MCP server providing computer-use automation on macOS via accessibility APIs, vision grounding, and autonomous task execution
Server defines 17 tools with basic schemas and descriptions. Naming follows verb_noun convention (sdd_run, sdd_status, sdd_stop, etc.), which is good. However, descriptions are inconsistent in depth and quality. Many parameter descriptions lack specificity about constraints, ranges, or valid values. Output schemas are not documented, callers cannot predict response structure. Error handling is minimal; no recovery guidance provided. Tools like sdd_intervene and sdd_execute have overlapping responsibilities (both execute UI actions), violating single-responsibility principle. Parameter validation rules are not stated in descriptions (e.g., what are valid action types for sdd_intervene?). No evidence of idempotency guarantees or side-effect documentation.
Click a UI element by its label
Get a screenshot of the screen or a specific region
Get the current semantic world state of the screen including all UI elements
Scroll in a direction
Get the list of interactive UI elements currently visible on screen, as extracted by the accessibility layer.
Queue a single UI action for immediate execution (no run loop). Useful for one-off interactions.
Use ShowUI-2B visual grounding to find the screen coordinates of a UI element described in natural language.
Output schemas not documented. Tools like sdd_status, sdd_elements, sdd_workflows, get_world_state return unspecified structures. LLMs cannot predict response fields or plan downstream tool calls.
Overlapping tool responsibilities. sdd_intervene and sdd_execute both execute UI actions (click, type, scroll, key, wait) with nearly identical parameters. This violates single-responsibility principle and forces LLMs to reason about which to use.
Parameter constraints not specified in descriptions. 'action' parameter accepts 'click, type, scroll, key, wait' but no enum constraint or validation guidance. LLMs may pass invalid action types. Similarly, 'direction' in scroll tool lacks enum constraint.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
Execute a single direct action in the running environment, bypassing the SDD run loop. Use for one-off corrections.
Inject a corrective instruction into the running SDD task. Applied at the next step. Use when the agent is going off-track.
Request a workflow recording session. SDD will watch you perform the task once and learn it as a reusable workflow.
Start an autonomous computer-use run. SDD will execute the goal step-by-step using vision + accessibility APIs. Monitor progress with sdd_status.
Execute a named workflow (built-in or learned). Faster and more reliable than a full sdd_run for known tasks.
Capture the current screen state as a JPEG image. Returns the last screenshot taken by SDD (updated every step during a run).
Get the current status of the active SDD run. Returns step count, current action, and completion state.
Send a stop signal to the currently running SDD task. The run will halt at the next step boundary.
List all available workflows: built-in primitives and learned workflows recorded by sdd_record.
Type text into a field
No error handling or recovery guidance. Tool descriptions do not explain what happens on failure, whether errors are retryable, or what the LLM should do next. E.g., sdd_run may fail if max_steps is too low, but no guidance provided.
Ambiguous parameter semantics. sdd_intervene and sdd_execute both accept 'x', 'y', 'selector' but descriptions do not clarify: are coordinates required for all actions? What if both selector and x/y are provided? Undocumented dependencies cause silent misuse.