MCP server providing Computer Use tools for macOS app automation via accessibility APIs and screen control
OpenAra defines 9 computer-use tools with complete input schemas and descriptions. All tools follow a clear verb_noun naming convention (list_apps, get_app_state, click, etc.) and include structured schemas with property types. Descriptions are clear and specify the tool's purpose. However, there are moderate gaps: (1) Output schemas are not documented, the source shows input schemas but no structured return types, forcing LLMs to infer response format. (2) Error handling is minimal, no error classification, recovery guidance, or constraint violation messaging. (3) Parameters lack granular constraint documentation (e.g., 'direction' in scroll_tool lists valid values in description but not as enum). (4) No documentation of parameter dependencies or mutually exclusive patterns (e.g., click accepts both element_index and x/y coordinates, which takes precedence?). (5) Descriptions are functional but minimal (averaging ~120 chars), many lack hints about when to use the tool vs. related ones. These gaps are typical for domain-specific tools but reduce LLM planning efficiency.
Click an element by index or pixel coordinates from screenshot. This tool is part of plugin `Computer Use`.
Drag from one point to another using pixel coordinates. This tool is part of plugin `Computer Use`.
Start an app use session if needed, then get the state of the app's key window and return a screenshot and accessibility tree. This must be called once per assistant turn before interacting with the app. This tool is part of plugin `Computer Use`.
List the apps on this computer. Returns the set of apps that are currently running, as well as any that have been used in the last 14 days, including details on usage frequency. This tool is part of plugin `Computer Use`.
Invoke a secondary accessibility action exposed by an element. This tool is part of plugin `Computer Use`.
Press a key or key-combination on the keyboard, including modifier and navigation keys. - This supports xdotool's `key` syntax. - Examples: "a", "Return", "Tab", "super+c", "Up", "KP_0" (for the numpad 0 key). This tool is part of plugin `Computer Use`.
Output schemas completely undocumented. Tools return structured results but no JSON schema or field documentation is visible. LLMs cannot infer response structure, breaking downstream planning and chaining.
Parameter constraints not formalized. 'direction' in scroll describes valid values as prose (up, down, left, right) but lacks enum constraint. 'mouse_button' does use enum correctly, creating inconsistency. Enums are machine-parseable and prevent hallucinated values.
Ambiguous parameter semantics. The 'click' tool accepts both 'element_index' AND 'x'/'y' coordinates. No documentation states which takes precedence, whether both can be passed, or what happens on conflict. This invites parameter confusion.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Scroll an element in a direction by a number of pages. This tool is part of plugin `Computer Use`.
Set the value of a settable accessibility element. This tool is part of plugin `Computer Use`.
Type literal text using keyboard input. This tool is part of plugin `Computer Use`.
Minimal error handling. No evidence of error classification, recovery guidance ('try X'), or constraint violation messaging. If a tool fails, the LLM receives a raw error with no actionable next step.
Tool descriptions lack context and composition guidance. Descriptions state WHAT tools do but not WHEN to use them vs. related tools or what prerequisites exist. Descriptions average ~120 chars, below the 194-char baseline for A+ tools.
No parameter validation hints. Descriptions do not state expected format, range, or examples for string parameters like 'app' (is it 'Safari' or 'com.apple.Safari'?). Parameter 'element_index' is not documented, is it a numeric index, string ID, or path?