Computer Use Protocol tools for AI agents. Exposes tools for UI tree snapshot, element search, action execution, and screenshots.
The CUP MCP server exposes 17 tools for UI automation and computer interaction. Tools have descriptions, but schemas are incomplete or missing for critical parameters. Tool naming follows verb_noun conventions (snapshot, action, find, open_app, screenshot), which is good. However, many parameter descriptions are sparse, and output schemas are entirely undocumented. The `action` tool accepts multiple conditional parameters (element_id, value, direction, keys) with complex interdependencies that are only partially documented in the description text, not in the schema. Some tools are duplicated across single-screen and multi-screen variants (e.g., tools #1 and #10 are both called 'snapshot'), which creates ambiguity for LLMs. Error handling and validation are not visible in the source provided.
Perform a UI action on an element on a specific screen. Element IDs come from the most recent snapshot of THAT screen. After performing an action, re-snapshot for fresh IDs. Element actions (require element_id): click — Click/invoke the element rightclick — Right-click to open context menu doubleclick— Double-click the element toggle — Toggle a checkbox or switch type — Type text into a text field (pass text in 'value') setvalue — Set element value programmatically (pass in 'value') select — Select an item in a list/tree/tab expand — Expand a collapsed element collapse — Collapse an expanded element scroll — Scroll a container (pass direction: up/down/left/right) increment — Increment a slider/spinbutton decrement — Decrement a slider/spinbutton focus — Move keyboard focus to the element Keyboard shortcut (no element_id needed): press — Send a keyboard shortcut (pass combo in 'keys') Examples: "enter", "ctrl+s", "ctrl+shift+p", "alt+f4"
Perform an action on a UI element or send a keyboard shortcut. IMPORTANT: Element IDs are only valid from the most recent tree snapshot (snapshot, snapshot_app, etc.). After performing any action, re-capture for fresh IDs. Element actions (require element_id): click — Click/invoke the element rightclick — Right-click to open context menu doubleclick— Double-click the element toggle — Toggle a checkbox or switch type — Type text into a text field (pass text in 'value') setvalue — Set element value programmatically (pass in 'value') select — Select an item in a list/tree/tab expand — Expand a collapsed element collapse — Collapse an expanded element scroll — Scroll a container (pass direction: up/down/left/right) increment — Increment a slider/spinbutton decrement — Decrement a slider/spinbutton focus — Move keyboard focus to the element Keyboard shortcut (no element_id needed): press — Send a keyboard shortcut (pass combo in 'keys') Examples: "enter", "ctrl+s", "ctrl+shift+p", "alt+f4"
Duplicate tool names across single-screen and multi-screen variants (snapshot, snapshot_app, snapshot_desktop, overview, action, find, open_app, screenshot). LLMs will conflate the variants and select the wrong version. The naming convention does not distinguish them, both are called 'snapshot' with different parameter sets.
The 'action' tool has complex conditional parameter logic that is only documented in prose description, not in schema. The tool accepts mutually exclusive parameter combinations (element_id+action_name vs keys+press) but does not declare this in the input schema. LLMs cannot infer which combination is valid without exhaustive trial.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 63 | - | v1 |
Search the last captured tree on a screen for elements matching criteria.
Search the last captured tree for elements matching criteria. Searches the FULL tree (including elements not shown in compact output) with semantic matching and relevance ranking. Results are sorted by relevance — best matches first.
List all connected screens with their number, name, OS, and hostname. Call this first to see what screens you can control. Each screen can be referenced by its number (1, 2, 3...) or name.
Open an application by name. Fuzzy-matches the name against installed apps on the system. Examples: "chrome" → Google Chrome, "code" → Visual Studio Code, "notepad" → Notepad, "slack" → Slack. Waits for the app window to appear before returning success. After opening, use snapshot() to capture the new app's UI tree.
Open an application by name on a screen.
List all open windows on a screen. Near-instant, no tree walking. Returns a lightweight window list showing app names and status. No element IDs are returned. Use this to quickly discover what apps are open before targeting a specific one with snapshot_app.
List all open windows. Near-instant, no tree walking. Returns a lightweight window list showing app names, PIDs, and bounds. No element IDs are returned (no tree walking is performed). Use this to quickly discover what apps are open before targeting a specific one with snapshot_app(app='...').
Capture a screenshot from a screen.
Capture a screenshot of the screen or a region.
Capture the UI accessibility tree from a screen. Returns a structured text representation where each UI element has an ID (e.g., 'e14') that can be used with the action tool. Also includes a window list header showing all open apps on that screen. Element IDs are ephemeral — only valid for THIS snapshot of THIS screen. After executing any action, re-snapshot for fresh IDs.
Capture the foreground (active) window's accessibility tree. Returns a structured text representation where each UI element has an ID (e.g., 'e14') that can be used with the action tool. The format shows: [id] role "name" @x,y wxh {states} [actions] val="value" Indentation shows the element hierarchy. Also includes a window list in the header showing all open apps. This is the primary tool for interacting with the current app's UI. Element IDs are ephemeral — they are only valid for THIS snapshot. After executing any action, you MUST call this again for fresh IDs.
Capture a specific app's window accessibility tree on a screen. Use this when you need to interact with a window that is NOT in the foreground, or when you know the exact app you want. The 'app' parameter is a case-insensitive substring match against window titles (e.g., "Spotify", "Firefox", "VS Code"). Element IDs are ephemeral — only valid for THIS snapshot.
Capture a specific app's window accessibility tree by title. Use this when you need to interact with a window that is NOT in the foreground, or when you know the exact app you want by name. The 'app' parameter is a case-insensitive substring match against window titles (e.g., "Spotify", "Firefox", "VS Code"). Returns the same compact format as snapshot, with element IDs that can be used with the action tool. Element IDs are ephemeral — only valid for THIS snapshot.
Capture the desktop surface (icons, widgets, shortcuts) on a screen. Falls back to a window overview if the platform has no desktop concept. Element IDs are ephemeral — only valid for THIS snapshot.
Capture the desktop surface (icons, widgets, shortcuts). Use this to see and interact with desktop items. Falls back to a window overview if the platform has no desktop concept. Returns the same compact format with element IDs for the action tool. Element IDs are ephemeral — only valid for THIS snapshot.
Output schemas are completely undocumented. All 17 tools lack formal response schema definitions. The descriptions mention what is returned (e.g., 'structured text representation with element IDs'), but the LLM has no machine-readable schema to guide parsing or downstream tool chaining.
Parameter descriptions for 'direction', 'keys', 'scope', and 'region_*' are extremely brief (under 30 chars) and lack examples or constraints. The 'scope' parameter in snapshot accepts 'foreground' or 'overview' but the description does not enumerate valid values, LLM must infer from example alone.
No error handling guidance. The source code provides no visibility into what errors these tools return, how they classify failures (retryable vs. fatal), or what the LLM should do when a call fails. For a critical interaction tool like 'action', absence of error documentation is dangerous.
The tool descriptions emphasize that element IDs are 'ephemeral' and 'only valid for THIS snapshot', but this critical constraint is not enforced in the schema. There is no parameter validation that prevents an LLM from passing a stale element ID from a previous snapshot. The tool must validate this and return a clear, actionable error.
The multi-screen variants accept a 'screen' parameter as 'screen number (1, 2, ...) or name (e.g., "windows", "mac")' but do not define the enum of valid screen names or the range of valid screen numbers. LLM will guess or pass invalid values.