Computer Use Protocol (CUP) MCP server exposing tools for UI tree capture, element search, action execution, and screenshots for AI agents
CUP MCP Server demonstrates solid definition quality with well-documented tool descriptions and clear parameter schemas. All 9 tools have descriptions ranging 100-400+ characters, exceeding the 34-character baseline. Parameter schemas are properly typed with Zod validation. However, there are notable gaps: (1) output schemas are not explicitly documented for any tool, forcing LLMs to infer result structure; (2) error handling guidance is absent, tools do not indicate what errors may occur or how to recover; (3) tool names, while descriptive, could better follow verb_noun conventions (snapshot_app, snapshot_desktop, snapshot are all variations of 'snapshot' that may cause LLM confusion); (4) no enum constraints for action parameter in the 'action' tool, which accepts a freeform string despite having a fixed set of valid actions.
Execute an action on a UI element or send a keyboard shortcut. IMPORTANT: Element IDs are only valid from the most recent snapshot. After executing any action, re-capture for fresh IDs. Element actions (require element_id): click, rightclick, doubleclick, toggle, type, setvalue, select, expand, collapse, scroll, increment, decrement, focus Keyboard shortcut (no element_id needed): press — pass combo in 'keys' (e.g., "ctrl+s", "enter")
Search the last captured tree for elements matching criteria. Searches the FULL tree with semantic matching and relevance ranking. If no tree has been captured yet, auto-captures the foreground window. QUERY MODE (recommended): Pass a freeform query describing what you're looking for. Examples: "the play button", "search input", "volume slider" STRUCTURED MODE: Pass explicit role, name, and/or state filters. Both modes can be combined: query + state="focused" narrows to focused elements.
Open an application by name. Fuzzy-matches the name against installed apps on the system. Examples: "chrome" → Google Chrome, "code" → Visual Studio Code. After opening, use snapshot() to capture the new app's UI tree.
List all open windows. Near-instant, no tree walking. Returns a lightweight window list showing app names, PIDs, and bounds. No element IDs are returned (no tree walking is performed). Use this to quickly discover what apps are open before targeting a specific one with snapshot_app(app='...').
Output schemas not documented. Tools return 'content' array with 'text' field, but LLMs cannot see the structure and must infer it. Risks misuse of results.
No error handling guidance. Tools do not document what errors may occur (e.g., app not found, element ID invalid, action failed), how to detect them, or how to recover. LLMs have no fallback strategy.
action parameter in 'action' tool accepts freeform string but has fixed valid values (click, rightclick, doubleclick, toggle, type, setvalue, select, expand, collapse, scroll, increment, decrement, focus, press). Should be an enum constraint to prevent hallucinated action values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 64 | - | v1 |
Page through clipped content in a scrollable container. When a tree element shows 'N more items — page(...)', call this tool to retrieve hidden children without re-capturing the UI. Deterministic pagination: returns consistent content windows based on the raw tree snapshot, unlike action('scroll') which changes the DOM. Works only with statically-laid-out lists. If the container uses virtual scrolling (dynamic content loading), use action('scroll') + snapshot() instead.
Capture a visual screenshot of a region or the whole screen. Useful when the accessibility tree doesn't provide enough context (e.g., colors, images, layout, or visual UI that isn't exposed in ARIA). If 'region' is specified, captures only that area (x, y, width, height). Otherwise, captures the entire visible screen.
Capture the foreground (active) window's accessibility tree. Returns a structured text representation where each UI element has an ID (e.g., 'e14') that can be used with the action tool. The format shows: [id] role "name" @x,y wxh {states} [actions] val="value" Indentation shows the element hierarchy. Also includes a window list in the header showing all open apps. This is the primary tool for interacting with the current app's UI. Element IDs are ephemeral — they are only valid for THIS snapshot. After executing any action, you MUST call this again for fresh IDs.
Capture a specific app's window accessibility tree by title. Use this when you need to interact with a window that is NOT in the foreground, or when you know the exact app you want by name. The 'app' parameter is a case-insensitive substring match against window titles (e.g., "Spotify", "Firefox", "VS Code"). Element IDs are ephemeral — only valid for THIS snapshot.
Capture the desktop surface (icons, widgets, shortcuts). Use this to see and interact with desktop items. Falls back to a window overview if the platform has no desktop concept. Element IDs are ephemeral — only valid for THIS snapshot.
Multiple snapshot* tools (snapshot, snapshot_app, snapshot_desktop) with similar names and nearly identical functionality. LLM may struggle to choose the right variant. Consider renaming to get_foreground_tree, get_app_tree, get_desktop_tree for clarity.
snapshot, snapshot_desktop, and overview tools have empty input schemas ({}), making them look parameterless. While technically correct, this invites LLM confusion about when each is appropriate. Descriptions mitigate but could be clearer in schema.
action tool's 'direction' and 'keys' parameters lack enum constraints or format documentation. 'direction' should constrain to [up, down, left, right]; 'keys' should document syntax (e.g., 'ctrl+s', 'shift+alt+t').