Universal cross-platform computer control MCP server: observe, understand, act, verify on Windows and Linux GUIs without any browser/application-specific MCP.
Strong tool naming (all verb_noun, clear prefixes) and comprehensive descriptions (avg 180 chars, well above 34-char baseline). All 29 tools have explicit schemas with typed parameters and descriptions. Good error handling patterns (error_payload wrapper). However, output schemas are not formally documented in the code, responses are inferred from implementation. Some parameter descriptions lack constraint details (e.g., threshold ranges, timeout bounds). No tool annotations (readOnlyHint/destructiveHint) visible in schema registration. Risk classifications are present in metadata but not exposed to LLM via schema.
Report health/availability of every backend (input, screenshot, accessibility, window management, clipboard, OCR, vision) plus security settings and emergency-stop state.
Click a UI element. Target can be: visible text ('Continue'), an element id from computer.observe ('element_17'), or coordinates ({'x': 500, 'y': 300}). The engine resolves it via accessibility first, then OCR, then vision - and verifies the result.
Close a window gracefully (WM_CLOSE). Requires confirm=true; modal confirmation dialogs remain visible for the agent to handle.
Double-click a UI element (same flexible target rules as computer.click).
Drag from one target to another (text, element ids or coordinates).
EMERGENCY STOP: immediately halt all automation (checked before every action and attempt). Persists until reset_emergency_stop.
Output schemas not formally documented. Tool responses are inferred from implementation (error_payload wrapper visible, but return types not declared in schema registration). LLMs cannot plan downstream calls without knowing what fields to expect.
Parameter constraints lack specificity. Numeric parameters (threshold, timeout_s, max_depth, limit) have no min/max bounds documented. LLMs may pass absurd values (threshold=999, timeout_s=0.001) that break behavior.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | <=2025-11-25 | v2 |
Find text anywhere on screen (OCR + accessibility) with fuzzy matching. Returns matches with bounding boxes so you can click them. Tolerates case, arrows and small typos ('Continue' matches 'Continue →').
Find a UI element visually: by image template name (from the configured template directory) or by natural-language description via the optional vision model. Returns a bounding box.
Focus a real OS window by (partial) title or application name, e.g. 'Chrome'. Works without any application-specific integration.
Get the title, application and process of the currently focused window.
Read the clipboard text.
Get the accessibility tree of the active (or a named) window as a nested structure of roles, names and bounding boxes. Uses Windows UI Automation or Linux AT-SPI; returns an explanatory error when no accessibility backend is available.
Press a key combination. Pass a string like 'ctrl+s' or a list like ['ctrl', 'shift', 't'].
Launch a GUI application ('chrome', 'code', a .desktop entry, or a full path). Subject to security.allowed_applications.
List all top-level OS windows with title, application, pid and state.
Maximize a window by title/application.
Minimize a window by title/application.
Move the mouse cursor to a target (text, element id or coordinates).
Observe the current computer state: screen geometry, active window, cursor, OCR'd on-screen text and accessibility UI elements. Levels: 'minimal' (fast), 'normal' (screenshot+OCR+accessibility), 'full' (adds focused element details). This is the primary tool for understanding the screen before acting.
Press a keyboard key ('enter', 'tab', 'escape', 'f5', ...).
Clear a previously triggered emergency stop.
Restore a minimized/maximized window.
Right-click a UI element (same flexible target rules as computer.click).
Run a shell command (security.allowlist applies; see computer.backend_status). Requires confirm=true. Output is truncated and credential-redacted.
Take a screenshot of the whole virtual desktop, a specific monitor or a [left, top, right, bottom] region. Saves it to the persistence directory and returns the path.
Scroll the mouse wheel. Positive amount scrolls up, negative down. Optionally at specific coordinates (x, y).
Write text to the clipboard (useful as a typing fallback).
Type text at the currently focused input. Handles Unicode and multiline text (clipboard fallback used automatically). The typed content is never logged.
Wait until something changes on screen. Expectations: 'any_change', 'window' (value = window title substring), 'text' (value = OCR text substring). Use after actions to await UI reactions instead of blind sleeps.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) not visible in schema. Risk metadata exists (READ_ONLY, WRITE, DESTRUCTIVE, REVERSIBLE) but is not exposed via MCP schema annotations. LLMs cannot infer safety/idempotency from tool definitions alone.
Error recovery guidance incomplete. While error_payload wrapper exists, specific tools (run_command, launch_application) lack actionable error messages. E.g., 'Command failed' vs 'Command timed out after 30s; try increasing timeout_s or breaking into smaller steps.'
Flexible target parameters (text, element_id, coordinates) lack enum/pattern constraints. computer.click accepts str|dict but no schema validation of dict structure (must have 'x' and 'y' keys). LLMs may pass malformed targets.