Human-mode desktop control MCP connector for Linux/X11 — see, click, type, and act on the accessibility tree.
claude-workman provides 14 well-named, action-verb tools for desktop automation with complete input schemas and descriptions. Naming is strong (screenshot, click, type_text, press_key all follow verb_noun convention). Descriptions are present and actionable (avg ~120 chars), explaining WHAT each tool does and WHEN to use it. However, output schemas are undocumented, tools return dicts/lists but the structure is not formally specified. Error handling is minimal; tools return success dicts but lack recovery guidance. Parameters are well-typed (int, str, bool) with defaults, but some lack constraint documentation (e.g., 'direction' in scroll accepts 'up|down|left|right' but this is only in the description, not an enum). The accessibility_tree tool returns complex nested structures without documented field definitions. Overall solid for a desktop automation tool, but output documentation and error guidance are gaps.
Dump actionable UI elements as {app, role, name, x, y, w, h} with screen coordinates. Filter by app name substring. Call enable_accessibility first.
Click at pixel (x, y). button 1=left 2=middle 3=right. count=2 double.
Find an element by role+name in the accessibility tree and click its center — element-accurate, no pixel guessing.
Press at (from_x, from_y), drag to (to_x, to_y), release.
Turn on AT-SPI tree export. enable_web also lets Chromium/Electron export their web content. Orca is killed afterwards so nothing is spoken aloud.
Raise a window by id or name-substring. On focus-stealing WMs (e.g. mutter) plain activation can silently fail, so the frontmost blocker is minimized first. Always screenshot to verify focus before typing.
Output schemas undocumented. Tools return dicts/lists (e.g., list_windows, accessibility_tree, click_element) but field structure is not formally specified. LLMs cannot reliably extract data without knowing what fields to expect.
scroll direction parameter lacks enum constraint. Description states 'up|down|left|right' but parameter is free-form string. LLMs may hallucinate invalid directions (e.g., 'up-left', 'vertical'). Should be enum: ['up', 'down', 'left', 'right'].
Error handling absent. Tools return success dicts but no guidance on failure modes, retryability, or recovery. E.g., click() may fail if coordinates are off-screen, but no error message tells the LLM what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
List on-screen windows with id, name, pid and geometry.
Move the pointer to (x, y) without clicking.
Press a key/combo in xdotool syntax: 'Return', 'Tab', 'ctrl+c', 'super+l', 'KP_0'.
Capture the whole display as a PNG. max_dim downscales the long side (e.g. 1600) for faster round-trips. Take one before AND after any action.
Capture a sub-rectangle of the screen (faster than a full grab).
Scroll up|down|left|right by `amount` wheel steps.
Start/stop the visual click cursor overlay — a Codex-style ring + click ripple that shows on the monitor exactly where Workman is acting (for human oversight). Requires GTK (python3-gi). Actions auto-notify it once running.
Type literal text into the focused window.
accessibility_tree output structure not documented. Returns list of dicts with {app, role, name, x, y, w, h} but no formal schema. LLMs must infer field types and presence from examples.
list_windows description is minimal (11 words). Does not explain what fields are returned or when to call it. Should include: 'Returns list of {id, name, pid, x, y, width, height}. Call before focus_window if you need to find a window by name.'