A C++ implementation of desktop automation and computer control tools, providing MCP server capabilities for desktop agent operations including screenshot capture, mouse/keyboard control, and application management.
Computer.cpp defines 10 tools for desktop automation with basic schemas and descriptions. However, critical gaps undermine production readiness: (1) Descriptions are minimal (10-50 chars), well below the 50-200 char baseline for LLM optimization. (2) Input schemas lack depth, most parameters have type and description, but lack constraints (enums, min/max, patterns). (3) No output schemas documented; tools return opaque 'result' objects. (4) Error handling is absent, no guidance on recovery, retryability, or actionable error messages. (5) Tool composition issues: observe_screen and screenshot are near-duplicates; action_sequence bundles multiple concerns. (6) No security annotations despite write operations (click, drag, activate_app). (7) Parameter descriptions lack format guidance (e.g., 'rect' structure not explained in parameter docs). This server is typical of community desktop automation tools, functional but not LLM-optimized.
Execute a sequence of actions including clicks, key presses, and text input
Activate and focus a specified application
Click at a specified rectangular region on the screen
Signal completion of the task
Drag from one rectangular region to another on the screen
Capture and observe the current screen state with optional accessibility information
Report a required value
Report visible feed posts
Descriptions are too short (10-50 chars) to guide LLM tool selection. Baseline is 50-200 chars. Examples: 'Capture a screenshot' (24 chars), 'Signal completion of the task' (29 chars). LLMs cannot distinguish when to use observe_screen vs screenshot.
No output schemas documented. Tools return opaque 'result' objects with unspecified fields (image, coordinate, accessibility). LLMs cannot plan downstream calls or extract required data.
observe_screen and screenshot are near-duplicates with overlapping functionality. Both capture screen state with maxDimension and frontmostWindowOnly. LLMs will waste reasoning cycles deciding between them.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Capture a screenshot of the current screen
Scroll within a specified rectangular region
action_sequence bundles click, key press, and text input into one tool. This violates single-responsibility principle and prevents agents from composing actions independently. Should split into separate tools or document when to use vs individual tools.
No error handling guidance. Tools lack descriptions of failure modes, retryability, or recovery steps. E.g., activate_app may fail if app is not installed, no guidance on what to do next.