Loading ToolBench
Computer Use MCP Server - macOS screen control via Model Context Protocol
Server has 31 tools with complete schemas and descriptions, but quality is uneven. Strengths: all tools have descriptions (10-200 chars), input schemas are present with types and constraints. Weaknesses: descriptions are often generic or lack actionable context; many tools lack output schema documentation; error handling is minimal; no tool annotations (readOnlyHint/destructiveHint); parameter descriptions sometimes vague (e.g., 'coordinate' in mouse_move). Naming is verb-forward and clear. Composition is good, tools are single-purpose and chainable. Overall: solid foundation but falls short of production-grade LLM optimization.
Execute a sequence of actions in ONE tool call. Each action is an object with 'action' field and relevant parameters. Actions execute sequentially and stop on the first error. Supported actions: key, type, mouse_move, left_click, left_click_drag, right_click, middle_click, double_click, triple_click, scroll, hold_key, screenshot, cursor_position, left_mouse_down, left_mouse_up, wait.
Double-click at the given coordinates. Selects a word in most text editors.
Get information about the currently active (frontmost) window: app name, window title, position, and size.
Get the current mouse cursor position in logical screen coordinates.
Get information about the screen: logical dimensions, physical dimensions, Retina scale factor, and current cursor position. Call this before taking screenshots to understand the coordinate system.
Press and hold a key or key combination for the specified duration, then release.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe read-only tools from destructive ones without explicit hints. run_shell_command, write_clipboard, computer_batch, and open_application should be marked destructive.
Output schemas not documented. Tools return JSON but LLMs have no schema to parse responses. E.g., request_access returns {status, granted, denied, clipboard_read, clipboard_write} but this is not declared in tool definition.
Error handling is absent. No recovery guidance, no error classification (retryable vs fatal), no actionable error messages. run_shell_command and computer_batch can fail silently or with generic exceptions.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | <=2025-11-25 | v2 |
Press a key or key combination (e.g. "return", "escape", "cmd+a", "ctrl+shift+tab"). Modifiers joined with '+'. This is the unified key press tool.
Press a keyboard shortcut (key combination). Modifiers: cmd/command, shift, alt/option, ctrl/control, fn. Example: ['cmd', 'c'] for copy, ['cmd', 'shift', '4'] for screenshot, ['cmd', 'space'] for Spotlight.
Press a single named key. Supported keys: return, tab, space, delete, escape, left, right, up, down, home, end, pageup, pagedown, forward_delete, f1-f12, a-z, 0-9, and common symbols.
Type a text string character by character. Handles Unicode characters.
Left-click at the given coordinates.
Press the left mouse button at the current cursor position and leave it held. Use mouse_move first to position the cursor. Call left_mouse_up to release.
Release the left mouse button at the current cursor position. Pairs with left_mouse_down.
List all connected displays with their dimensions and properties.
List the applications currently granted access in this session.
List all visible windows on screen with their app name, title, position, and size.
Middle-click (scroll-wheel click) at the given coordinates.
Click the mouse at the specified screen coordinates (logical coordinates).
Drag the mouse from one point to another (logical coordinates).
Move the mouse cursor to the specified screen coordinates (logical coordinates).
Open a macOS application by name. Examples: 'Safari', 'Terminal', 'Finder', 'LM Studio'.
Read the current clipboard contents as text.
Request permission to control a set of applications for this session. Call this before interacting with specific applications. Returns the list of granted and denied apps.
Right-click at the given coordinates. Opens a context menu in most applications.
Execute a shell command and return its output (stdout, stderr, return code).
Switch which monitor subsequent screenshots capture. Use 'auto' to return to automatic (main) display selection. Use list_displays to see available monitors first.
Capture a screenshot of the screen. Returns the image as base64 PNG along with coordinate metadata for mapping image pixels to screen coordinates. Use get_screen_info first to know the screen dimensions. To click on something visible in the screenshot, map the pixel position: click_x = (pixel_x / image_width) * logical_screen_width, click_y = (pixel_y / image_height) * logical_screen_height.
Triple-click at the given coordinates. Selects a line in most text editors.
Wait for a specified duration in seconds.
Write text to the clipboard.
Take a higher-resolution screenshot of a specific region of the last full-screen screenshot. Use this to inspect small text, button labels, or fine UI details. Coordinates in subsequent click calls always refer to the full-screen screenshot, never the zoomed image.
Parameter descriptions lack actionable constraints. E.g., mouse_move has 'coordinate' param but does not specify coordinate system (logical vs physical), bounds, or format. take_screenshot's max_dimension defaults to 1024 but no guidance on when to adjust.
No permission gating or access control enforcement. request_access auto-grants all apps without user confirmation (comment: 'Auto-grant all requested apps (no UI confirmation in local build)'). Destructive tools like run_shell_command and write_clipboard should verify permissions before execution.