Loading ToolBench
Ultra-fast macOS automation server providing UI automation, app lifecycle management, system control, file operations, and calendar/contacts integration via MCP protocol
macctl has 35 tools with consistent naming (verb_noun pattern) and basic descriptions, but critical schema quality issues undermine usability. Most tools lack proper type definitions in their input schemas, parameters are declared as strings when they should be numbers (x, y, windowID, timeout, amount, limit). Descriptions are present but often generic (10-50 chars), below the 50-200 char baseline for LLM optimization. No output schemas documented. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Security concerns: macctl_shell executes arbitrary commands with no validation or sandboxing. No tool annotations (readOnlyHint, destructiveHint) despite clear risk stratification (READ_ONLY, WRITE, DESTRUCTIVE, IRREVERSIBLE). Parameter descriptions lack constraints (enums, ranges, formats). Overall, the server is functional but would not pass code review for production agent deployment.
Launch a macOS app by bundle ID.
List all running macOS applications with bundle IDs and PIDs.
Quit a macOS app.
Create a calendar event.
Fetch calendar events. Default: next 7 days.
Click a UI element by text query or element ID from macctl_see.
Read current clipboard (text, HTML, or files).
Type mismatches in input schemas: numeric parameters (x, y, windowID, timeout, amount, limit) declared as strings instead of numbers. This forces LLMs to pass '123' instead of 123, causing type coercion errors and silent failures.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., element IDs from macctl_see, window IDs from macctl_window_list). Agents must guess at response structure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | <=2025-11-25 | v2 |
Write to clipboard. Provide 'text' or 'html'.
Search contacts by name.
Read NSUserDefaults value. Returns value and type (string/int/bool).
Write NSUserDefaults value.
List directory contents.
Read a file as text.
Get file metadata: size, dates, iCloud status.
Write text to a file. Creates parent directories automatically.
List available keyboard layouts.
Switch keyboard layout.
Send keyboard shortcut. Named actions: new-tab, save, find, new, close, build, run. Or combos: cmd+s, cmd+shift+n.
Kill process by PID or name (SIGTERM by default).
List all running processes sorted by memory usage.
Create a new reminder.
List incomplete reminders.
List displays: name, resolution, scale, brightness.
Capture screenshot as PNG. Returns file path.
Scroll in an app window.
Enumerate interactive UI elements. Returns element IDs for macctl_click. Use before clicking to find the right element.
Execute a shell command via /bin/zsh. Returns stdout, stderr, exitCode.
Search for files by name using Spotlight.
Get system state: volume, brightness, WiFi SSID, Bluetooth.
Get or set volume (0.0-1.0). Omit value to read.
Type text into an app. Use 'into' to target a specific text field for faster AX setValue.
Toggle window fullscreen.
List all visible windows with IDs, positions, sizes. windowID is used by other window tools.
Move and resize a window.
Tile window to left or right half of screen.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk stratification in metadata (READ_ONLY, WRITE, DESTRUCTIVE, IRREVERSIBLE). LLMs cannot distinguish safe reads from destructive writes without explicit hints.
macctl_shell executes arbitrary shell commands with no input validation, sandboxing, or permission checks. A compromised or misdirected agent can delete files, exfiltrate data, or modify system state. Requires immediate mitigation.
Descriptions are generic and often under 50 characters (e.g., 'List all running apps', 'Get volume, brightness, WiFi SSID, Bluetooth'). Below the 50-200 char baseline for LLM optimization. Lack context on when to use each tool vs similar alternatives.