Loading ToolBench
A macOS automation agent that dispatches tool calls to native handlers, file I/O, git operations, web search, accessibility actions, and MCP tools
This server has severe definition quality gaps. Of 38 tools, most lack proper input schemas, many show empty {} or missing parameter definitions entirely. Descriptions are present but often generic (10-30 chars). No output schemas documented. Tools like 'if_to_switch', 'extract_function', 'plan_mode', 'project_folder', 'list_tools', 'send_message', 'memory', 'ax_open_app', 'xcode_list_projects', 'list_agents', 'read_agent', 'lookup_sdef' have empty input schemas {}. Error handling is absent, no recovery guidance, no categorization. No tool annotations (readOnlyHint/destructiveHint). Security concerns: tools access filesystem, git, accessibility APIs, and UI without visible permission gates or input sanitization. Composition is poor, many tools do multiple things (e.g., 'ax_open_app' opens AND presumably returns state). This is a desktop automation tool masquerading as an MCP server, not a production-grade agent toolkit.
Find an accessibility element
Get the accessibility audit log
Get child elements of an accessibility element
Get the currently focused accessibility element
Get accessibility properties of an element
Get the frame/bounds of a window
Inspect an accessibility element
12 tools have empty input schemas ({}): if_to_switch, extract_function, plan_mode, project_folder, list_tools, send_message, memory, ax_get_audit_log, xcode_list_projects, list_agents, lookup_sdef.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Baseline: 100% of A+ tools have documented return types.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
List accessible windows for an application
Open an application
Read the content of the focused element
Take a screenshot
Copy an image file to clipboard or destination
Edit a file by replacing old_string with new_string
Extract a function from source code
Create or checkout a git branch
Create a git commit
Show git diff
Apply a git patch
Show git commit log
Get git repository status
Search using Google
Convert if-statement to switch statement
List available agent scripts
List files in a directory matching a pattern
List available tools
Look up scriptable definition for an application
Access memory/context
Enter plan mode for task planning
Get or set the project folder
Read an agent script
List directory contents with detailed information
Read file contents
Search for files containing a pattern using grep
Send a message
Mark the current task as complete
Search the web
Write content to a file
List Xcode projects
No error handling guidance. Tools fail silently or return raw shell errors. No recovery hints, categorization (retryable/user-fixable/fatal), or actionable error messages. E.g., git_commit could fail due to auth, dirty state, or invalid message, agent gets no guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot distinguish safe tools from destructive ones. write_file, edit_file, git_commit, git_branch, ax_open_app should be marked destructive; list_files, read_file, git_status should be marked readOnly.
Security: No visible permission gates, input sanitization, or secret injection. Tools access filesystem (read_file, write_file, edit_file), git (git_commit, git_branch), accessibility APIs (ax_*), and UI without validation. Agents could be tricked into path traversal, command injection, or destructive actions.