Loading ToolBench
A Model Context Protocol server that runs on a macOS machine and exposes tools (shell, file transfer, binary execution, screenshots) over a network port, protected by a bearer token. Allows an AI client running anywhere to drive the Mac.
Server exposes 6 tools with clear verb-noun naming (system_info, run_command, read_file, write_file, list_directory, screenshot). All tools have descriptions (avg ~100 chars, within baseline 34-392 range). Input schemas are present and typed. However, parameter descriptions are sparse or missing entirely, most params lack guidance on format, constraints, or valid ranges. Output schemas are undocumented. Error handling is minimal; no recovery guidance or actionable error messages. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Missing pagination for list_directory. No per-tool permission scopes declared.
List files and directories in a directory on the macOS machine.
Read a file from the macOS machine and return its contents as text or base64.
Run a shell command on the macOS machine and return exit code, stdout, stderr.
Capture a screenshot of the macOS desktop and return it as base64-encoded PNG.
Return macOS version, CPU architecture, hostname and current user. Call this first to confirm you are on macOS and to learn the architecture (arm64 for Apple Silicon, x86_64 for Intel) before running native binaries.
Write content to a file on the macOS machine. Creates or overwrites the file.
Parameter descriptions missing or incomplete. 'command' in run_command lacks format guidance (shell syntax, escaping rules). 'encoding' in read_file/write_file lacks explanation of when to use base64 vs utf-8. 'timeout' lacks range (min/max seconds). LLMs cannot infer constraints from names alone.
Output schemas undocumented. Callers do not know what fields to expect from run_command (does it return timed_out? stdout/stderr structure?), read_file (encoding of response?), or list_directory (file vs directory distinction? metadata?). Forces LLMs to guess or fail on parsing.
No tool annotations despite clear risk levels. run_command and write_file are destructive/irreversible but lack destructiveHint. system_info, read_file, list_directory, screenshot are read-only but lack readOnlyHint. Agents cannot distinguish safe from risky tools without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. No examples of error responses, no categorization (retryable vs fatal), no actionable messages. If run_command times out or write_file fails due to permissions, the LLM receives no guidance on next steps.
list_directory lacks pagination. No limit parameter, no offset/cursor, no total count. Large directories could return thousands of entries, bloating context. Baseline pattern requires pagination for list tools.