Loading ToolBench
A personal AI assistant agent that works on file-based, local-first sapphire-framework workspaces together with you
Strong naming conventions (verb_noun pattern across all 13 tools). Descriptions are detailed and contextual (avg ~150 chars), explaining WHAT, WHEN, and constraints. Input schemas are complete with proper JSON Schema types and descriptions for all parameters. However, output schemas are not documented, LLMs cannot infer what fields to expect from responses. Error handling is minimal; no recovery guidance or categorization. Missing tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels assigned. Some descriptions could be more concise per LLM optimization guidelines (50-200 chars ideal).
Terminate an interactive shell session on the editor's machine. Only available over ACP.
Read output from an interactive shell session on the editor's machine. Only available over ACP.
Start an interactive shell session on the editor's machine. Only available over ACP. Returns a terminal handle that can be used with client_shell_output and client_shell_kill.
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. List the contents of a directory, one entry per line. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Entries are sorted alphabetically.
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Recursively list all files and directories under a path, one per line, in depth-first order. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Symlinks are not followed.
Output schemas not documented. LLMs cannot infer response structure (fields, types, pagination). Forces agents to guess what data is available for downstream tool chaining.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. file_delete, shell, and client_shell_kill are DESTRUCTIVE/IRREVERSIBLE but lack annotations to guide agent safety policies.
No error handling guidance. Tools lack recovery hints (e.g., 'file not found, check path with dir_list'). Agents cannot self-correct on failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Append to a file, creating it if it does not exist. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Cannot append to system directories or the agent's own config files.
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Delete a file. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Cannot delete directories or system files.
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Read a file with optional line-based pagination. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Returns lines prefixed with their 1-indexed line number in 'N|content' format. Use offset and limit for large files. Cannot read binary files or device paths (/dev/, /proc/).
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Write a file, creating it if it does not exist or overwriting it if it does. Accepts absolute paths, ~/... paths, or workspace-relative paths (resolved against the workspace root). Cannot write to system directories or the agent's own config files.
Reaches whichever machine this conversation is about: inside an ACP session, the machine the connected editor is running on; otherwise this agent's own machine. Run a shell command and return its output. Accepts bash syntax. The working directory is the workspace root. Commands run with a 30-second timeout.
List auto-enrolled voices that have not been named yet, with how much they have spoken and what they said. Use with speaker_promote to give one a name.
Register an auto-enrolled voice in the workspace, optionally naming it at the same time -- e.g. after the user says 'that was Tanaka-san'. Past transcripts pick the name up retroactively, because they store the speaker id and the name is resolved when they are read; renaming the directory later works the same way. Naming an existing speaker merges this voice into them and adds to their reference audio rather than replacing it. This is a manual escape hatch: it does not check whether the voice has cleared the automatic promotion thresholds.
Read ambient (always-on microphone) transcripts for a time window, optionally filtered to one speaker. Speakers are shown by their current name; names that look like random words are auto-enrolled voices nobody has named yet.
shell tool accepts unbounded command strings with 30-second timeout but no rate limiting or injection guards documented. LLMs can be tricked into command injection.
client_shell_* tools return opaque 'handle' strings with no documented format or lifecycle. Agents cannot reason about handle validity or reuse.