Loading ToolBench
A persistent real PTY (pseudo-terminal) session manager for MCP, supporting local shell spawning and SSH remote execution with human takeover capability, output exploration, and audit logging.
Strong tool definitions with comprehensive descriptions and well-structured schemas. All 8 tools have detailed, LLM-optimized descriptions (100-400 chars) explaining WHAT, WHEN, and HOW. Input parameters use jsonschema tags with clear type info and descriptions. Tool names follow verb_noun pattern (terminal_open, terminal_send, terminal_status). Output structures documented via struct tags. Key strength: descriptions explicitly guide agent behavior (e.g., 'held=true means human takeover, pause writes'). Gaps: no enum constraints on string parameters (mode, op, key accept free-form strings), no explicit output schema documentation in code comments, missing error recovery guidance in descriptions.
Close the session, releasing its child process and concurrency slot. If held=true the session is under human takeover: this call was NOT executed; wait until held clears.
Send a control key to the session, or perform a recovery action. Control keys are written to the PTY as raw control bytes and work even while a command is running: ctrl-c (SIGINT, interrupt), ctrl-d (EOF), ctrl-z (suspend), ctrl-\ (SIGQUIT), ctrl-l (clear screen), ctrl-u / ctrl-k (erase to line start/end), ctrl-a / ctrl-e (move to line start/end), ctrl-w (erase word), ctrl-r (reverse search), ctrl-g (cancel), tab, esc, enter, backspace. Recovery actions: flush (drop queued input + clear the current line + Enter), hard (reopen the shell) and rearm (re-inject the sentinel prompt after you switched into a new shell, e.g. after su/docker exec/chroot, if the session appears stuck). If held=true the session is under human takeover: this call was NOT executed; wait until held clears.
Read-only exploration of an oversized result referenced by output_ref (returned by terminal_send / terminal_output when a single output exceeds the size cap). It inspects a fixed snapshot and does NOT advance the since_last cursor; the next since_last will NOT re-return this result. Do NOT read the whole result sequentially into context: first op=stat (size_bytes/line_count/max_line_bytes), then op=grep (pattern + before/after context) to locate, then op=read a local slice (line_offset 0-based, limit lines; a negative line_offset reads from the end; byte_offset continues a long-line read using the byte_offset from the previous read). Errors are usually at the end - use op=read with a negative line_offset. pattern is a Go regular expression; max_bytes is clamped to the server cap.
String parameters lack enum constraints. 'mode' (local|ssh), 'op' (stat|read|grep), 'key' (ctrl-c|ctrl-d|...) accept free-form strings, inviting hallucinated values. LLMs cannot validate against undeclared enums.
No explicit output schema documentation. Struct tags define fields (e.g., listOutput.Sessions) but no code comments or separate schema docs explain return types, field meanings, or pagination. LLMs must infer structure from descriptions.
Error handling guidance missing. Descriptions do not explain what errors are retryable, what user-fixable errors look like, or how to recover from 'session not found' or 'output too large' scenarios.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
List all sessions on this instance with their status snapshot (session_id, host, status, idle_seconds, held). held=true means that session is under human takeover; pause write operations until it clears.
Start a persistent real PTY session and return {session_id, state, terminal_url}. mode=local spawns a shell (or the given command) on this host; mode=ssh opens 'ssh <host>' running bash (host is required). The session starts in state=loading; poll terminal_status until it becomes idle before interacting. terminal_url is a read-only web terminal a human can open to watch the session live, and optionally 'take over' to type commands manually. While a human has taken over, the session returns held=true and the model's send/close/control are blocked; use terminal_output(mode=since_last) to observe what the human is doing.
Observe session output without relying on injected markers. Two modes, do not mix them to 'fetch everything': mode=tail (default, 'a quick glance'): returns only the last ~tail_bytes of the current screen to judge whether the command finished (check state and prompt). It does NOT advance the since_last cursor and can be called repeatedly. mode=since_last ('fetch the complete increment'): returns every new byte since the previous since_last call, losing nothing, and advances the delivery cursor. If a single increment is too large the return is truncated (truncated=true + output_ref) and the cursor has already advanced past it; inspect it with terminal_explore. Correct way to fetch a full result: poll with tail until state=idle, then read with since_last. To see what a human did during takeover you MUST use mode=since_last: it reconstructs every command the human typed as "[rc=n] $ command" (with exit code) together with its output. held=true means a human is currently in control.
Type a command into the session and block up to wait_ms (capped by max_block_seconds) for it to settle. Returns this command's new output plus state (running/idle/dead), prompt and exit_code. state=running means the command has not finished yet (e.g. a large core still loading) - keep polling with terminal_output. If the output is too large the return is truncated: truncated=true and an output_ref is returned and this output already advanced the delivery cursor; use terminal_explore (op=stat/grep/read) to inspect it selectively; you can continue running commands afterwards. If held=true the session is under human takeover: this call was NOT executed, do not retry write operations; wait or do other work and watch with terminal_output(mode=since_last) until held clears.
Lightweight status query (empty output). Returns state, prompt, exit_code and held. held=true means a human has taken over: the model should pause write operations and only read until held becomes false; use terminal_output(mode=since_last) to see what the human executed.
Numeric parameters lack bounds. 'wait_ms' and 'limit' accept unbounded integers; no min/max constraints documented. LLMs could pass absurd values (wait_ms=999999999, limit=1000000) causing timeouts or resource exhaustion.
No confirmation/dry-run pattern for destructive operations. terminal_close and terminal_control (ctrl-c, ctrl-z) are write operations but lack a dry-run or confirmation step. Agents could accidentally kill sessions.