Loading ToolBench
Remote Linux filesystem and diagnostics via SSH
Server has 7 tools with complete input schemas and descriptions. Naming follows verb_noun convention (connect, disconnect, list_sessions, elevate_sudo, list_directory, find_files, read_file). Descriptions are detailed (100-250 chars) and explain WHEN to use each tool. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. Parameters lack explicit constraints (enums, ranges, patterns). Error handling is present but generic ('error' key without recovery guidance). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Session-based design requires careful sequencing but is well-documented in descriptions.
Open an SSH session to a Linux host and return a session_id; this is the required first step before any other tool. The AD password is elicited once and cached only in server memory, and the response includes hostname, OS, kernel, and last boot for triage.
Close an active SSH session and release it; sessions are not auto-cleaned, so always disconnect when done. If you don't have the session_id, call list_sessions first.
Elevate the session to run commands via sudo, prompting once for the sudo password (cached for the session and cleared on disconnect). Required before tools needing root, such as journalctl, service management, and kill_process.
Recursively find files matching a name pattern, returning path, size, and modification time (max 100 results). Use list_directory to browse a single folder instead.
List files and directories at a path showing permissions, owner, size, modification time, and name (max 200 entries). Use this to explore one folder; use find_files to search recursively by pattern instead.
Output schemas not documented. LLMs cannot predict response structure (e.g., what fields does connect() return? Is it {session_id, hostname, os, kernel, last_boot}?). This breaks tool chaining and forces agents to guess field names.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. connect/disconnect/elevate_sudo are WRITE operations; list_directory/find_files/read_file are READ_ONLY. Annotations help LLMs reason about safety and retry logic.
Parameters lack explicit constraints. find_files max_depth defaults to 5, capped at 10, but no enum or range constraint in schema. read_file end_line defaults to 200, no documented max. LLMs may pass invalid values; validation happens at runtime.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
List active SSH sessions with host, connection time, last used time, and command count. Use it to find a session_id or check whether you are already connected to a host.
Read file contents as numbered lines (max 500 per call) via efficient SFTP streaming that handles huge files. Range mode reads start_line through end_line; tail mode (tail=True) reads the last N lines for recent log entries.
Error responses lack recovery guidance. Errors return {'error': '...'} without actionable next steps. E.g., if elevate_sudo fails, should the agent retry, ask the user, or call a different tool? Pattern:recovery-guide requires explicit guidance.
Session-based design requires strict sequencing (connect → elevate_sudo → read_file). No tool prevents out-of-order calls or validates session_id existence before use. Agents may call read_file with a stale session_id and get a cryptic error.