Loading ToolBench
SSH MCP server with streaming, hang detection, dual logging, and snapshots
The SSH MCP server has reasonable naming and some parameter descriptions, but significant quality gaps prevent a higher score. 14 of 17 tools have structured input schemas visible in server.ts, but descriptions vary widely in quality (10-150 characters, well below the 194-char baseline). Critical issues: (1) No output schemas documented for any tool, LLMs cannot plan downstream calls or extract field names reliably. (2) Error handling is mentioned in code (HangDetector, AuditLogger) but tool descriptions do not guide recovery or classify errors. (3) Missing parameter descriptions on several tools (e.g., 'tail' parameter in job_output has a description but lacks units/constraints). (4) Tool names are generally verb-first and clear (list_hosts, exec, file_download), but three tools (snapshot_create, snapshot_list, snapshot_restore) operate on the same resource without consistent naming pattern. (5) Security-critical tool sudo_exec has no warning in description about destructive privilege escalation. Baseline: 549 production tools average 194-char descriptions and 100% of A+ tools document return types; this server documents neither.
Execute a shell command on a remote SSH host with real-time streaming output
Download a file from a remote host via SFTP
Check if a file exists on a remote host
List directory contents on a remote host
Get file information and metadata
Upload a file to a remote host via SFTP with optional snapshot
Terminate a running background job
List all background jobs
No output schemas documented for any tool. LLMs cannot predict field names, types, or structure of responses. This breaks downstream tool chaining and forces agents to reason about unpredictable payloads.
sudo_exec description (31 chars: 'Execute a command with sudo privileges (if enabled in config)') does not warn that this is a destructive operation requiring authorization checks. LLMs cannot assess risk.
Error handling is implemented in code (HangDetector, AuditLogger) but tool descriptions do not guide recovery or classify errors as retryable, user-fixable, or fatal. Agents cannot plan next steps on failure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
Get output from a background job
Start a command as a background job and return job ID
Get the status of a background job
List all configured SSH hosts and their connection status
Clean up old snapshots based on retention policy
Create a snapshot of a remote path (git commit or tar backup)
List available snapshots
Restore a snapshot to a remote path
Execute a command with sudo privileges (if enabled in config)
Snapshot tools (snapshot_create, snapshot_list, snapshot_restore, snapshot_cleanup) share a resource domain but lack consistent verb naming. Descriptions do not explain when to use each (e.g., when to call snapshot_create vs snapshot_restore, or the difference between snapshot_cleanup and manual deletion).
Parameter 'tail' in job_output has description 'Only get last N lines' but lacks units, constraints, or defaults. Is tail=0 valid? What is the maximum? LLMs cannot infer numeric constraints from text alone.
No pagination support declared for list_hosts, job_list, snapshot_list, file_list. If these return many results, responses could exceed context limits. Descriptions should mention limits and pagination parameters.
Tool descriptions average ~72 chars, well below the 194-char baseline for production tools. Many descriptions (e.g., 'Check if a file exists on a remote host', 'Get file information and metadata') are too brief to guide LLM selection or explain dependencies with other tools.