Loading ToolBench
This HTTP-based SSH server exposes 7 tools with basic schema definitions but significant gaps in definition quality. Tool names are action-oriented (connect, exec, upload, download, ls, status, disconnect) which is good, but descriptions are minimal (7-29 characters), parameters lack constraints and comprehensive documentation, and output schemas are entirely undocumented. The server does not follow the MCP tool pattern for structured responses, error guidance, or input validation hints. Parameter descriptions in the source code are terse ('Connection identifier', 'Remote destination path') without explaining constraints, formats, or when to use each tool. No parameter enums, no bounds, no constraint hints. Error responses in the code return generic HTTP 500 errors without recovery guidance. The upload/download tools accept file paths as strings without validating for path traversal. The exec tool has no restrictions, validation, or dry-run support despite being DESTRUCTIVE. Output is unstructured JSON with minimal metadata.
Connect to SSH server
Disconnect from SSH server
Download file from remote SSH server
Execute command on remote SSH server
List directory contents on remote SSH server
Get connection status for SSH connection
Upload file to remote SSH server
Tool descriptions are critically short (7 - 29 characters), well below the 50 - 200 character baseline for LLM optimization. Example: 'Execute command on remote SSH server' (35 chars). Descriptions lack guidance on WHEN to use the tool, what it returns, or prerequisites.
Output schemas are completely undocumented. No tool declares what fields are returned, data types, or structure. LLMs cannot plan downstream tool calls or extract the right data. Responses appear to be unstructured JSON with 'success' and 'error' fields inferred from code, not formally declared.
Parameters lack enums, bounds, and detailed constraints. Example: 'port' is typed as 'number' with no min/max (SSH default 22, typical range 1 - 65535). 'command' accepts any string with no length limit or injection hints. 'remotePath' has no format validation despite being a security-critical input vulnerable to path traversal attacks.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 37 | - | v1 |
exec tool is DESTRUCTIVE (can delete files, modify systems) but has no dry-run, confirmation, or warning in the description. No guidance for LLMs on consequences or recovery. Pattern: confirmation-request.
No parameter type specifications for critical inputs. 'privateKey' and 'passphrase' are strings without format hints (PEM, OpenSSH, encrypted flag?). 'id' is a bare string with no uniqueness or format guidance. Parameter descriptions do not indicate if values are optional, required, or have defaults.
Error handling returns generic HTTP status codes (500) with minimal guidance. Example: 'Connect error: error.message' dumps raw exception text instead of actionable recovery hints like 'Authentication failed. Check password or private key format.' LLMs cannot plan recovery.
upload and download tools accept file paths as strings without path traversal validation. An LLM could be tricked into reading/writing outside intended directories (e.g., remotePath='../../etc/passwd'). No sanitization visible in src/server.ts.
No documented output structure or field references for tool chaining. Example: connect returns {success, message} but does not document what an LLM should pass to exec next. If exec requires a connection ID, the response must return the ID that exec accepts, no guarantee visible in schema or description.
ls tool executes 'ls -la ${dirPath}' with string interpolation (src/server.ts line ~100), vulnerable to command injection if dirPath is not sanitized. An LLM could pass 'path; rm -rf /' and execute arbitrary commands.