Loading ToolBench
Secure MCP SSH automation server with policy controls, resources, prompts, stdio, and HTTP.
ssh-mcp-pro demonstrates solid foundational quality with 17 well-named tools following verb_noun conventions (list_hosts, create_enrollment_token, run_shell). All tools have descriptions (avg ~80 chars, within baseline 34-392 range). Input schemas are present and properly typed with JSON Schema. However, parameter descriptions are sparse or missing entirely, most parameters lack explanatory text beyond the name. Output schemas are undocumented; LLMs cannot predict response structure. Error handling is minimal; no recovery guidance or actionable error messages visible. Security-sensitive tools (run_shell_as_root, file_write) lack explicit permission/scope documentation. Composition is sound, tools chain logically (list_agents → get_agent_install_command → run_shell). Risk classification is present but not integrated into tool metadata.
Create a one-time token and install command for enrolling a new outbound agent.
Read bounded Docker logs for an allowlisted container.
List Docker containers on an agent host when Docker is available.
Restart an allowlisted Docker container through the agent local policy.
Read a bounded text file from an allowed path.
Write text content to an allowed path when files.write is enabled.
Return npm-based install/run commands for a pending or enrolled agent.
Parameter descriptions missing or minimal. Most parameters (agent_id_or_alias, unit_or_file, container, path, command, policy) lack explanatory text. LLMs cannot infer whether 'path' accepts absolute paths, globs, or relative paths; whether 'command' is shell syntax or a structured format; or what 'policy' object structure is expected.
Output schemas undocumented. No visible response structure definitions for any tool. LLMs cannot predict what fields are returned (e.g., does list_hosts return {hosts: [{name, id, status}]} or {data: [...]}?), forcing them to guess and potentially misparse results.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Read recent control-plane audit events for the authenticated user.
Collect basic OS, uptime, load, and disk status from an online agent.
List enrolled agents and their connection state.
List enrolled host aliases available through outbound agents.
Restart an allowlisted service through the agent local policy.
Revoke an agent so no new actions can be dispatched.
Run a bounded shell command. Disabled unless shell.exec is explicitly granted.
Run a bounded privileged command. Disabled unless sudo.exec is explicitly granted.
Tail a configured system service unit or allowed log file through the agent.
Update an agent capability profile and local policy.
Destructive/irreversible tools (run_shell_as_root, revoke_agent, file_write) lack confirmation or dry-run support. No error handling guidance visible. LLMs cannot self-correct on invalid input (e.g., 'Invalid service name' vs 'Service not found, available: nginx, postgres').
Permission/scope declarations missing. Tools like run_shell_as_root and file_write do not declare required permissions (e.g., 'requires: shell.exec, sudo.exec'). Agents cannot be configured with least-privilege access; audit trails lack scope context.
update_agent_policy accepts a bare 'policy' object with no schema or description. LLMs cannot determine valid policy structure, required fields, or constraints. This invites malformed requests and silent failures.