Loading ToolBench
MCP-only relay for v2 binary protocol. Pure MCP server for tool access to legacy Unix systems via binary streaming protocol.
17 tools with consistent naming (all verb-prefixed: read_, write_, edit_, list_, execute_, etc.) and basic schema coverage. Most tools have acceptable 1-2 sentence descriptions and documented input parameters with types. However, descriptions are terse (avg 50-80 chars, below the 194-char baseline for A+ tools), output schemas are not documented, and error handling guidance is absent. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Parameter descriptions are minimal, e.g., 'Path to the file' lacks format/constraint detail. No pagination docs for multi-result tools. Security-sensitive tools (remove_file, execute_command) lack confirmation or permission-gate patterns. The server is functionally organized but lacks the depth required for LLM-optimized tool definitions.
Copy a file FROM the Linux host TO the remote legacy system. Use this to transfer source archives, patches, or binaries to the remote system.
Download a file from URL using the Linux host's modern TLS.
Edit a file by replacing a specific string. The old_string must match exactly.
Execute a shell command. Returns stdout, stderr, and exit status.
Check if a file or directory exists.
Get file metadata (size, modification time, type).
Find files matching a name pattern (like glob/find).
No output schemas documented. LLMs cannot predict what fields are returned (e.g., does list_directory return an array of strings, or objects with {name, type, size}?). This forces trial-and-error downstream planning.
Descriptions are terse (avg 50 - 80 chars). Baseline for A+ tools is 194 chars. Examples: 'Write content to a file on the remote system. Creates or overwrites.' (80 chars) lacks guidance on when to use vs. edit_file, what happens if parent dir doesn't exist, whether it's atomic. LLMs need context for tool selection.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot auto-infer risk; they rely on annotations to decide retry safety. execute_command, remove_file, and write_file are destructive and should be annotated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | 2026-07-28+ | v2 |
Get the current working directory on the remote system.
List directory contents. Defaults to current working directory.
List contents of a directory on the Linux host. Use this to see what files are available to transfer to the remote system.
Create a directory on the remote system.
Move or rename a file.
Read the contents of a file from the remote system. Returns file contents with line numbers. By default reads up to 2000 lines.
Remove a file from the remote system.
Search for a pattern in files (like grep). Returns matching lines.
Copy a file FROM the remote legacy system TO the Linux host. Use this to back up files or retrieve build artifacts from the remote system.
Write content to a file on the remote system. Creates or overwrites.
No error handling guidance. When execute_command fails, what does it return? Stdout/stderr? Exit code? Can it be retried? When remove_file fails with 'permission denied', should the LLM ask the user or try a different path? No actionable recovery hints.
Destructive operations lack confirmation pattern. remove_file should support a dry_run parameter or require explicit confirmation. Agents make mistakes, an accidental call to remove_file('/*') could be catastrophic.
Parameter descriptions lack constraints. 'Path to the file' does not specify absolute vs. relative, max length, disallowed characters, or symlink behavior. 'Command to execute' does not mention timeout, shell type, or env var inheritance. LLMs default to unsafe assumptions.
No pagination docs. If search_files or list_directory return 10,000 items, how does the LLM page through them? No mention of limit, offset, max_results, or result count. Large lists can exhaust context.
Security: No permission or secret injection guidance. If the relay process runs as a high-privilege user, tools like remove_file and execute_command are maximally dangerous. Should check agent/user permissions before executing.