Loading ToolBench
A collection of MCP servers written in Go, including filesystem, PostgreSQL, shell, Redis, HTTP, and Home Assistant servers
Filesystem server has 10 tools with consistent naming (verb_noun pattern: read_file, write_file, etc.) and descriptions present for all tools. However, descriptions are brief (50-150 chars, below the 194-char baseline) and lack actionable context. Input schemas are visible and properly typed (string, boolean), but parameter descriptions are minimal and lack constraints (e.g., no mention of FS_ROOT behavior, FS_MAX_FILE_SIZE limits, or path traversal prevention). Output schemas are not documented, callers cannot predict response structure. Error handling returns text messages but lacks recovery guidance or error classification. Security concerns: safePath() validation is mentioned but not visible in provided code; no evidence of permission gates or audit logging.
Append content to a file under FS_ROOT, creating it (and parent dirs) if missing. Refuses if resulting file would exceed FS_MAX_FILE_SIZE.
Create a directory under FS_ROOT, including parent directories. Succeeds if the directory already exists.
Delete a file under FS_ROOT. Requires FS_ALLOW_DELETE=true. Refuses to delete directories.
Grep-like text search across files under a directory. Returns 'path:line: matching content'. Skips files larger than FS_MAX_FILE_SIZE.
Stat a path under FS_ROOT. Returns JSON with type, size, mtime, mode.
List entries in a directory under FS_ROOT. Directories are suffixed with '/'. Use recursive=true to walk subdirectories.
Output schemas not documented. Callers cannot predict response structure (e.g., does list_directory return {entries: [...]} or {files: [...], dirs: [...]}?). LLMs cannot plan downstream operations without knowing what fields to extract.
Parameter descriptions lack constraints and context. 'Path relative to FS_ROOT' does not explain: what happens if path escapes FS_ROOT? What is FS_ROOT? What is FS_MAX_FILE_SIZE? LLMs cannot validate inputs without explicit constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Move or rename a file or directory within FS_ROOT. Both source and destination must resolve inside the sandbox.
Read the complete contents of a text file under FS_ROOT. Returns an error if the file exceeds FS_MAX_FILE_SIZE.
Find files under FS_ROOT matching a glob pattern. Supports multi-segment globs (e.g. '**/*.go' matches Go files at any depth). Patterns are matched against the relative path of each file. Returns one path per line.
Write content to a file under FS_ROOT, creating parent directories as needed. Overwrites if the file exists. Refuses content larger than FS_MAX_FILE_SIZE.
Error messages lack recovery guidance. 'append would exceed max size' tells LLM the operation failed but not what to do next (split file? delete old content? ask user?). Errors should guide the agent's next step.
Destructive operations (delete_file, write_file, move_file) lack confirmation or dry-run support. Agents make mistakes, no safeguard prevents accidental data loss.
Tool descriptions are brief (50 - 150 chars, below 194-char baseline) and lack actionable context. 'Read the complete contents of a text file' does not explain when to use read_file vs search_files, or what happens if file exceeds FS_MAX_FILE_SIZE.