Loading ToolBench
An MCP server providing file system operations with JWT authentication and security restrictions for protected files
The server provides 7 tools with basic structure, but quality varies significantly. Most tools have descriptions and parameter schemas, but descriptions are thin (10-30 chars for some), parameter type coverage is incomplete, output schemas are undocumented or inferred, and error handling lacks recovery guidance. Tool naming follows verb_noun convention well (get_*, list_*, create_*, delete_*, write_*). Path sanitization is a positive security feature. However, lack of output schema documentation, missing parameter descriptions in critical tools, and generic error handling prevent this from reaching 65+. The server is functional but below production baseline (70+).
Creates a new directory at the specified path.
Deletes a file or directory at the specified path. [Truncated in source - inferred from code context]
Reads and returns the complete content of a specified file.
Retrieves metadata about a specified file or directory.
Returns the server's protection configuration. Used by agents to know which files and tools are restricted.
Lists all files and subdirectories in a specified directory.
Output schemas are not formally documented. Tools return dicts or lists, but the structure of returned fields is documented only in prose docstrings, not in structured JSON Schema. LLMs cannot reliably plan downstream tool calls or extract specific fields without executing and observing the response.
Destructive operations (write_file_content, delete_path) lack confirmation or dry-run support. An agent mistake could overwrite or delete critical files without warning. No confirmation_request pattern implemented.
Error messages in list_directory and other tools are generic ('Error reading file: {e}'). They do not guide the LLM to the next action (e.g., 'File not found. Try list_directory() to see available files.').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 34 | - | v1 |
Writes the specified content to a file, overwriting if it already exists.
Tool descriptions vary in length and depth. create_directory has a 20-character description, below the 34-char baseline (p10). Short descriptions lack context for LLM tool selection. E.g., 'Creates a new directory' does not say what it returns or what happens if the directory already exists.
delete_path description in source code is marked '[Truncated in source - inferred from code context]'. The actual description cannot be verified. Tool definitions should be explicit and visible; inferred definitions introduce ambiguity.
No pagination support for list_directory. If a directory contains thousands of files, the entire list is returned, potentially exhausting the LLM context window. No limit, offset/page, or next_cursor parameters.
Return types are not formally documented in JSON Schema format. While docstrings mention return structures (e.g., 'A dictionary with the following keys'), LLMs cannot parse prose descriptions reliably. Formal schemas are required.