Loading ToolBench
Custom MCP server exposing filesystem operations as tools via FastMCP framework. Supports file reading, writing, directory listing, searching, and file operations with natural language command planning via Google Gemini.
Server has 12 tools with basic descriptions and schemas, but significant gaps limit production readiness. Most tools have descriptions (10/12), but many are generic or lack actionable detail. Input schemas are present but parameter descriptions are sparse or missing. The nl_command tool is particularly problematic, it delegates filesystem operations to an LLM without validation, creating security and reliability risks. Error handling is minimal; most tools raise exceptions without recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk stratification. Output schemas are undocumented. Naming is generally clear (verb_noun pattern), but parameter naming lacks consistency (e.g., 'path' vs 'src'/'dst'). This is a typical community server with functional tools but insufficient LLM-optimization.
Copy a file within FS_ROOT.
Create a directory (and parents).
Delete a file or directory (with recursive option).
Return basic metadata for a file or directory.
List entries in a directory under FS_ROOT.
Move/rename a file or directory within FS_ROOT.
Natural-language command. Uses Gemini to plan a single filesystem action under FS_ROOT. Returns a textual summary or error. Requires GOOGLE_API_KEY in environment/.env.
nl_command delegates filesystem operations to an external LLM without input validation, output verification, or error recovery. This creates security (prompt injection), reliability (hallucinated paths), and auditability risks. The tool description is vague about what actions are allowed or how errors are handled.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk stratification: 6 READ_ONLY, 4 WRITE, 1 DESTRUCTIVE, 1 WRITE (nl_command). LLMs cannot distinguish safe from dangerous operations without explicit hints.
Output schemas are not documented. Tools return dicts/lists/strings, but LLMs cannot plan downstream calls without knowing field names and types. E.g., list_directory returns [{path, name, is_dir, size}] but this is inferred from code, not declared.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Read and return file contents.
Find files by name substring within a directory tree.
Search for text within files under a directory tree.
Return a friendly welcome message for the user.
Write text to a file. If file does not exist: create it and write content. If file exists: append content at the end. Returns both characters and actual bytes written.
Error handling is minimal. Most tools raise FileNotFoundError or ValueError with no recovery guidance. E.g., 'File not found: /path' tells the LLM nothing about what to do next. No categorization of retryable vs fatal errors.
Parameter descriptions are sparse or missing. E.g., search_files has 'name_contains' with type 'string|null' but no description of what substring matching means (case-sensitive? regex? glob?). LLMs must guess intent.