Loading ToolBench
Python recreation of Claude Code with enhanced features, including AI-powered tools, code analysis, file operations, web search, and advanced reinforcement learning-based tool optimization
This MCP server has significant quality gaps across definition structures. Of 10 tools, most have adequate descriptions (8/10 have 50+ chars), but critical issues emerge in schema validation, parameter documentation, and naming conventions. The server exposes file system operations (View, Edit, Replace, MakeDirectory, ListDirectory) with dangerous paths and no validation guidance. Tool naming is inconsistent: 'GenerateImage' violates verb-first convention (should be 'generate_image'), while 'View', 'Edit', 'Replace' lack object specification. Several tools lack parameter descriptions or type constraints (e.g., CodeAnalyze's 'file_path' has no format guidance). Output schemas are not visible in the provided code. Error handling is absent, no guidance on recovery, retryability, or user-fixable vs. fatal errors. The RL-based tool optimizer in tool_optimizer.py is highly complex (2500+ lines) but the actual tool definitions are minimal. The rubric baseline expects 194-char average descriptions (p10=34, p90=392); most tools here are in the 60-150 char range, below production standard. The server is STDIO-only, which carries a hard protocol cap, and would score 50 max on protocol readiness regardless of schema quality.
Analyze code to extract structure, dependencies, and complexity metrics
This is a tool for editing files. For moving or renaming files, you should generally use the Bash tool with the 'mv' command instead.
Generate an image using AI based on a text prompt
List files and directories in a given path with detailed information.
Create a new directory on the local filesystem.
Write a file to the local filesystem. Overwrites the existing file if there is one.
Convert text to speech using AI
Verb-first naming convention violated. 'GenerateImage' and 'TextToSpeech' use PascalCase and lack underscores; should be 'generate_image', 'text_to_speech', etc. Single-word tools like 'View', 'Edit', 'Replace' are ambiguous, missing object clarity ('view_file', 'edit_file', 'replace_file'). LLMs rely on name parsing to infer intent before reading descriptions.
File system tools (View, Edit, Replace, MakeDirectory, ListDirectory) accept absolute paths with no validation guidance. Descriptions state 'must be an absolute path' but provide zero constraints on path traversal, symlink attacks, or sensitive directory access (e.g., /etc, /root). No error handling for invalid paths, permission denials, or out-of-bounds access. This is a critical security gap.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Reads a file from the local filesystem. The file_path parameter must be an absolute path, not a relative path.
Search the web for information using various search engines
Search Wikipedia for information on a topic
Output schemas are not documented. None of the 10 tools show what they return. LLMs cannot plan downstream tool calls without knowing the response structure. Required by pattern:tool and evidenced by baseline 'documented return types' 100% in A+ servers.
Parameter descriptions are generic or missing. 'file_path' in View lacks format guidance (regex, length limit, allowed directories). 'prompt' in GenerateImage has no guidance on length, content constraints, or output fidelity. CodeAnalyze 'analysis_type' default is 'all' but no description explains the performance or latency implications.
No error handling or recovery guidance. Tools return nothing or fail silently. No indication whether errors are retryable (transient) or fatal. 'User not found. Try search_users()' pattern (pattern:recovery-guide) is absent. LLMs cannot self-correct when View fails (file not found? permission denied? path invalid?).
Destructive operations (Edit, Replace, MakeDirectory) lack confirmation or dry-run support. An agent can silently overwrite or delete files. Pattern:confirmation-request and pattern:idempotent-operation are not applied. Edit's 'old_string' and 'new_string' approach is fragile, if text changes between discovery and execution, the operation fails silently or modifies the wrong content.
No pagination or result limits for ListDirectory and WebSearch. ListDirectory can return thousands of files and blow context windows. WebSearch returns 'num_results' param (max 10 in schema) but no documentation of how pagination works if the user wants more. Pattern:paginated-result not applied.
Tool composition is weak. WebSearch and WikipediaSearch return data, but downstream tools cannot directly use that data, no chaining IDs. Pattern:tool-chain requires View to accept search results, but search tools likely return URLs/summaries incompatible with View's 'file_path' param. Forces extra manual reasoning.
GenerateImage and TextToSpeech accept 'save_path' as optional, defaulting to unspecified behavior. What happens if save_path is omitted? Is output returned in-line? Saved to a temp directory? No description clarifies. This ambiguity forces the LLM to guess, increasing errors.