Loading ToolBench
MCP server for Python code execution and environment management. Supports both in-process execution (default, fast) and subprocess execution (for environment isolation).
The server provides 11 tools with mostly adequate naming and descriptions, but exhibits significant gaps in schema completeness, parameter documentation, and error handling guidance. Tool names generally follow verb_noun conventions (execute_python_code, read_file, write_file, list_directory, delete_file, install_package, uninstall_package, create_directory, get_python_environments, get_working_directory). Descriptions are present for all tools and range from 60-200 characters, meeting the 10-1024 character baseline. However, parameter descriptions are inconsistent and sometimes vague. Input schemas are visible in the code but lack detailed type specifications and constraints for several tools. The server implements file operation guardrails (ALLOW_SYSTEM_ACCESS flag, path validation) which is positive for security. Error handling documentation is minimal, tools lack recovery guidance, and no clear error classification (retryable vs fatal) is present. The timeout parameter appears on code execution tools but lacks range documentation. Session management for inline mode is implemented but not well-documented in tool descriptions.
Create a directory. Path must be within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true. Creates parent directories as needed.
Delete a file. Path must be within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true
Execute Python code using in-process REPL (fast, persistent session state) or subprocess (isolated environment). Choose mode based on your needs: fast persistent state vs. environment isolation.
Execute shell commands with optional stdin input. Supports timeout and working directory configuration. Note: only available if MCP_ALLOW_SYSTEM_ACCESS=true
Get list of available Python environments and their versions
Get the current working directory for code execution
Install Python packages using pip. Supports various package specifications (name, URL, file path, git URL, requirements.txt)
Missing output schema documentation. No tool documents what fields are returned, their types, or structure. LLMs cannot plan downstream operations without knowing the response shape.
Timeout parameter on execute_python_code and execute_shell_command lacks documented constraints. Baseline should specify min/max (e.g., 1 - 3600 seconds). LLMs may pass absurd values without bounds.
No error guidance in tool descriptions. Errors should indicate: 'If code fails, check the stderr output' or 'If file not found, call list_directory() to discover available paths.' Currently, no recovery paths are documented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
List contents of a directory. Path must be within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true
Read the content of a file. Path must be within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true
Uninstall Python packages using pip
Write content to a file. Path must be within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true. Creates parent directories as needed.
Tool descriptions lack usage context. 'get_python_environments' returns 40 chars, insufficient for LLM to understand when/why to call it. Should describe: 'Call this to discover available Python versions before specifying python_path in other tools.'
File operation tools (read_file, write_file, delete_file, list_directory, create_directory) lack documentation of the path validation logic and security boundaries. Description mentions 'within the working directory unless MCP_ALLOW_SYSTEM_ACCESS=true' but this security-critical detail should be more prominent and actionable.
execute_python_code offers 'inline' and 'subprocess' modes but does not document the trade-offs or when to use each. Descriptions should say: 'inline: fast, persistent state, single-threaded; subprocess: isolated, slower, fresh environment.' This guides LLM mode selection.
Destructive tools (delete_file, execute_shell_command, uninstall_package) lack confirmation or dry-run patterns. No mechanism to prevent accidental data loss. Consider requiring confirmation for high-risk operations.