Loading ToolBench
MCP server for secure Python script execution with automatic dependency management
Three well-defined tools with strong descriptions and detailed schemas. execute_python has comprehensive docstring with examples; check_environment and validate_script have clear, actionable descriptions. All tools have proper input schemas with types and descriptions. However, output schemas are not formally documented in the code, they are described only in natural language. No tool annotations (readOnlyHint/destructiveHint) present. Error handling is mentioned in descriptions but lacks explicit recovery guidance structure.
Check the execution environment and report status. Returns information about Python version, uv version, platform, sandbox configuration, and cache status.
Execute a Python script with automatic dependency management. The script can include PEP 723 inline metadata (# /// script blocks) for declaring dependencies. Additional dependencies can also be passed via the dependencies parameter and will be merged. Args: script: Python source code to execute. May include PEP 723 metadata. dependencies: Extra PEP 508 dependency specifiers to make available. timeout_seconds: Maximum execution time (1-300, default 30). Returns: Formatted output with stdout, stderr, exit code, and duration.
Validate a Python script's PEP 723 metadata and dependencies without executing it. Checks metadata syntax, dependency format, and requires-python compatibility. Args: script: Python source code to validate. May include inline dependency metadata (# /// script blocks, see https://peps.python.org/pep-0723/). dependencies: Extra dependency specifiers to validate, using standard pip-style format like "requests>=2.28" (see https://peps.python.org/pep-0508/). Returns: Validation result with metadata details or error information.
Output schemas not formally documented. Tool descriptions mention what is returned ('Formatted output with stdout, stderr, exit code, and duration' for execute_python; 'information about Python version, uv version, platform...' for check_environment) but do not provide JSON Schema definitions. LLMs cannot extract return fields reliably.
Missing tool annotations. execute_python modifies state (executes user code, requires permissions), yet lacks destructiveHint annotation. check_environment and validate_script are read-only but lack readOnlyHint. These annotations guide LLM reasoning about side effects and safety.
Error handling descriptions lack recovery guidance. execute_python mentions timeout (1-300 seconds) but provides no instructions for what to do if execution times out. validate_script mentions 'error information' but does not guide the LLM on next steps (e.g., 'Fix dependencies and retry' vs 'Contact administrator').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
timeout_seconds parameter in execute_python lacks explicit validation rules in description. While the code clamps to [1, 300], the description only states 'Maximum execution time (1-300, default 30)' without saying 'must be between 1 and 300'. LLMs may not respect the bounds without explicit declarative constraints.
No pagination or result limiting declared. check_environment returns an unstructured string without structured fields. If extended to large environments with many modules or cache entries, responses could bloat. No limit or pagination mechanism is advertised.