Loading ToolBench
Python executor tool in separate docker container, using playWright and headless browser for web scraping, and return output in print statements!
Single tool 'python-scraper-executor' has a reasonable description (127 chars, within 10-1024 range) but critical gaps in schema completeness and parameter documentation. Input schema visible with 'code' (required, string) and 'libraries' (optional, array), but array items lack type specification. No output schema documented. Parameter descriptions are minimal (e.g., 'Python code to be execute' has grammar error). No error handling guidance, no recovery paths, no idempotent/destructive hints. Tool name uses 'executor' (generic verb) rather than action-specific verb like 'execute_python' or 'run_python_code'. No mention of Docker dependency, timeout behavior, or resource limits in description.
Python executor tool in separate docker container, using playWright and headless browser for web scraping, and return output in print statements!
Array parameter 'libraries' lacks item type specification. Schema shows array but does not declare items as strings, leaving LLM uncertain about valid values.
No output schema documented. LLM cannot predict return structure (string? object? array?), making downstream tool chaining impossible and forcing LLM to guess field names.
Tool name 'python-scraper-executor' uses generic 'executor' verb. Should be 'execute_python_code' or 'run_python_script' to clearly signal the action and distinguish from other execution tools.
No error handling guidance. Tool can fail (Docker not installed, code syntax error, timeout, missing libraries) but returns bare error strings with no recovery hints. LLM cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Tool marked IRREVERSIBLE (risk) but description does not warn about side effects. No mention of Docker resource usage, network access, or that code execution is destructive/non-idempotent.