Loading ToolBench
AI-powered mobile & web test automation with Playwright + Mobilewright — Planner, Generator, and Healer agents with Android/iOS orchestration
Server has 6 tools with explicit Zod schemas and descriptions. However, critical gaps limit production readiness: (1) Tool names lack action verbs, 'run_planner', 'run_generator', 'run_healer', 'run_orchestrator' are generic and do not clearly convey intent; 'health_check' and 'list_repo_commands' are acceptable. (2) Descriptions are present but brief (avg ~80 chars), lacking WHEN/WHY context and prerequisites. (3) Output schemas are undocumented, tools return JSON-stringified results with no declared structure, forcing LLMs to parse unstructured text. (4) Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. (5) No parameter validation or constraint documentation beyond enums. (6) Security: tools spawn child processes with user-supplied arguments (url, plan, spec paths), no sanitization against command injection or path traversal.
Get basic MCP server and environment health information.
Return frequently used npm commands for this repository.
Run Generator agent to create executable tests from a Markdown plan.
Run Healer agent to execute and auto-repair failing tests.
Run Android or iOS orchestration pipeline (Planner -> Generator -> Healer).
Run Planner agent to generate a Markdown test plan from a target URL.
Generic tool names ('run_*') do not convey action intent. LLMs cannot infer whether 'run_planner' generates, executes, or validates. Rename to 'generate_test_plan', 'create_test_suite', 'execute_test_repair' to match Arcade pattern:tool verb_noun convention.
Output schemas are undocumented. Tools return JSON-stringified results via textResult() with no declared structure. LLMs cannot plan downstream calls or extract specific fields. Document return type: {exitCode, stdout, stderr, scriptPath} or equivalent.
No error handling guidance. Tools return raw exit codes and stderr without categorizing errors as retryable, user-fixable, or fatal. Add error classification and recovery hints per pattern:recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Command injection risk: runNodeScript() constructs shell commands with user-supplied 'url', 'plan', 'spec' parameters without sanitization. Malicious input like 'plan=file.md; rm -rf /' could execute arbitrary commands. Use execFile() with argument array instead of shell=true.
Descriptions lack WHEN/WHY context. 'Run Planner agent to generate a Markdown test plan' does not explain when to call it vs run_orchestrator, what prerequisites exist, or what the output enables. Add dependency hints per pattern:tool-description.