The server defines one tool (run_tests) with a complete input schema and good parameter structure. However, there are significant gaps: the output schema is entirely undocumented, descriptions lack guidance on when to use the tool vs alternatives, and error handling patterns are not evident from the source. The tool name is clear and actionable, but parameter descriptions could be more prescriptive about constraints and dependencies. Security parameters are well-structured but lack usage guidance. The tool accepts sensitive configuration (environment variables, working directories, shell options) without documented restrictions or safety warnings.
Output schema is completely undocumented. LLMs cannot plan downstream actions or extract test results without knowing what fields the tool returns. The parsed test results structure (from TestParserFactory.parseTestResults) is never defined in the tool response spec.
Tool description (29 chars: 'Run tests and capture output') is too brief and lacks actionable context. Does not explain: when to use this vs running tests manually, what frameworks are supported (only visible in enum), what 'capture output' means, or what happens on test failure. LLMs cannot determine if this tool fits their intent.
Security parameters (allowSudo, allowSu, allowShellExpansion, allowPipeToFile) lack usage guidance and safety warnings. Descriptions state the default but not WHY these restrictions exist or when they are safe to enable. No indication that enabling these flags introduces command-injection risk.
Expand tool description to 100-150 chars explaining: (1) what the tool does (runs arbitrary test commands), (2) when to use it (verify code quality, validate patches, run CI checks), (3) what it returns (structured pass/fail counts and raw output), (4) framework support (bats, pytest, flutter, jest, go, rust, or generic). Example: 'Execute test commands (bats, pytest, jest, go, rust, flutter, or generic) in a sandbox, capturing results, exit codes, and structured pass/fail counts. Use to validate code, run CI checks, or verify patches. Returns parsed test outcome and raw output.'
Add explicit constraints to numeric parameters: timeout (min 1000, max 600000), page limits if paginating results (min 1, max 100).
Add minimum/maximum range constraints to the timeout parameter in the schema: {"type": "number", "minimum": 1000, "maximum": 600000, "description": "Test execution timeout in milliseconds (1-600 seconds, default: 300000)"}
Document framework selection guidance in the framework parameter description. Example: 'Testing framework to use for parsing results. Choose based on your project: bats for bash scripts, pytest for Python, jest for Node.js, go for Go projects, rust for Rust projects, flutter for Flutter, or generic for other frameworks.'
The 'env' parameter (object with additionalProperties) accepts arbitrary environment variables with no restrictions or warnings. LLMs could accidentally pass credentials, API keys, or sensitive data via environment variables, which enter logs and traces. No guidance to prevent this misuse.
Parameter 'workingDir' has no format constraints or safety warnings. LLMs could pass absolute paths outside the intended sandbox, enabling path traversal. No guidance on directory restrictions or validation.
No documented error recovery guidance. Tool description does not explain: what happens if a test fails? What happens if the command times out? What happens if workingDir does not exist? LLMs need error categorization (retryable vs user-fixable vs fatal) to plan recovery.
Framework enum includes 7 values but descriptions for each framework are missing. LLMs cannot infer which framework to select without examples or guidance. What syntax differences exist between 'bats' and 'generic'? When should an LLM choose 'generic' vs a specific framework?
Timeout parameter (default 300000ms = 5min) is not enforced in the schema (no minimum/maximum). LLMs could pass absurdly large timeouts (e.g. 999999999) causing agents to hang indefinitely. Numeric bounds must be explicit.
Command parameter accepts free-form strings with no format validation hints. LLMs could pass shell metacharacters ($(), backticks, pipes) assuming they are safe, but allowShellExpansion and allowPipeToFile gates override this. Dependency between 'command' format and security flags is undocumented.
run_tests
Add warning to env parameter description: 'Environment variables for test execution. DO NOT pass credentials, API keys, or secrets, use server-side secret injection instead. LLM-provided env vars are logged and may leak sensitive data.'
Add safety guidance to workingDir: 'Working directory for test execution (must be an absolute path within /home or /tmp; no symlinks or traversal attacks). If not provided, defaults to current directory.'
Document command injection risks and add guidance: 'Test command to execute (e.g., "pytest tests/" or "bats tests/*.bats"). Command is passed to a shell unless allowShellExpansion is false. Avoid user input in command strings, pass via env parameters instead to prevent injection.'
Add comprehensive error handling guidance to the tool description or return docs: 'On test failure, returns parsed results with tests_passed=0 and tests_failed>0 (not an error). On timeout, returns error 'Test execution exceeded <timeout>ms, increase timeout or optimize tests'. On missing workingDir, returns error 'Working directory does not exist: <path>, verify path exists and is readable'. On unsupported framework, returns error 'Unknown framework: <framework>, must be one of: bats, pytest, flutter, jest, go, rust, generic'.'
Add explicit security best practices to tool description: 'For security, this tool enforces: sudo is disabled by default, shell expansion is enabled by default (set allowShellExpansion=false to disable), pipes to files are disabled by default (set allowPipeToFile=true to allow). Always validate command input and run in a sandboxed directory.'