Single tool 'execute' with basic schema and minimal description. The tool has a complete JSON Schema with typed parameters and reasonable constraints (timeout 1000-300000ms), but the description is generic and lacks critical operational context. No guidance on error recovery, side effects, or safety considerations. Parameters are typed but lack depth in their descriptions. The schema is visible and well-formed, preventing a lower score, but multiple rubric violations and missing best practices keep the score in the 'poor' range.
Tool description does not declare irreversible side effects. The 'Risk: IRREVERSIBLE' annotation exists in metadata but is not present in the tool description text that LLMs see. Description must explicitly state 'This tool executes arbitrary code and has irreversible side effects.'
Tool description is only 24 characters ('Execute Babashka (bb) code'), below the 34-character p10 baseline and well below the 194-character average for production tools. Missing: WHEN to use this tool, what it requires (Babashka installation), and what the output structure is. LLMs cannot distinguish this from other code-execution tools without richer context.
Parameter 'code' description is minimal ('Babashka code to execute'). Does not explain: language syntax expectations, memory/resource limits, or what happens to stdout/stderr. Production baselines average 72 characters per param description; this is ~26 characters.
execute
Recommendations
Rewrite tool description to at least 100 characters, including: (1) What it does (executes Babashka code), (2) When to use it (data processing, code generation, scripting tasks), (3) Prerequisites (Babashka must be installed at $BABASHKA_PATH), (4) Output structure (JSON with stdout, stderr, exitCode fields). Example: 'Execute Babashka (bb) code for data processing or scripting tasks. Requires Babashka installed on the system. Returns structured output: {stdout, stderr, exitCode}. This is an irreversible operation, code can modify files, call APIs, and have side effects.'
Expand 'code' parameter description to 50+ characters with syntax and scope guidance. Example: 'Babashka code to execute. Must be valid Clojure syntax compatible with Babashka. Standard library available; requires explicit imports for external dependencies. Code has full file system and network access.'
Add explicit error recovery guidance in the tool description or in a new 'recovery' field. Example: 'If execution fails, stderr contains the error message. Common causes: syntax errors (invalid Clojure), missing libraries (import), timeouts (code too slow), file not found. Use search_babashka_docs or retry with simplified code.'
Document the output schema in the tool definition. Add a 'outputSchema' field to the ListToolsRequest response: {type: 'object', properties: {stdout: {type: 'string'}, stderr: {type: 'string'}, exitCode: {type: 'integer'}}, required: ['stdout', 'stderr', 'exitCode']}.
Refactor command execution to use child_process.execFile() with an argv array instead of string concatenation. Example: execFile(CONFIG.BABASHKA_PATH, ['-e', request.params.arguments.code], {timeout}) prevents shell injection even with malicious 'code' input.
No output schema documented. Tool returns JSON with 'stdout', 'stderr', 'exitCode' fields, but this structure is not declared in ListToolsRequest response or visible to the LLM. LLMs cannot predict what fields to extract from the result, forcing manual parsing and increasing hallucination risk.
No error recovery guidance. Tool catches exceptions and returns them as JSON with isError:true, but does not document: (a) what errors are retryable, (b) what the user should do when execution fails, or (c) whether stderr content can be parsed for actionable information.
Command injection vulnerability in parameter handling. Code is escaped with '\"' but this is insufficient against shell metacharacters. The tool constructs a shell command string rather than using execFile with argv array. An attacker-controlled 'code' parameter containing backticks, semicolons, or $(…) could break out of the -e argument.
No rate limiting or concurrency control. Multiple concurrent calls to 'execute' with compute-intensive code could overwhelm the system or the underlying Babashka process. No mention of queuing, max concurrent executions, or per-agent call throttling.
Timeout parameter description lacks actionable constraints. States 'Timeout in milliseconds (default: 30000)' with min/max bounds, but does not explain: what happens when timeout is exceeded, whether partial output is returned, or whether the process is killed cleanly.
No idempotency guidance. Executing code like '(slurp "file.txt")' twice produces the same result, but code like '(spit "file.txt" "data")' is destructive. Tool description does not advise on whether re-executing the same code is safe or what the side effects are.
execute
Add concurrency control: implement a queue or semaphore limiting parallel executions to N (e.g., 5). Document in the tool description: 'Concurrent executions are limited to X to prevent resource exhaustion.'
Enhance timeout parameter description: 'Timeout in milliseconds (default: 30000, range: 1000-300000). If execution exceeds this time, the process is terminated and stderr will contain a timeout error. Timeout errors are retryable; consider simplifying the code or increasing the timeout if legitimate.'
Add disclosure in tool description about side effects and idempotency: 'This tool executes arbitrary code. Code can have side effects: modify files, call external services, change state. Re-executing the same code may or may not produce the same result depending on what the code does. For safe exploration, wrap destructive operations in guards (e.g., check file existence before writing).'
Consider adding a 'dry_run' parameter (boolean, optional, default false) if the use case supports it. This would allow the LLM to preview code without executing it. Example: 'If dry_run is true, only return the code without executing, allowing the LLM to review before confirming execution.'
Add explicit permission/scope declaration to the server capabilities: scope 'execute:babashka' or similar. Log all calls with user/agent ID, parameters, result, and timestamp for audit trails.