Loading ToolBench
A Model Context Protocol server that executes JavaScript code in an isolated environment
Single tool 'execute_js' has a well-structured schema with proper type definitions and reasonable parameter constraints (timeout 100-30000ms, memory 1MB-100MB). Description is adequate (58 chars) but generic. Parameters are typed and constrained but lack depth in descriptions. No output schema documented. Error handling exists but lacks recovery guidance. Tool is irreversible (code execution) but no confirmation/dry-run pattern implemented. Naming is clear and verb-based.
Execute JavaScript code in an isolated environment
No output schema documented. LLMs cannot predict return structure (result, console, executionTime, memoryUsage fields). Breaks downstream tool chaining and forces LLMs to guess field names.
Irreversible operation (code execution) lacks confirmation/dry-run pattern. Agents cannot preview execution results before committing. No recovery guidance if code has side effects.
Parameter descriptions are minimal (10-30 chars). 'Maximum execution time in milliseconds' is clear, but 'Memory limit in bytes' lacks guidance on when to adjust or what happens if exceeded. Baseline is 72 chars per param.
Error messages are generic ('Execution error: ...') and do not guide recovery. No distinction between retryable errors (timeout) vs fatal errors (syntax). LLM cannot determine next action.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Tool description does not state irreversibility or consequences. LLMs need explicit warnings: 'WARNING: Code execution is irreversible. Side effects (file writes, network calls) cannot be undone.'