Loading ToolBench
The sandbox-mcp server exposes 15 tools with basic descriptions and input schemas present in the source code. However, the implementation has significant quality gaps that prevent it from being production-grade. Tool names are descriptive and follow verb_noun patterns (health, run, exec, spawn, list, destroy, snapshot_create, snapshot_list, snapshot_delete, env_set, env_get, file_read, file_write, file_list, file_delete). All tools have brief descriptions in the Python source (sandbox_mcp_server.py). Schemas are visible for all tools with typed parameters. However, descriptions are uniformly short (10-50 chars), parameter descriptions are minimal, output schemas are completely absent from the visible source code, error handling is not evident in the tool definitions, and there is no guidance for multi-step tool composition. The server operates over STDIO only, which creates a hard transport ceiling. Tools like 'run', 'exec', 'spawn', 'destroy', 'snapshot_delete', and 'file_delete' perform destructive or high-impact operations but lack confirmation patterns, dry-run support, or detailed error recovery guidance in the visible code.
Destroy a sandbox.
Get an environment variable from a sandbox.
Set an environment variable in a sandbox.
Execute a command in an existing sandbox.
Delete a file from a sandbox.
List files in a sandbox directory.
Read a file from a sandbox.
Write a file to a sandbox.
No output schemas documented in visible source code. Tools like 'run' and 'exec' return stdout/stderr, but the response structure is not documented. LLMs cannot plan downstream tool chains without knowing what fields are available.
Descriptions are uniformly too brief (10-30 characters). The rubric baseline requires descriptions to explain WHAT the tool does, WHEN to use it, and WHAT it returns. Current descriptions like 'Check if a sandbox is healthy and responsive.' (43 chars) lack context for when an LLM should choose this tool over alternatives.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | <=2025-11-25 | v2 |
Check if a sandbox is healthy and responsive.
List all active sandboxes.
Run a command in a temporary sandbox (auto-created).
Create a snapshot of a sandbox for fast boot.
Delete a snapshot.
List available snapshots.
Spawn a persistent sandbox.
Destructive tools (destroy, snapshot_delete, file_delete) lack confirmation or dry-run patterns. No visible error recovery guidance, recovery tools, or undo mechanisms. High risk of irreversible agent mistakes.
No enumerated constraints visible for parameters that should have them. The 'image' parameter in 'run' and 'spawn' accepts free-form strings with no enum, allowing hallucinated container image names.
No error handling patterns visible. Tools like 'exec' and 'file_read' will fail if the sandbox doesn't exist, but there is no guidance for LLM recovery (e.g., 'Try spawn() first' or 'Did you mean one of: ...').
Parameter descriptions are absent or minimal. Parameters like 'timeout' lack units, range constraints (e.g., '1-3600 seconds'), and guidance on default behavior. The 'memory' parameter format is not formally constrained (accepts 'string' with no pattern or enum).
Tool composition guidance missing. The 'spawn' tool and 'run' tool both create/use sandboxes, but it's not clear when to choose spawn (persistent) vs run (temporary). No descriptions guide multi-step workflows.