Loading ToolBench
MCP stdio bridge for Nemeton. Reads JSON-RPC 2.0 from stdin, forwards each request to Nemeton's local HTTP server, writes responses to stdout. Provides tool surface discovery via static snapshot when app is unreachable.
Nemeton provides 30 tools with consistent naming (verb_noun pattern) and basic schemas. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization). Input schemas are present but minimal, most parameters lack detailed descriptions beyond the parameter name itself. No output schemas documented. Error handling and recovery guidance absent. Tool composition is reasonable (single responsibility per tool), but parameter descriptions are sparse, making it difficult for LLMs to understand constraints, formats, and valid ranges. The server reads from a baked-in static snapshot when offline, indicating tool definitions are hardcoded rather than dynamically introspected.
Clone an existing VM, optionally under a new name.
Run a command via the VM console and return its output.
Create a copy-on-write snapshot of a VM.
Create a new VM from a Linux distribution image.
Delete a VM snapshot.
Delete a VM and its disk image.
Download a file from a VM to the host.
List files in a directory inside a VM.
Parameter descriptions are minimal or missing. Most parameters have only a 1-3 word description (e.g., 'VM name or identifier'). LLMs need format constraints, valid ranges, and usage context. Example: 'cpu' parameter lacks min/max bounds; 'distro' lacks enum of valid distributions.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls without knowing response structure. Example: create_vm likely returns a VM object with id, name, state, but this is not specified.
No error handling or recovery guidance. Tools lack descriptions of failure modes, retryability, or what the LLM should do if a call fails. Example: delete_vm offers no guidance on what happens if the VM is running or in use.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Upload a file from the host into a VM.
Immediately power off a VM.
Get the suspend/resume status of a VM.
Get configuration and current state of a single VM.
Launch a GUI application inside a VM.
List open GUI windows in a VM.
List APFS copy-on-write snapshots of a VM.
List all configured virtual machines with their current state.
Pause a running VM.
Read recent output from a VM's serial console.
Reorder the VM list.
Restore a VM to a previous snapshot.
Resume a paused VM.
Send text or keystrokes to a VM's serial console.
Run a command inside a VM over SSH.
Start a VM.
Stop all running VMs.
Gracefully shut down a running VM.
Suspend a VM to disk.
Capture a screenshot of a VM's display.
Update a VM's configuration (CPU, memory, etc.).
Get the IP address of a running VM.
Destructive operations (delete_vm, delete_snapshot, stop_all) lack confirmation or dry-run support. Agents can irreversibly delete VMs without a safety gate.
Tool descriptions are too brief (10-50 chars vs. baseline 50-200 chars). Descriptions like 'List all configured virtual machines with their current state' lack context on when to use this tool vs. get_vm, or what fields are returned.