Loading ToolBench
A harness for driving a Godot 4 project from an agent: editor addons, a runtime bridge into the running game, an MCP server in front of them, and a CLI that installs and diagnoses the Godot side.
gdharness defines 23 tools with complete JSON schemas and descriptions. All tools have input schemas with typed parameters and descriptions. However, descriptions are often terse (10-50 chars), falling below the 50-200 char LLM-optimized baseline. Tool names follow verb_noun convention well (editor_status, script_edit, runtime_inspect). Parameter descriptions are present but minimal, many lack guidance on valid ranges, formats, or dependencies. Output schemas are not documented in the tool definitions. Error handling guidance is absent. The server demonstrates solid foundational structure but lacks the depth needed for A-grade production readiness.
Break into the debugger at a breakpoint.
Continue execution after a breakpoint.
Inspect a variable or expression in the debugger.
Read a node's properties, or a scene's structure.
Start, stop, or restart the editor on a project.
The editor's console output, in bursts as it arrives.
Play or stop the scene in the editor.
Set a node's properties.
Tool descriptions are terse (10-50 chars), below the 50-200 char LLM-optimized baseline. Examples: 'The editor's state...' (35 chars), 'Refresh the class cache.' (24 chars). LLMs need context on WHEN to use each tool and what it returns.
Output schemas are not documented. Tool definitions show input schemas but no return type documentation. LLMs cannot plan downstream calls or extract required fields (e.g., what does editor_status return? What fields does runtime_inspect expose?).
Parameter descriptions lack actionable constraints. Examples: 'nodePath' says 'Node path from the scene root' but does not specify format, escaping rules, or error handling for invalid paths. 'limit' in runtime_inspect defaults to 100 but no min/max bounds stated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
The editor's state: whether it is open, what it is playing, what it has open, and what it has said.
Reimport resources or refresh UIDs.
Read the project's structure and settings.
Read or set project settings.
Refresh the class cache.
Read or edit a resource file.
Call a method on a node in the running game.
Send input to the running game.
Read a node's properties from the running game.
Take a screenshot of the running game.
Set a node's properties in the running game.
Wait for the running game to reach a state.
Open a scene in the editor.
Save the open scene.
Create, read, or edit a script.
No error handling guidance. Tools like script_edit (WRITE risk) and runtime_call (WRITE risk) have no documented error cases, recovery steps, or validation rules. LLMs cannot self-correct on failure.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are absent. Risk field is present in metadata but not exposed in MCP tool definitions. LLMs cannot infer which tools are safe to retry or have side effects.