Loading ToolBench
Unreal Engine MCP using Unreal Python Remote Execution
Four tools with mixed quality. run_python_code and run_python_file have adequate descriptions (80-100 chars) but lack input schema validation details. get_toolset_schema and execute_tool have good descriptions (150+ chars) explaining prerequisites and usage patterns. However, NO tool declares input parameter types in visible schema definitions, all parameters appear as bare strings/objects without JSON Schema type constraints. Parameter descriptions exist but lack format/constraint details (e.g., 'path' doesn't specify absolute vs relative, 'args' doesn't bound array length). No output schemas documented. Error handling descriptions are absent, tools don't explain what happens on failure or how to recover.
Execute a tool registered in Unreal Editor via ToolsetRegistry. Use the get_toolset_schema tool to discover available tools and their schemas. tool_name should be the short name (e.g. "get_level_info", not the fully qualified "ECABridge.get_level_info"). Check the tool's inputSchema via get_toolset_schema before calling to ensure correct arguments.
Look up registered toolsets and tools in Unreal Editor. Requires UE 5.8+ with the Toolset Registry plugin enabled. Three levels of detail: - No args: list all toolset names and descriptions - toolset only: list all tool names and descriptions in that toolset - toolset + tool_name: get full input/output schema for that tool Always check the schema before calling execute_tool — it tells you exactly what arguments are required.
Execute Python code within Unreal Editor. Use this for custom logic when no ToolsetRegistry tool is available. - Access Unreal API via 'import unreal' - Use the 'unreal-python-stub' resource to get the path to the full API stub (unreal.py) for reference
Execute a Python script file within Unreal Editor. Use this instead of run_python_code when running an existing .py file on disk.
Input parameters lack JSON Schema type definitions. All parameters defined as bare strings/objects without explicit 'type' field, minLength, maxLength, pattern, or enum constraints. Violates pattern:constrained-input.
No output schemas documented. Tools return results but LLMs cannot predict structure (e.g., execute_tool returns JSON with 'result', 'error', 'is_complete' but this is not declared in tool definition). Violates pattern:tool.
Error handling guidance missing. Descriptions do not explain failure modes, recovery steps, or when to retry. E.g., run_python_code can fail with Python exceptions but tool description offers no guidance on interpreting or recovering from errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
Parameter descriptions lack format/constraint details. 'path' parameter in run_python_file does not specify absolute vs relative, no validation rules. 'args' array has no length bounds. Violates pattern:tool-description.
run_python_code and run_python_file are destructive (WRITE risk) but descriptions do not explicitly state this or warn about side effects. Violates pattern:command-tool.