Loading ToolBench
Unreal Copilot MCP server with skills and analysis for Blueprint/Asset/C++
UnrealCopilot has a moderate definition quality baseline. All 11 tools are explicitly registered with descriptions and clear verb-noun naming (search, get_*, detect_*, trace_*, find_*, list_*, read_*, run_). Input schemas are present and typed for all tools. However, parameter descriptions are sparse or missing context for several tools, output schemas are not documented, and error handling guidance is absent. The tool composition is reasonable (each tool has one responsibility), but descriptions lack the depth needed for confident LLM selection in ambiguous scenarios. The skill-related tools (list_unreal_skill, read_unreal_skill, run_unreal_skill) have minimal descriptions that don't explain when to use them or what dependencies exist. The 'run_unreal_skill' tool (WRITE risk) lacks any discussion of error handling, idempotence, or confirmation patterns.
Detect UE macros (UPROPERTY/UFUNCTION/UCLASS) in a C++ file
Find C++ class usage in Blueprint/Asset + C++ code
Get Blueprint graph (EventGraph/function graphs)
Get details (C++/Blueprint/Asset)
Get inheritance hierarchy (C++ or Blueprint)
Get references (incoming/outgoing/both)
List Unreal skills (SKILL.md summaries)
Read Unreal skill files (default SKILL.md)
Output schemas are not documented for any tool. LLMs cannot infer the shape of responses, forcing them to guess at field names and types. This breaks tool chaining and increases hallucination risk.
run_unreal_skill is a WRITE tool (executes code) but has no error handling, idempotence guarantee, confirmation pattern, or documentation of consequences. This is a safety and auditability gap for an action that modifies state.
run_unreal_skill accepts arbitrary inline Python code ('skill_name' can be 'inline Python code'). No input validation or code injection protection mentioned. This is a security risk if the server runs untrusted agent code without sandboxing.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Run Unreal skill script or inline Python
Unified search across C++/Blueprint/Asset (scope: project/engine/all)
Trace full reference chain (Blueprint/Asset)
Parameter descriptions are sparse for several tools. Examples: get_details 'target' param has no guidance on format (class name? file path? asset path?). find_cpp_class_usage 'cpp_class' says 'Example: ULyraHealthSet' but omits the rule (must match class name exactly? case-sensitive? wildcard-friendly?). This invites LLM misuse.
list_unreal_skill, read_unreal_skill, and run_unreal_skill descriptions lack clarity on when to call each one and in what sequence. No dependency hints. An LLM may call run_unreal_skill before list_unreal_skill, wasting a round-trip.
Potential unbounded result sets for search, get_references, find_cpp_class_usage, and trace_reference_chain. Schema shows some params (max_results=200, max_depth=10) but no guidance on pagination, result limits in description, or what happens when limits are exceeded (truncated? partial result? error?).
No error handling or recovery guidance for any tool. Examples: search with no results, get_details on missing target, read_unreal_skill on nonexistent skill. Descriptions do not tell LLMs what to do when calls fail.
get_details and get_hierarchy descriptions do not specify output format (tree structure? flat list? JSON object with fields?). This forces LLMs to experiment and wastes tokens.
run_unreal_skill has dual responsibilities: execute skill scripts OR inline Python. Per pattern:tool, each tool should do one thing. The description obscures this ambiguity.
trace_reference_chain and find_cpp_class_usage return complex nested structures (reference chains, usage locations) but output schema is not documented. LLMs cannot reliably extract or chain on these results.