Loading ToolBench
Generic MCP server + Kit socket bridge for Omniverse Kit / Isaac Sim
Kit-MCP exposes 8 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-100 chars, mostly adequate) and input schemas with type definitions. However, output schemas are entirely undocumented, LLMs cannot predict return structures for chaining. Parameter descriptions are minimal (often 1-2 sentences). Error handling is absent: no guidance on retryability, recovery steps, or actionable error messages. The run_python tool is dangerously permissive (arbitrary code execution) with no confirmation or dry-run pattern. File operations (put_file, get_file) lack validation constraints (path traversal, size limits). Tool naming is generally clear (verb_noun style: add_cube, stat_file, get_file) but lacks context about side effects in descriptions. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels assigned. Average tool score: 52/100.
Spawn a cube prim under /World/Objects in the live stage.
Fetch a file from the bridge box (base64 encoded).
List all registered tools with their schemas (builtin).
Liveness check: echoes its message back with the namespace.
Write a file to the bridge box (base64 encoded content).
Hot-reload one or more tool packages (builtin).
Execute arbitrary Python code on the Kit process (builtin).
No output schemas documented. LLMs cannot predict return structures, breaking tool chaining and forcing agents to guess field names. list_tools, stat_file, get_file, add_cube all return data but schema is invisible.
run_python accepts arbitrary code with no confirmation, dry-run, or sandboxing. IRREVERSIBLE risk level demands a confirmation_request pattern or explicit safety guardrails.
File operations (put_file, get_file) lack input validation: no path traversal guards, no size limits, no allowed-directory constraints. Agents could write/read arbitrary files.
No error handling guidance. Tools return success/failure but do not tell LLMs what to do next: retry? ask user? abort? No recovery paths documented.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
Verify file existence, size, and SHA256 hash on the bridge box (builtin).
Tool annotations missing. Despite clear risk levels (READ_ONLY, WRITE, IRREVERSIBLE), no readOnlyHint, destructiveHint, or idempotentHint in schemas. LLMs cannot infer safety properties.