Loading ToolBench
Real-time Roblox Studio MCP server for AI agents: one WebSocket, an in-Studio agent runtime, push events, live playtest hot-patching, Open Cloud and vision.
Strong foundation with 9 well-defined tools, comprehensive parameter schemas using Zod, and detailed descriptions. Most tools have clear purposes and rich input documentation. However, output schemas are not explicitly documented in the visible code, error handling lacks recovery guidance in descriptions, and some tool names are overly generic ('run', 'observe', 'job'). Tool composition is excellent, tools chain well via IDs and references. Security considerations are present (session isolation, geometry policy validation). Descriptions average ~150-300 chars, meeting the 10-1024 baseline, but lack explicit 'when to use' guidance for similar tools (e.g., 'observe' vs 'look').
Roblox Open Cloud API: read/write assets, places, universes, and creator metadata.
Stream events from the bridge: Monitor events (playtest, input, vision watches, etc.) and journal events (logs, diffs).
Send input to a running playtest: keyboard, mouse, gamepad, or touch events.
Query or cancel a background job (run, observe, playtest, input, events, skills).
Look at the Roblox Studio window through a vision model and get a short TEXT answer instead of an image — the screenshot never enters your context (a 768 px frame is ≈ 500 image tokens on the sidecar, a few hundred characters back to you). One-shot: {question, max_width? (768), region? {x,y,w,h} window px, model?} → {answer, model, provider, captured_ms, model_ms, usage, frame_path}. Ask concrete visual questions: 'is there a red error in the Output panel? quote it', 'where is the Play button (region or px)?', 'is the character standing on the platform or falling?'. The model sees only the screenshot and answers 'not visible' when it cannot tell. Watch: {watch: {question, interval_s (5, min 2; 15 on claude-cli), max_frames (60), stop_when? (substring, or /regex/i), diff_only? (true)}} → {watch_id}. Frames are analysed in the background, one model call at a time; each answer arrives as a Monitor event {type:'vision', watch_id, frame, answer, changed, provider}. With diff_only, frames whose bytes differ < 2% from the last analysed frame are skipped (no model call, no event). The watch ends on max_frames, when the answer matches stop_when, on {stop: watch_id | 'all'}, or after 3 consecutive failures; the last event has done:true and reason. {list: true} shows running watches with counts and the last answer. Prefer observe tree|props|find|player for state — they are exact and free. look is for what only pixels can tell: rendering, layout, UI text, visual glitches, what a playtest looks like. Works with an Anthropic API key (ANTHROPIC_API_KEY / `ant auth login`, ~2 s per look) OR a logged-in Claude Code install (`claude` on PATH, ~10–15 s per look on the subscription); STUDIO_LIVE_VISION_PROVIDER = auto (default) | api | claude-cli. Models: STUDIO_LIVE_VISION_MODEL (look; default claude-opus-5 / sonnet on the CLI) and STUDIO_LIVE_WATCH_MODEL (watch; default claude-sonnet-5 / haiku).
Five tools (input, events, skills, job, cloud) lack visible input schemas in the provided source. Only tool names and brief descriptions are shown; parameter types, constraints, and structures are not documented in the code excerpt.
Tool names 'run', 'observe', and 'job' are generic and do not clearly convey action. 'run' could mean execute, test, or deploy. 'observe' is passive and vague. 'job' is ambiguous (query vs cancel). Recommend: 'execute_luau', 'query_studio_state', 'manage_job'.
Output schemas are not documented in tool descriptions. Agents cannot predict what fields to expect from 'run', 'observe', 'playtest', or 'look' responses. This forces trial-and-error parsing and risks context loss.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Query the Studio DataModel state: status, tree, props, find, diff, logs, script, geometry, stats, selection, player, screenshot, windows, selftest.
Control playtests: start, stop, status, run_until, install/uninstall controllers, list, hotpatch, add_players, push instances.
Execute Luau code in the Studio DataModel (edit, server, or client). Globals: S (resident API), ARGS, print/warn (captured), game, workspace, task. A top-level `return` is captured.
Manage agent skills: list, install, uninstall, run.
Error handling descriptions lack recovery guidance. E.g., 'run' mentions 'geometry_violation' but does not explain how to fix it or which tool to call next. Descriptions should include: 'If geometry_violation occurs, call observe geometry to inspect overlaps, then adjust part positions.'
Tool 'input' has only a one-line description ('Send input to a running playtest: keyboard, mouse, gamepad, or touch events.') with no parameter documentation visible. Cannot assess schema quality or parameter constraints.