Loading ToolBench
Query the running engine for real type signatures, members, enum values, input actions and console commands, resolve content paths before they silently load the error model, read project, package and compiler state including assembly staleness, list compile errors, reload an externally edited .sbproj or ProjectSettings config, and rebuild from source on disk.
Strong tool definitions with excellent descriptions (avg 180 chars, well above baseline 72 chars minimum). All 14 tools have clear, actionable descriptions explaining WHAT, WHEN, and WHY. Input schemas are complete with proper types and enums. However, output schemas are not explicitly documented in the source code, only inferred from return type names. Parameter descriptions are consistently present and detailed. Tool names follow verb_noun pattern well (project_find_type, project_compile_errors). No security issues detected (read-only operations, no secrets in params). Main gap: output schema documentation and error handling guidance.
Compare what each compiler last built against what the process has actually loaded. Recompiling does not always cure a stale assembly: the editor goes on serving the version it loaded, and compile_status reads green the whole time.
Return current compile diagnostics as structured rows with file and line, so errors can be read without scraping read_console. Errors sort first.
List the project's compilers with their build state. A compiler sitting at NeedsBuild true while IsBuilding is false has work queued that nothing has started, which is what a stalled build looks like from the outside.
List the console commands and convars the engine currently knows, optionally only the ones this project's own assemblies registered. A console command given the wrong argument form prints its usage and changes nothing, which from the outside is indistinguishable from having run, so check the real name and shape here before driving anything through console_command.
Resolve one or more content paths against everything currently mounted, applying the same _c suffix rule the engine applies when it loads a resource. A path that resolves to nothing does not throw at runtime: Model.Load hands back the engine's error model, so a typo in a .item or a prefab compiles clean, passes every headless test and ships an orange world.
Output schemas not explicitly documented in source code. Return types are inferred from method signatures (ProjectInfo, CompilerList, etc.) but no formal JSON Schema definitions visible for LLM consumption.
No error handling guidance in tool descriptions. Tools like project_find_type and project_content_path do not document what happens on failure (e.g., type not found, path invalid) or how the LLM should recover.
Tools with empty input schemas (project_input_actions, project_console_commands, project_info, project_assembly_freshness) lack parameter documentation, though they accept no parameters. This is acceptable but could benefit from explicit 'no parameters required' clarification.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
List files under a directory in the mounted content filesystem. This is the "then what IS the right path" companion to project_content_path: mounted packages routinely disagree with their own CDN manifests about the path prefix, and the manifest spelling fails at runtime with no symptom at all.
List an enum's named values with their numeric values. Enums have no methods or properties, so project_type_members reports one as empty, which reads as "the type does not exist" when it means "wrong tool".
Search every loaded type for members whose name contains a fragment, and report which type each one is declared on. Reach for this when you know roughly what a method is called but not what it hangs off, which is the case project_type_members cannot answer because it needs the exact type name up front.
Search the running engine for a type by name and report what it is. Ask this before writing an API you are not certain about.
Report which project the editor has open, where it sits on disk, and the compiler settings currently live in memory. Start here when an on-disk change is not taking effect: the settings reported are what Roslyn is actually using, which is not necessarily what the .sbproj on disk now says.
List the input actions this project defines, with their keyboard and gamepad bindings. Input actions are strings resolved at runtime, so Input.Down( "jump" ) on an action that does not exist compiles cleanly and silently never fires.
Reconcile the package references written in the .sbproj against what is actually installed in the cloud cache. install_package mounts a package for this session and writes nothing to the project, so a package installed over MCP works perfectly until the editor restarts and then is simply gone.
Ask each compiler what source changes it has actually noticed since its last build. This is the direct answer to "did my edit register", and it separates a file the compiler never saw from a file it saw and rejected.
List a type's methods and properties with real signatures, read from the running engine, and mark anything carrying [Obsolete]. This is the ground truth an API reference is only an approximation of, so prefer it whenever the two might disagree, and always for a type you are about to call something unfamiliar on.
Pagination not implemented for tools returning lists (project_find_type, project_find_member, project_console_commands, project_content_search). Limits are enforced (default 20-60, max 500) but no cursor/offset or total_count fields documented for chaining.
Tool names use 'project_' prefix consistently, which is good for grouping but slightly verbose. Consider whether shorter names (e.g., 'find_type', 'compile_errors') would improve discoverability without losing context.