Loading ToolBench
RimWorldForge defines 11 tools with complete JSON schemas and basic descriptions. All tools have input schemas with proper type definitions and required fields. However, descriptions are terse (avg 120 chars, well below the 194-char baseline), lack WHEN/WHY context, and parameter descriptions are entirely absent. No output schemas documented. Error handling returns JSON but lacks recovery guidance. Tool names follow verb_noun convention (rw_*) but are domain-specific and could be clearer for LLM selection. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications in metadata.
Build a clean mod folder + deterministic ZIP + SHA-256 receipt after validation.
Return honest implemented/unsupported capability matrix and evidence tiers.
Inspect one indexed Def and return its source plus raw XML when available.
Search the indexed vanilla/DLC Def corpus before inventing XML.
Detect RimWorld, DLCs, Managed assemblies, Mods folder and Player.log.
Apply a validated generation plan to a Forge workspace: About.xml, Def XML, LoadFolders, asset manifest.
Build a searchable Def index from Core + installed DLC. By default auto-discovers mods: game Mods folder plus every Steam library Workshop root (override with RIMWORLD_WORKSHOP_ROOT/RIMWORLD_MODS_ROOT env or mods_root).
No parameter descriptions. All 11 tools have input schemas with typed properties but zero descriptions for parameters (e.g., 'game', 'plan', 'workspace'). LLMs cannot infer what 'game' means (game name? path? ID?) or what format 'plan' expects without explicit guidance.
No output schemas documented. Tools return JSON with 'ok' flag and various fields (results, def, error) but no formal schema definition. LLMs cannot plan downstream calls or extract required fields (e.g., does rw_def_search return 'def_name' or 'name'?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Analyze a RimWorld Player.log for XML, cross-reference, assembly, Harmony, texture and exception failures.
Validate a typed RimWorldForge generation plan without writing files.
Create a transactional RimWorld mod workspace outside the live game.
Static-validate About.xml, Def/Patch XML, duplicate names, texture paths and conservative reference candidates.
Tool descriptions lack WHEN/WHY context. Descriptions state WHAT (e.g., 'Build a searchable Def index') but not WHEN to call it or how it differs from similar tools. Average 120 chars vs 194-char baseline. No guidance on prerequisites or dependencies.
No tool annotations despite clear risk metadata. Metadata identifies rw_index, rw_project_new, rw_generate, rw_build as WRITE operations and others as READ_ONLY, but no readOnlyHint/destructiveHint annotations in tool definitions. LLMs cannot distinguish safe from risky calls.
Error handling lacks recovery guidance. Errors return JSON with 'error' and 'error_type' but no actionable next steps. E.g., 'RimWorld installation not found' should suggest 'Set RIMWORLD_GAME_ROOT env var or pass game parameter'.