Loading ToolBench
Agent-native semantic control plane for Dwarf Fortress
Seven tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling guidance. Tool names follow verb_noun convention (fortress_*), but descriptions are terse (avg ~80 chars, below 194-char baseline). Most parameters lack type constraints and detailed descriptions. No evidence of error recovery guidance, confirmation patterns for destructive ops, or output schema documentation. Schemas are present but incomplete, many parameters lack 'type' and 'description' fields. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels declared in metadata.
Commits a previously planned operation by providing the plan digest, executing the sealed plan
Performs health checks and diagnostics on the fortress control plane and session state
Provides explanations and diagnostics about fortress state or recent operations
Opens a new session with the fortress control plane, establishing permissions and session context
Creates an immutable plan for fortress operations, returning a plan digest for later commitment
Queries the current state of the fortress world, returning structured data about the game state
Waits for a pending task to complete and returns the task identifier
Output schemas not documented. No tool returns a documented structure, LLMs cannot plan downstream calls or extract required fields. fortress_open_session returns session_id; fortress_plan returns plan_digest; fortress_query returns fortress state, but none are formally specified.
Parameter descriptions are generic or missing. 'query_type' in fortress_query says 'The type of query to execute (e.g., summary, detailed, spatial)', no enum constraint, no guidance on which to use when. 'permissions' in fortress_open_session is described as 'List of [capability, permission_level] tuples' but lacks format detail, valid capability names, or permission_level values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
No error recovery guidance. fortress_commit accepts a plan_digest but provides no error message if the digest is invalid, expired, or already committed. fortress_plan offers dry_run but no confirmation pattern for irreversible commits. Agents cannot self-correct on failure.
Descriptions too brief to guide LLM selection. fortress_wait: 'Waits for a pending task to complete and returns the task identifier' (11 words). fortress_explain: 'Provides explanations and diagnostics about fortress state or recent operations' (12 words). Below 20-char threshold; cannot disambiguate from similar tools or explain when to call.
No tool annotations despite declared risk levels. fortress_commit is WRITE; fortress_plan is REVERSIBLE; fortress_query is READ_ONLY, but no readOnlyHint, destructiveHint, or idempotentHint annotations in tool definitions. Agents cannot reason about safety or retry semantics.