Loading ToolBench
Adaptive instruction server that renders templated instructions for AI models based on capability tiers (AGI, Engineer, Monkey)
Helmsman defines 4 tools with clear action verbs and documented input schemas. However, tool descriptions lack depth regarding use cases and error recovery. Parameter descriptions are minimal or missing. Output schemas are not documented, critical for agent planning. The project context model and instruction rendering are sophisticated, but the MCP interface underexploits this capability. Tool names follow verb_noun convention but descriptions fall short of LLM-optimization baselines (target 50-200 chars; most here are 100-150 chars but lack actionable 'when to use' guidance). No error handling guidance visible in tool definitions.
Render unified diff showing what instructions change between two capability tiers, including token count comparison
Get tailored instructions for the current model and project context. Returns instructions optimized for the model's capability tier: - AGI: Minimal instructions (expensive tokens, high capability) - Engineer: Balanced instructions - Monkey: Verbose step-by-step guidance (cheap tokens, needs help)
Render a specific skill template from .skills/{name}.j2 with model context
List all available skills from .skills directories (hierarchical search with project override)
Output schemas not documented for any tool. LLMs cannot plan multi-step workflows without knowing what fields to expect (e.g., does 'instructions' return text, or {content, token_count}?). This violates pattern:tool and pattern:response-shaper.
Tool descriptions lack 'WHEN to use' and error recovery guidance. E.g., 'instructions' does not explain when an agent should call it vs. using a static prompt. Missing: 'Call this tool to fetch model-tier-specific instructions for cost/capability optimization. Returns a plain-text instruction string optimized for the specified model tier.' Pattern:recovery-guide not followed.
Parameter constraints missing. E.g., 'model_id' in 'instructions' and 'skill' tools has no format, enum, or regex. No guidance on valid tier names for 'diff_tiers' (tier_a, tier_b). This forces LLMs to guess or pass invalid values. Pattern:constrained-input not followed.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 55 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
No error handling guidance. If 'skill' name not found or 'model_id' is invalid, tool returns an error. Descriptions do not indicate what to do next (e.g., 'Try skills tool to discover available skill names' or 'Ensure model_id is a known Claude version'). Pattern:recovery-guide not followed.
'skills' tool description is 41 characters, below LLM-optimization baseline. Lacks actionable 'when to use' context. Should state: 'Discover all available skill templates (Jinja2 files) in .skills directory. Call this first to see what skills are available before using the skill tool. Returns an array of skill names.'