Loading ToolBench
Reuse your AI stuff across the agent stack: COMPLETE skills into agent definitions and REALIZE them into running agents (glue over skill, aw, py2mcp).
coact exposes 11 tools with basic descriptions but significant quality gaps. Tool names are reasonably verb-forward (plan, complete, emit, realize, diff, estimate, inventory, back, scaffold, publish, describe), but descriptions are inconsistent in depth and actionability. Most tools lack detailed parameter descriptions, many parameters have only a brief phrase (e.g., 'Skill source path or identifier') without format constraints, valid ranges, or error recovery guidance. Input schemas are present but minimal: most parameters are simple strings or booleans with no enum constraints, regex patterns, or validation hints. Output schemas are not documented at all, LLMs cannot predict what these tools return or chain results to downstream calls. Error handling is absent from descriptions; no guidance on retryability, user-fixable vs. fatal errors, or recovery paths. The 'realize' tool accepts 6 parameters with complex interdependencies (backend, dest, scope, skills_source, dry_run) but these relationships are not documented. The 'publish' tool's 'source' parameter accepts 'Tool refs' but the format is never specified. Overall, the tools read as CLI wrappers rather than LLM-optimized interfaces.
Best-effort, LOSSY agent→skill extraction (prints a SKILL.md stub).
Complete a skill into an agent definition (print the .md, or write to --dest).
Refine an NL description into a draft IntegrationSpec (opt-in LLM, aix-backed).
Show what extras an agent adds over its source skill (the §3.2 table).
Complete a skill and emit it to a target (claude-agents-md, sdk-agent-dict, ...).
Surface the fan-out token multiplier for an agent set (the cost gate).
Enumerate a project's skills, derived agents, and MCP-exposed tools.
No output schemas documented. LLMs cannot predict return types, field names, or structure. Breaks tool chaining and forces agents to guess what data is available.
Parameter descriptions are minimal (10-30 chars) and lack format constraints, valid ranges, or examples. E.g., 'Skill source path or identifier' does not specify file extension, directory structure, or how to distinguish a path from an identifier.
No enum constraints on string parameters. 'target' in emit() accepts arbitrary strings; valid values (claude-agents-md, sdk-agent-dict, etc.) are mentioned in description but not enforced. LLMs will hallucinate invalid targets.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Dry-run the skill→agent completion: show each field + its provenance.
Publish a capability to a chatbot host (default: a local Claude Desktop .mcpb bundle).
Realize an agent/skill via a backend (host materializes files; sdk/mcp build runnables).
Emit a STARTER multi-agent fleet shim you own (sequential; D8 — not a topology engine).
Complex parameter interdependencies undocumented. 'realize' has 6 params (backend, dest, scope, skills_source, dry_run) with unclear relationships. Which combinations are valid? Does dry_run only work with backend='host'? Agents cannot reason about this.
No error handling guidance. Descriptions do not explain failure modes, retryability, or recovery paths. E.g., 'complete' could fail if skill file is malformed, but LLMs have no guidance on what to do next.