Loading ToolBench
claw-pilot defines 14 tools with mixed quality. Strengths: all tools have descriptions (50-500+ chars), most parameters have types and descriptions, input schemas are present with constraints (enums, min/max). Weaknesses: no visible output schemas documented in the definitions; descriptions lack context on when/why to use tools; parameters often undescribed or generic; no structured error handling guidance; several tools conflate multiple concerns (instance management + runtime control); missing LLM-optimization patterns in naming and descriptions. The server is clearly designed for a multi-agent orchestration system, but the tool interface does not follow chat-data-model patterns, names and parameters use internal terminology (slug, namedKeyId) instead of user-friendly identifiers. STDIO transport limits remote accessibility.
Report mission completion for this flow step. Call this tool as your FINAL action — calling it is MANDATORY. The flow engine records the structured SITREP from your arguments. Without this call, your step is marked as failed regardless of what other work you performed.
Create a new claw-pilot instance. Requires a slug, default model, and named API key ID. Optionally provide agent definitions.
Delete an instance and all its data. Cannot delete the system instance (cp-system).
Get detailed information about a specific instance by its slug.
List all claw-pilot instances with their status, port, and model. Returns an array of instance objects.
Start a stopped instance runtime daemon.
Output schemas not documented. Tool descriptions define inputs (parameters) but never describe what fields the agent receives in return. LLMs cannot plan downstream tool calls or extract returned values without documented output structure.
Instance management tools use opaque internal identifiers (slug, namedKeyId) as parameters instead of user-friendly names. Users do not think in slugs, they think in instance names. Requires agents to perform extra lookup calls or map user intent to slugs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Search the agent's long-term memory (MEMORY.md and memory/*.md files) using full-text search. Returns the most relevant excerpts for the query. Use this instead of reading MEMORY.md in full when looking for specific information such as past decisions, key files, or ongoing tasks.
Request additional LLM steps to continue your mission. Call this when you receive the steps-limit warning and need more steps to complete your work (e.g. you still have files to edit, commits to push, or a PR to open). The engine will grant the extension up to a hard cap. After the extension, continue your work and call `complete_step` when done.
Delete a file from your own agent workspace. Identity files (SOUL.md, IDENTITY.md, AGENTS.md) and any glob in your protected-paths list are refused. The allowed-paths whitelist also applies.
Delete a file from the instance shared workspace. Identity files and protected paths are refused.
List user-created files in your workspace AND in the instance shared workspace (shared read/write with all agents of the instance). Entries from the shared workspace are prefixed with '@shared/'. Returns each file's path, size, and a short title extracted from the first H1 heading or frontmatter 'description:' value. Does NOT include identity files (SOUL.md, AGENTS.md, BOOTSTRAP.md, USER.md, HEARTBEAT.md, MEMORY.md) or memory files (memory/*.md) — those are handled separately by the system prompt and memory subsystem. Use this tool when the user references notes, documents, drafts, or project files they may have stored in the workspace, before asking them to repeat information you could find yourself. Pass dir to scope the listing (supports '@shared/<path>' to scope shared).
Full-text search over user-created files in your workspace and the shared workspace. Returns excerpts from matching files with context.
Create or overwrite a file in your own agent workspace. Path must be workspace-relative (e.g. 'notes/today.md'). Identity files (SOUL.md, IDENTITY.md, AGENTS.md) and any glob in your agent's protected-paths list are refused. If an allowed-paths whitelist is configured, the path must match at least one of its globs. Allowed extensions: .md, .txt, .json, .yaml, .yml, .csv, .log. Maximum 1 MB per file. A per-agent byte quota may further restrict writes.
Create or overwrite a file in the instance shared workspace (shared read/write with all agents of the instance). Path must be workspace-relative and start with '@shared/' (e.g. '@shared/notes/team.md'). Identity files and protected paths are refused. Allowed extensions: .md, .txt, .json, .yaml, .yml, .csv, .log. Maximum 1 MB per file.
No error handling guidance. Tool descriptions do not indicate what can fail, which errors are retryable, or how an agent should recover. E.g., 'cp_delete_instance' is destructive but offers no confirmation step or dry-run capability.
Instance management tool descriptions are generic and lack LLM guidance. 'List all claw-pilot instances with their status, port, and model' does not explain WHEN to call this (discovery), what to do with the results, or whether pagination is needed.
Tool composition mixes concerns. cp_create_instance accepts both instance config AND agent definitions in a single call. This should split: create_instance (config only) + add_agents_to_instance (agents). Overly broad tools reduce composability.
Workspace tools describe file restrictions (protected paths, allowed extensions, size limits) but do not guide what happens on violation. LLM receives a failure but no actionable recovery path (e.g., 'try a different path matching allowed-paths glob').
Parameters lack usage examples embedded in descriptions. For instance, 'path' in ws_write_file says 'workspace-relative path' but does not clarify format (e.g., 'notes/today.md' not '/absolute/notes/today.md'). LLMs infer formats from descriptions alone.