Loading ToolBench
Catclaw demonstrates solid tool design with 25 well-structured tools across contacts and memory domains. All tools have descriptions (avg 156 chars) and explicit JSON schemas with typed parameters. Naming follows verb_noun convention consistently (contacts_create, memory_write, kg_query). However, several parameter descriptions lack actionable constraints (e.g., 'platform' accepts free-form strings instead of enum), and output schemas are not documented in the source. Error handling guidance is minimal. Tool composition is strong, each tool has a single responsibility and chains well (e.g., contacts_get returns contact_id for use in contacts_reply). Security considerations are present (approval gates, ai_paused flags) but not explicitly documented as permission requirements.
Pause AI for a contact. While paused, inbound messages from this contact are mirrored to the forward channel but NOT dispatched to the agent. Admin should reply manually.
Resume AI handling for a contact.
Bind a platform user id to a contact. Same contact can have multiple channels (LINE+TG+Discord).
Create a new contact (a person you communicate with on any channel: LINE/Discord/Telegram/Slack). Returns the new contact id. agent_id defaults to the default agent.
Delete a contact and all its channel bindings + drafts (cascade).
Approve a pending outbound draft and trigger send. (Stage 3: full pipeline; Stage 2: marks status only.)
Platform parameter lacks enum constraint. contacts_get, contacts_bind_channel, and contacts_unbind_channel accept 'platform' as free-form string instead of enum ['line','discord','telegram','slack']. LLMs will hallucinate invalid platform names.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns contact + bound channels') but the actual response structure is not specified in the schema. LLMs cannot plan downstream tool calls without knowing response fields.
Error handling lacks recovery guidance. No tool descriptions explain what to do if a call fails (e.g., 'If contact not found, try contacts_list() to discover available contacts'). Agents have no fallback strategy.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 78 | <=2025-11-25 | v2 |
Discard a pending outbound draft.
Send a draft back to the agent for revision with a note (e.g. 'be more empathetic').
List outbound drafts pending approval (or other status). Filter by contact_id and status.
Get a contact by id, or look up by platform user id (e.g. LINE userId). Returns contact + bound channels.
List contacts. Filter by agent_id, role, or tag.
Send a reply to a contact. Goes through the outbound pipeline: draft → mirror to forward channel (if set) → approval gate → send. If approval_required=true the message will be queued and an admin must approve via the work card; if false it sends immediately.
Remove a channel binding by (platform, platform_user_id).
Update a contact. Only provided fields are changed.
Add a fact triple to the knowledge graph (e.g. 'user' 'prefers' 'Rust').
Mark a fact as no longer valid (set expiry date).
Query all known facts about an entity. Verify facts before stating them.
Get facts in chronological order, optionally filtered by entity.
Delete a memory by ID.
List rooms (topics) in a wing with memory counts.
List all wings (agent/project namespaces) with memory counts.
Search memories using hybrid full-text + semantic search. Always search before answering about past work, decisions, or preferences.
Show palace overview: rooms, halls, memory counts, and KG stats for the specified wing.
Find rooms that exist across multiple wings (agents). Tunnels reveal shared knowledge areas — use with cross_wing search to explore them.
Store a memory in the palace. Use halls: facts (objective truths), events (what happened), discoveries (insights), preferences (likes/dislikes), advice (lessons learned). Set importance 8-10 for critical info.
Permission requirements not declared. Tools like contacts_delete and contacts_draft_approve perform sensitive operations but do not state required permissions (e.g., 'Requires admin role'). Audit trail is unclear.
Pagination not implemented. contacts_list and memory_search lack limit/offset or cursor parameters. Large result sets will blow context windows and degrade LLM reasoning.