Loading ToolBench
Universal collaboration protocol + CLI for multi-agent LLM teams
16 tools with complete input schemas and descriptions present. Naming follows verb_noun pattern (collab_*) consistently. Descriptions average ~100 chars, meeting baseline. However, parameter descriptions are minimal (10-30 chars), lacking context on when/why to use tools. No output schemas documented. Error handling absent, tools return success/failure with no recovery guidance. Missing idempotency hints for write operations. No tool annotations (readOnlyHint/destructiveHint). Composition is sound (each tool has one responsibility), but chaining support unclear, output fields not documented to enable downstream tool calls.
Issue a badge for an agent.
Create a conflict record when two agents try to modify the same file.
List unresolved conflicts.
Perform agent handshake: check SHARD, badge, inbox messages, and active tasks for an agent.
Check unread inbox messages for an agent. Returns messages sorted by priority.
Read a specific message and mark it as read.
Send a message to another agent's inbox.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., task_id from collab_task_create, message_id from collab_inbox_send). Breaks tool chaining.
Parameter descriptions are minimal (10-30 chars). E.g., 'Agent ID to check inbox for' lacks context on format, constraints, or when to use. Baseline is 72 chars. LLMs cannot infer valid values.
No error handling or recovery guidance. Tools return success/failure with no actionable next steps. E.g., if collab_task_update fails due to permission, LLM has no guidance on what to try next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Trigger memory compaction: auto-archive old SHARD entries and check L1 sizes.
Get memory layer statistics: SHARD size, L1 files, L2 archives.
Write a memory fragment to L1 memory layer (max 50 lines per file).
List discovered LAN peers (only available when collab node is running).
Update SHARD.md live memory. Requires L3+ permission.
Get the global collaboration status: badges, tasks, inbox, memory stats, and conflicts.
Create a new task.
List tasks with optional filters.
Update a task's status. Only the assignee can change status.
Write operations (collab_inbox_send, collab_task_create, collab_shard_update) lack tool annotations (destructiveHint, idempotentHint). LLMs cannot distinguish safe retries from irreversible actions.
No pagination support for list tools (collab_task_list, collab_conflict_list, collab_peer_list). Large result sets will blow context window. Missing limit, offset, and total_count fields.