Loading ToolBench
Multi-agent CLI communication substrate — connect Claude Code, Codex, and Gemini CLI agents in a shared tmux session
14 tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Most tools lack descriptions of what they return, making it difficult for LLMs to plan downstream calls. Parameter descriptions are present but often minimal (10-30 chars). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Error handling guidance is absent. Tool names are reasonable (verb-noun style) but some are ambiguous (Tell vs Ask distinction unclear to LLMs). No pagination support documented despite SubscribeStream suggesting streaming data. Average tool description ~80 chars, below the 194-char baseline for A+ tools.
Send a question from one agent to another, expecting a reply
Authenticate an agent by token and retrieve agent info (name, cli_kind)
Check for replies to a specific ask
Get detailed information about a specific agent
List all registered agents in the broker
Post a reply to a previous ask
Read messages for an agent since a given timestamp
Register a new agent with the broker, returning an authentication token
No output schemas documented. Tools like ListAgents, GetAgent, ReadMessages, CheckReplies lack descriptions of what fields they return. LLMs cannot plan downstream calls or extract required IDs without knowing response structure.
Missing tool annotations. All 14 tools lack readOnlyHint, destructiveHint, and idempotentHint despite clear risk levels (Shutdown=IRREVERSIBLE, RemoveAgent=DESTRUCTIVE, Authenticate=READ_ONLY). LLMs cannot reason about safety without these hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | <=2025-11-25 | v2 |
Remove an agent from the broker
Set an agent's state to idle or busy
Shut down the broker server
Subscribe to a stream of messages from the broker
Send a message from one agent to another (or broadcast if to is null)
Wait for a reply to an ask with a timeout
Ambiguous tool names and overlapping semantics. Tell (broadcast/unicast message) vs Ask (request-reply) distinction is unclear. ReadMessages, CheckReplies, WaitForReply all operate on message state but naming does not clarify the difference or when to use each.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. E.g., Authenticate does not explain what happens on invalid token; RemoveAgent does not explain if removal fails due to active agent.
Parameter descriptions are minimal (10-30 chars). E.g., 'Agent name', 'Message text', 'State value' lack context on format, constraints, or dependencies. Baseline for A+ tools is 72 chars average per parameter.