Loading ToolBench
Handles DMs and communication with AI agents via NATS. Enables sending messages to agents, listing available agents, querying message history, and viewing conversations.
Server has 4 tools with basic schemas and descriptions, but significant gaps prevent production use. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization), and input schemas are present but minimal. dm_agent has a WRITE risk but no confirmation/dry-run pattern. Parameters lack detailed constraints (e.g., agent_name is a free-form string with an enum list in description, not a formal enum). No output schemas documented. Error handling is absent, no guidance on what to do if an agent doesn't exist or NATS connection fails. Tool composition is reasonable (single responsibility), but descriptions are too terse to guide LLM selection effectively.
Send a DM to an agent
List all available agents
View conversation between two users
View messages (all or for specific agent)
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. All tools return 'dict' with no structure specification.
Destructive tool (dm_agent) lacks confirmation/dry-run pattern. Agents can send unintended messages without verification. No error recovery guidance if send fails.
agent_name parameter uses free-form string with enum list in description text, not a formal enum constraint. LLMs cannot parse the valid options reliably and may hallucinate agent names.
Descriptions are 10-50 characters, well below the 50-200 char baseline. Too terse to guide LLM selection. E.g., 'List all available agents' lacks context on when to call it vs other tools.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
No error handling or recovery guidance. If NATS connection fails, agent receives no actionable error. If agent_name doesn't exist, no suggestion of valid alternatives.