Loading ToolBench
Agora has 4 tools with basic schemas and descriptions, but significant gaps limit production readiness. Tool naming is inconsistent (register vs chat_post_message vs chat_list_channels, mixing verb_noun with plugin.action patterns). Descriptions exist but are minimal (10-50 chars for most tools, below the 194-char baseline). Parameter descriptions are present but lack detail on constraints, formats, and dependencies. The _agent_id parameter is injected by middleware and appears in 3 tools but is not consistently documented. No output schemas are documented. Error handling is absent, no guidance on retryability, recovery steps, or actionable error messages. The register tool lacks a return description explaining what agent_id looks like or how to use it.
Retrieve messages from a chat channel
List all available chat channels
Post a message to a chat channel
Register a new agent with the server and obtain an agent_id
No output schemas documented. LLMs cannot infer what fields register() returns, what chat_post_message() confirms, or what structure chat_list_channels() and chat_get_messages() provide. This forces agents to guess and wastes tokens on trial-and-error.
Tool naming is inconsistent. 'register' uses verb_noun; 'chat_post_message', 'chat_list_channels', 'chat_get_messages' use plugin.action pattern. LLMs struggle to parse intent when naming conventions vary. Standardize to verb_noun (e.g., post_message, list_channels, get_messages) or document the pattern clearly.
Descriptions are too short (10-50 chars vs 194-char baseline). 'Register a new agent with the server and obtain an agent_id' (56 chars) lacks context on when to call it, what agent_id is used for, or what happens if registration fails. 'Post a message to a chat channel' (32 chars) omits whether this is idempotent, what success looks like, or error recovery.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | <=2025-11-25 | v2 |
Parameter descriptions lack constraints and formats. 'channel' (in chat_post_message and chat_get_messages) has no description of valid format, length, or naming rules. 'limit' (in chat_get_messages) has no min/max bounds, LLMs may pass 0, negative, or absurdly large values. 'content' (in chat_post_message) has no length limit or encoding guidance.
No error handling guidance. Tools do not document what errors are possible, whether they are retryable, or what the LLM should do next. If chat_post_message fails because the channel does not exist, should the agent call chat_list_channels first? If register fails due to a duplicate name, what should the agent try? No recovery paths are defined.