Loading ToolBench
Multi-Agent Coordination System - MCP Server Component for managing agents across multiple communication channels (Slack, Discord, SignalR, Redis)
CoorChat MCP server has 16 tools with basic schemas and descriptions, but significant gaps prevent production use. Token tools are well-named (verb_noun pattern) but many command tools use generic names (command_help, command_ping). Descriptions exist but are often generic (e.g., 'Display available commands' lacks context on when to call). No output schemas documented. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. Security concerns: webhook_handle accepts raw payloads without documented validation. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Average tool description ~80 chars (below 194-char baseline), and parameter descriptions are sparse or missing entirely.
List all active agents (requires shared state store like Redis)
Start an agent with specified ID, role, and channel type
Display available commands for the agent
Check if agent is responsive
Show agent status and uptime information
Display agent identity and role information
Initialize configuration file with environment variables for specified channel type
Display current configuration including channel type, agent settings, and integration credentials (masked)
Missing parameter descriptions on 8+ tools. Rubric requires 100% of A+ tools have param descriptions. agent_list, role_list, config_show, webhook_stop, and all command_* tools have empty input objects with no guidance on what they control or return.
No output schemas documented for any tool. Rubric baseline: 100% of A+ tools have documented return types. LLMs cannot plan downstream calls or extract required fields without knowing response structure.
Generic command tool names (command_help, command_ping, command_status, command_whoami) lack action clarity. Rubric: tool names should convey what happens when called. These names suggest CLI commands, not agent actions. Consider rename to agent_help, agent_ping, agent_status, agent_whoami or merge into single agent_info tool.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | <=2025-11-25 | v2 |
List all available predefined roles with their descriptions and capabilities
Suggest roles based on provided capabilities (tools and languages)
Generate a secure token for channel, API, or webhook authentication
Hash a token using SHA-256 algorithm
Validate token format and check requirements (minimum 16 characters, alphanumeric with _ and -, no whitespace)
Handle incoming GitHub webhook events (issues, pull requests, push) with signature validation
Start the webhook server to listen for GitHub events
Stop the webhook server
No error handling or recovery guidance. webhook_handle accepts GitHub payloads but provides no documentation of validation, error cases, or what to do if signature validation fails. Rubric requires: error responses must tell LLM what to do next.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk levels. agent_start, config_init, webhook_handle, webhook_start are WRITE operations but lack destructiveHint. Rubric: tool annotations enable least-privilege agent configs.