Loading ToolBench
A Model Context Protocol server for interacting with Slack workspaces
Server has 10 tools with complete input schemas and descriptions, but quality is inconsistent. Naming follows verb_noun convention well (get_, send_, list_, add_, reply_). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Critical gaps: no output schemas documented, no error handling guidance, no parameter validation rules stated, and parameter descriptions lack format/constraint details. Tool composition has issues: get_channel_messages_on_slack and get_channel_history_on_slack are near-duplicates (both fetch messages, differ only in naming). anyOf constraints for channel_id/channel_name are present but not explained in descriptions. No mention of rate limits, pagination limits, or data loss risks. Security: SLACK_BOT_TOKEN exposed in Dockerfile as dummy default, though this is mitigated by requiring env override.
Add an emoji reaction to a message in Slack. Provide either channel_id or channel_name.
List public channels in the workspace with pagination
Get recent messages from a channel. Provide either channel_id or channel_name.
Fetch the last N messages from a Slack channel (by channel_id or channel_name).
Fetch recent messages where the bot/user is mentioned (optionally by channel).
Get all replies to a specific message thread in Slack. Provide either channel_id or channel_name.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., channel_id, user_id, thread_ts) from responses. Baseline: 100% of A+ tools document return types.
Duplicate/overlapping tools: get_channel_messages_on_slack and get_channel_history_on_slack both fetch messages from a channel. Names do not clearly distinguish intent. LLMs will waste reasoning cycles deciding between them.
Parameter descriptions lack format/constraint details. E.g., 'thread_ts' says 'format 1234567890.123456' but does not explain what happens if format is wrong. 'reaction' says 'without colons' but no validation guidance. Baseline: descriptions should state format, range, and allowed values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Get the profile information of a specific user in Slack
List users in the Slack workspace with pagination
Reply to a specific message thread in Slack. Provide either channel_id or channel_name.
Post a new message to a Slack channel. Provide either channel_id or channel_name.
No error handling guidance. Tool descriptions do not explain what to do if a channel is not found, user does not exist, or API rate limit is hit. Baseline: error responses must tell LLM what to do next.
anyOf constraints for channel_id/channel_name are present in schemas but not explained in parameter descriptions. LLMs may not understand that exactly one is required, leading to ambiguous calls.