Loading ToolBench
Model Context Protocol server for Mattermost that enables Claude and other AI assistants to interact with Mattermost channels, messages, and users
This server shows significant gaps in definition quality. Tool names follow verb_noun conventions well (list_channels, get_channel_history, post_message), but parameter descriptions are inconsistent. Several tools lack comprehensive descriptions of their outputs, which breaks the agent's ability to plan downstream operations. Input schemas are present and typed, but parameter descriptions are minimal (typically single-line, often generic). The server does not declare output schemas for any tool, forcing LLMs to infer result structure. Error handling guidance is absent, tools fail but don't tell agents what to do next. Risk annotations (destructiveHint, readOnlyHint) are manually tracked in metadata but not exposed in tool schemas.
Add an emoji reaction to a Mattermost message
Get message history from a specific Mattermost channel
Get all replies in a message thread
Get profile information for a specific Mattermost user
List users in Mattermost
List all channels available in Mattermost
Post a message to a Mattermost channel
No output schemas documented for any tool. LLMs cannot plan downstream operations or extract required chaining IDs (channel_id, message_id, user_id). This forces trial-and-error parsing and risks context loss between tool calls.
Tool descriptions are generic and under 100 characters for most tools (baseline 194 chars). Examples: 'List all channels available in Mattermost' (50 chars), 'List users in Mattermost' (24 chars), 'Run the topic monitoring process' (31 chars). Descriptions lack WHEN to call the tool, WHAT it returns, and any prerequisites.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | 2026-07-28+ | v2 |
Reply to a message thread in Mattermost
Run the topic monitoring process to check monitored channels for relevant posts
Parameters lack constraint documentation. 'limit' parameter in mattermost_get_channel_history has no min/max bounds; 'emoji_name' in mattermost_add_reaction provides no guidance on format (with or without colons?); 'offset' and 'limit' in mattermost_get_users lack bounds. LLMs will pass arbitrary values, risking API failures.
Error handling provides no recovery guidance. callToolHandler.ts returns generic error JSON without classifying errors as retryable, user-fixable, or fatal. LLMs receive a bare error message and cannot decide: retry? ask user? abort? Example: 'Channel not found', should the LLM try search_channels() or ask the user to specify a different channel?
Tool annotations (risk: READ_ONLY vs WRITE) are tracked in metadata but not exposed in tool schemas via destructiveHint/idempotentHint. MCP clients cannot distinguish safe tools from destructive ones, agents lack visibility into which calls are irreversible.
mattermost_run_monitoring has an extremely vague description ('Run the topic monitoring process to check monitored channels for relevant posts', 31 chars) and no documented input schema. The tool appears to take no arguments but has no @description, @parameters, or return schema visible. It should declare: What does 'monitoring' mean? What is the output format? What are success/failure criteria?
Destructive tools (mattermost_post_message, mattermost_reply_to_thread, mattermost_add_reaction) lack confirmation or dry-run support. An agent can be tricked via prompt injection into posting unwanted messages. No pattern::confirmation-request or dry_run parameter is present.
mattermost_get_channel_history and mattermost_get_users accept 'limit' parameters but do not return pagination metadata (total_count, next_cursor, has_more). Large datasets cannot be iterated efficiently, agents will repeatedly fetch with increasing offsets and eventually hit API limits or exhaust context.