Loading ToolBench
MCP server for fetching and analyzing Rollbar errors from Slack
The server defines 3 tools with basic schemas and descriptions, but falls short of production-grade quality. Tool names follow verb-noun convention (fetch, analyze, post), but descriptions lack LLM-optimized depth and fail to explain when/why to use each tool or their dependencies. Schemas are present but minimal, parameters have types and basic descriptions, but lack constraint declarations (enums, ranges, patterns). Output schemas are not documented. Error handling is generic ('Error: {message}') with no recovery guidance. The 'analyze_errors_and_suggest_actions' tool accepts a raw JSON string rather than structured input, violating the principle of constrained input. No validation of required environment variables (SLACK_BOT_TOKEN) until runtime, risking agent confusion. The composition reveals a linear workflow (fetch → analyze → post) but tools are not designed to guide agents through this flow, descriptions don't mention prerequisites or downstream use.
Analyze a list of Rollbar errors and generate a prioritized action plan with specific recommendations. Takes the JSON output from fetch_rollbar_errors.
Fetch Rollbar errors posted to a Slack channel within a specified timeframe. Returns structured error data including severity, occurrences, and links.
Post a todo list or action plan to a Slack channel with optional @mentions to notify team members.
analyze_errors_and_suggest_actions accepts 'errors' as a free-form JSON string instead of structured input. This violates the constrained-input pattern and forces LLM serialization, risking malformed JSON and silent errors.
Tool descriptions lack WHEN/WHY context and do not document tool dependencies. 'analyze_errors_and_suggest_actions' should explicitly state 'Requires output from fetch_rollbar_errors' to guide agent composition.
No output schemas documented for any tool. LLM cannot predict response structure, required fields, or data types for downstream tool calls. Required for tool chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Error messages are generic ('Error: {message}') and provide no recovery guidance. Tool should return actionable errors like 'Slack token not set. Configure SLACK_BOT_TOKEN environment variable' or 'Channel "xyz" not found. Available: #discuss, #alerts.'
Parameter 'mention_users' in post_todo_list_to_slack expects comma-separated usernames but lacks format documentation and validation. Should clarify: use Slack handles (no @)? Are spaces allowed? Should validate and return actionable errors on invalid handles.
Tool names use 'and' (analyze_errors_AND_suggest_actions), signaling multiple responsibilities. Per Arcade patterns, should split into analyze_errors + suggest_actions for single concern separation.
Parameter 'hours' in fetch_rollbar_errors lacks range constraints. Should document: 'Number of hours to look back (1-365, default 24)' to prevent LLM from passing invalid values like 0, -1, or 10000.
No pagination support in fetch_rollbar_errors. If error list exceeds context limits, LLM receives all errors at once, wasting tokens. Should support limit and offset/cursor parameters.