Loading ToolBench
MCP server for analyzing PR changes, suggesting templates, integrating GitHub Actions CI/CD events, and sending Slack notifications
This is a collection of educational starter/solution code for an MCP server course, not a production-grade server. The server has 6 tools with moderate definition quality but significant gaps. Tool naming follows conventions (all start with action verbs: analyze_, get_, suggest_, send_). Descriptions exist for all tools but are brief (24-75 chars) and lack context about when to use them or what they return. Input schemas are present with basic types and descriptions for most parameters, but output schemas are entirely undocumented. Error handling is minimal, no recovery guidance or categorization. The code shows a teaching progression across three modules (PR templates, GitHub Actions, Slack), but integration between them is unclear. STDIO-only transport is a hard cap at 50, but definition quality also pulls down the score independently.
Get the full diff and list of changed files in the current git repository.
List available PR templates with their content.
Get recent GitHub Actions events received via webhook.
Get the current status of GitHub Actions workflows.
Send a formatted notification to the team Slack channel.
Let Claude analyze the changes and suggest the most appropriate PR template.
Output schemas completely undocumented. No tool documents what it returns, LLMs cannot plan downstream calls or extract needed fields. Critical for tool composition.
Tool descriptions are too brief (24-75 chars) and lack context. 'Get the full diff and list of changed files in the current git repository' does not explain WHEN to call this vs similar tools, what the return format is, or whether it's safe to retry.
No error handling or recovery guidance in any tool. When git commands fail, subprocess exceptions will be raw, no indication of what the LLM should try next. No categorization of retryable vs fatal errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 29 | - | v1 |
get_pr_templates has NO input parameters documented in schema, function signature shows no required/optional params, but schema description says 'Input: {}'. This is correct (empty schema), but the docstring should clarify that it takes no input and returns a list of available templates.
Parameter naming inconsistency: 'base_branch' (with underscore) vs 'workflow_name' (with underscore), OK, but some parameters like 'message' in send_slack_notification are generic and don't hint at format/length constraints. No max character limit documented despite Slack API constraints.
Tool composition broken: analyze_file_changes returns undefined structure, suggest_template takes 'changes_summary' (free-form string) rather than structured output from analyze_file_changes. LLM must manually parse and reformat.
Destructive tool (send_slack_notification) has no confirmation step or dry-run capability. An agent could spam a Slack channel without any gate. Pattern:confirmation-request not applied.
Prompts feature enabled but not visible in provided code snippets. Cannot assess prompt quality or whether they guide tool usage appropriately.