Loading ToolBench
MCP server for interacting with GitHub repositories, issues, pull requests, and user information
The server defines 5 tools with explicit FastMCP registration and reasonable naming (verb-first: get_, create_). Tool descriptions exist but are terse and lack critical context for agent selection. Input schemas are present with parameter types and descriptions, but lack constraints (enums, ranges, format specs). Output schemas are undocumented, callers don't know what fields to expect. Error handling is minimal: write operations return raw GithubException messages without recovery guidance. Tools combine multiple responsibilities (e.g., create_pull_request_with_branch does branch creation + commit + PR creation). Overall: functional but below production quality. Median baseline for community servers is 45-55; this lands at the high end of that range due to decent naming and basic schema presence, but significant gaps in description depth, output documentation, and error guidance prevent a higher score.
Creates an issue on a GitHub repository.
Creates a new branch and then a pull request on GitHub.
Fetches the content of a file from a GitHub repository.
Recursively fetch all files and directories from a GitHub repo starting at 'path'. Returns a dict with directories as nested dicts and files as strings (file paths).
Fetch Github user Bio
Output schemas are entirely undocumented. Tools return dicts, strings, or GithubException messages with no declared structure. LLMs cannot plan downstream calls or extract required fields (e.g., PR URL, issue URL) without guessing.
Tool descriptions are generic and lack actionable context. 'Fetch Github user Bio' (27 chars) doesn't explain when to use this vs. get_file_content or get_github_files. No mention of prerequisites, return format, or error cases. Baseline for A+ tools is 50-200 chars with explicit WHAT/WHEN/WHY.
create_pull_request_with_branch violates single-responsibility principle. It creates a branch, commits files, and opens a PR in one tool. Should split into separate tools (create_branch, commit_files, create_pull_request) so agents can compose them. Current design prevents partial automation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No input validation or constraint documentation. 'path' parameter in get_file_content assumes 'main' branch hardcoded in URL, fails silently for repos with different default branch. No enum constraints, format specs, or required/optional field clarity in descriptions.
Error handling is opaque. create_pull_request_with_branch and create_github_issue return raw GithubException strings ('error': str(e)) without recovery guidance. LLM receives 'Repository not found' and has no next step. Should categorize errors (retryable, user-fixable, fatal) with actionable guidance.
Parameter 'files_to_commit' in create_pull_request_with_branch is documented as 'Optional dict' but the description doesn't explain format, constraints, or what happens if omitted. Default behavior is unclear, PR created with no commits? Missing field descriptions violate baseline requirement that 100% of A+ tool params have descriptions.
No pagination support in get_github_files. Recursive fetch can return hundreds of files without limit, exhausting context window. Should add max_depth and limit parameters with defaults, and document result constraints.