Loading ToolBench
An MCP server that exposes GitHub repository data as tools an AI assistant can call, providing access to repository metadata, issues, pull requests, commits, and CI/build status.
Six tools with clear verb-noun naming and reasonable descriptions (avg 180 chars, within baseline 194). All tools have input schemas with typed parameters and descriptions. However, output schemas are undocumented, responses are JSON-stringified text blocks with no structured field definitions. Error handling is present but generic ('Error fetching X: {message}'). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Parameter descriptions lack format/constraint details (e.g., 'ref' in get_commit_status accepts 'commit SHA, branch name, or tag' but no regex or format spec). No pagination guidance despite list_* tools capping at 50 items. Missing dependency hints and recovery guidance in error messages.
Get the combined CI/build status for a specific commit, branch, or tag (e.g. 'main', a commit SHA). Returns overall state (success/failure/pending) plus individual check results. Use this to check if a build passed.
Get the code diff for a specific pull request by number, truncated if very large. Use this when you need to see the actual code changes, not just PR metadata — for example, to review or summarize what a PR changes.
Get basic information about a GitHub repository, including stars, forks, open issue count, primary language, and description.
List open issues (not pull requests) for a GitHub repository, including title, author, labels, and comment count. Use this to see what bugs or feature requests are currently outstanding.
List pull requests for a GitHub repository, filterable by state (open, closed, or all). Returns title, author, branches, and draft status. Use this to see what's being worked on or what recently merged/closed.
Output schemas undocumented. All tools return JSON-stringified text in a single 'text' content block. LLMs cannot parse structured fields (e.g., 'stars', 'forks', 'author', 'status') without additional reasoning. Responses should declare field types and structure.
Parameter 'ref' in get_commit_status lacks format specification. Description says 'commit SHA, branch name, or tag' but no regex, length, or example format. LLMs may pass invalid refs.
Error messages are generic ('Error fetching X: {message}'). No recovery guidance. If a repo is not found, the LLM receives no hint to try search or alternative names. Errors should guide next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
List the most recent commits on a branch, with message, author, and date. Use this to see recent activity or find a commit SHA to use with other tools.
No tool annotations. All tools are read-only but lack readOnlyHint annotation. This prevents clients from optimizing caching, retry logic, or permission checks.
List tools (list_open_issues, list_pull_requests, list_recent_commits) cap results at 50 but provide no pagination mechanism (no cursor, offset, or next_token). Large result sets may be truncated silently.