Loading ToolBench
A Model Context Protocol server for GitHub operations written in Go, enabling AI models to interact with GitHub repositories, issues, pull requests, and code search functionality.
The server provides 17 tools with basic schema definitions and descriptions. All tools have names following verb_noun convention and non-empty descriptions (baseline p10=34 chars). However, critical gaps exist: schemas lack formal type constraints (no enums for constrained fields like 'sort', 'order', 'state', 'direction'), parameter descriptions are minimal (often 15-30 chars vs baseline 72 chars), output schemas are completely undocumented, error handling is absent from tool definitions, and no tool annotations (readOnlyHint, destructiveHint, idempotentHint) are declared despite clear read-only vs write operations. The server sits squarely in the C (Fair) range, definitions are present but lack the LLM-optimization and machine-readable constraints needed for production reliability.
Add a comment to an existing issue
Create a new branch in a GitHub repository
Create a new issue in a GitHub repository
Create or update a single file in a GitHub repository
Create a new GitHub repository in your account
Fork a GitHub repository to your account or specified organization
Get the contents of a file or directory from a GitHub repository
No output schemas documented. LLMs cannot anticipate response structure, plan downstream calls, or validate returned fields. Every tool returns unstructured JSON without field descriptions.
No enum constraints on free-form string parameters. Fields like 'sort' (stars/forks/updated/best match), 'order' (asc/desc), 'state' (open/closed/all), and 'direction' are defined as bare strings. LLMs will hallucinate invalid values, causing API errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Get details of a specific issue in a GitHub repository
Get all tags for a GitHub repository
Get list of commits of a branch in a GitHub repository
List issues in a GitHub repository with filtering options
Push multiple files to a GitHub repository in a single commit
Search for code across GitHub repositories
Search for issues and pull requests across GitHub repositories
Search for GitHub repositories
Search for users on GitHub
Update an existing issue in a GitHub repository
Missing tool annotations. 9 tools are read-only (search_*, get_*) and 8 are destructive writes (create_*, update_*, fork_*, push_*). Absence of readOnlyHint and destructiveHint prevents agents from understanding operation consequences and safety constraints.
Parameter descriptions are generic and lack constraint details. E.g., 'page_number of results' (10 chars) vs 'Page number, must be >= 1'.
No error handling guidance. None of the 17 tool definitions document what errors can occur, how to recover, or whether failures are retryable. A tool that calls GitHub API can fail (rate limit, auth, network) but agents get no recovery hints.
No pagination limits documented. 'per_page' parameters accept arbitrary integers but tools do not state max results (e.g., GitHub API caps at 100). Unbounded requests can blow context windows or hit service limits.
No idempotency guarantees stated. Multi-step tools like 'push_files' and 'create_or_update_file' may have side effects; lack of idempotence declaration means agents retrying after network failure risk duplicate operations (double commits, duplicate issues).
No 'ref' parameter validation hints. 'get_file_contents' accepts 'ref' (branch/tag/commit SHA) but description lacks format guidance. LLMs may pass invalid SHA formats, causing failures.