Loading ToolBench
GitHub MCP Server providing tools for repository and file management
This MCP server defines 18 GitHub-focused tools with mostly complete schemas and descriptions, but suffers from critical issues in error handling, parameter validation guidance, and output documentation. While naming follows verb_noun conventions consistently (list_accounts, create_repository, delete_file), descriptions are generic and lack LLM-optimized guidance on when to use each tool. Many parameters lack constraints (enums, ranges, formats), forcing LLMs to guess valid inputs. Output schemas are not documented anywhere in the source, responses are inferred from implementation, not contractually defined. The server handles sensitive operations (file deletion, repository deletion via clone/rename patterns) without confirmation workflows or dry-run capabilities. Error responses appear to be generic HTTP status codes without recovery guidance.
Clone a GitHub repository to a local directory
Compare a file in GitHub with a local file
Create a commit with multiple file changes
Create or update a file in a GitHub repository from a local source file
Create a new GitHub repository
Delete a file from a GitHub repository
Get details about a specific commit
No documented output schemas for any tool. Responses are inferred from implementation; LLMs cannot predict what fields are returned. This forces agents to inspect responses at runtime and risks downstream tool chaining failures if assumed fields are missing.
Destructive operations (delete_file, revert_commit, clone_repository+rename, sync_directory) lack confirmation workflows or dry-run capability. Agents can irreversibly delete repository content without a safety prompt.
Descriptions are generic and lack LLM-optimized context (10-50 chars on average). They do not explain WHEN to use each tool vs similar tools. E.g., 'Pull a file from a GitHub repository' vs 'Get the content of a file', unclear distinction without additional context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Get the content of a file from a GitHub repository
List available GitHub accounts
List commits in a GitHub repository
List files in a GitHub repository
Pull a directory from a GitHub repository to a local path
Pull a file from a GitHub repository to a local path
Push a local file to a GitHub repository
Rename a GitHub repository
Revert a commit in a GitHub repository
Select a GitHub account to use for subsequent operations
Sync a directory between local and GitHub (pull remote changes then push local changes)
Parameters lack constraints and validation guidance. E.g., 'branch' has no default documented in description (spec shows 'default: main' but LLMs read descriptions, not schema). No enums for 'operation' in create_commit changes array (add/modify/delete). No ranges for array sizes or numeric limits.
No error recovery guidance in tool descriptions. Tools do not document what errors are retryable, what indicates user misconfiguration, or how to recover from common failures (e.g., 'branch not found', 'permission denied', 'file exists').
Parameter 'owner' is marked 'optional if account already selected' but there is no documentation of how account selection persists across requests or how to know which account is currently active. This hidden state violates stateless request handling.
create_commit accepts a 'changes' array with nested objects (path, sourcePath, operation) but operation enum is not declared in the schema. LLMs must guess valid values (add/modify/delete) from context.
Tools that return lists (list_accounts, list_commits, list_repository) have no documented pagination or result limit behavior. Large repositories could return thousands of files/commits, exhausting context windows. No mention of offset, limit, or total_count in output.
Parameter descriptions use placeholder language ('optional if account already selected', 'default: main') but schema defaults are not consistently applied. LLMs reading descriptions will assume 'branch' is required when it may have a default.