Loading ToolBench
Provides comprehensive GitLab automation tools for streamlined project management
The server defines 15 tools with basic parameter schemas and descriptions, but exhibits systematic gaps in description quality, parameter documentation, and error handling guidance. Most tool descriptions are present but terse (many under 100 chars). Parameter descriptions are inconsistent, some fields like 'project_id' and 'user_id' are documented as 'optional' with fallback behavior, but the mechanism for setting defaults is not visible in the code (config file reference is incomplete). No tool declares its output schema explicitly. Error handling is minimal: the code catches exceptions and wraps them in generic McpError/GitLabMCPError, but error responses provide no recovery guidance. The destructive tool (delete_all_project_issues) has no confirmation or dry-run option. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. The naming is generally clear and verb-based, but some descriptions conflate behavior, e.g., 'create_gitlab_issue' description doesn't explicitly state it creates a new issue (redundant with the name). Per-tool scores range 35-60, averaging 42.
Add a comment/note to an issue in a GitLab project
Add a user to a GitLab project with a specific access level (admin mode only)
Assign a user to an issue in a GitLab project
Close an issue in a GitLab project
Create a new issue in a GitLab project
Create a new project in GitLab (admin mode only)
Create a milestone in a GitLab project (admin mode only)
No output schemas documented. Tools return API responses without explicit schema declarations. LLMs cannot predict available fields or structure downstream calls. Example: list_gitlab_users returns a JSON array from '/api/v4/users' but the schema (field names, types, required fields) is undocumented.
Destructive tool lacks confirmation/dry-run. delete_all_project_issues immediately deletes issues without a confirmation step or dry-run option. Agents can trigger catastrophic data loss in a single mistake.
Error handling provides no recovery guidance. The code catches GitLabMCPError and returns McpError, but error messages are raw exception strings (e.g. 'Curl command failed: <stderr>'). LLMs have no hint about what to do next: retry? call a different tool? ask the user?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
Delete all issues in a GitLab project (admin mode only)
Get closed issues assigned to a specific GitLab user (uses default IDs if not specified)
Get all users in the GitLab instance
Get open issues assigned to a specific GitLab user (uses default IDs if not specified)
Get all issues in a GitLab project (uses default project_id if not specified)
Get all members of a GitLab project (uses default project_id if not specified)
Get all accessible projects in GitLab
Reopen a closed issue in a GitLab project
Parameter descriptions conflate optional parameters with fallback behavior. Tools like list_project_issues state 'project_id (optional, uses default if not provided)' but the default source is not documented in the visible code. The config file reference is incomplete ('confi' truncation). LLMs cannot determine how to trigger fallback behavior.
No tool annotations. Tools lack readOnlyHint, destructiveHint, and idempotentHint tags. The server manually classifies risk as READ_ONLY/WRITE/DESTRUCTIVE in the metadata, but this information is not reflected in MCP tool definitions. LLMs cannot programmatically distinguish safe vs. risky operations.
Missing pagination controls. List tools (list_gitlab_users, list_project_issues) hardcode per_page=100 in the curl request but do not expose limit or offset/page_number parameters to the LLM. If a project has >100 issues, the LLM cannot retrieve the rest. Result limiting and pagination are not discoverable.
Incomplete descriptions (many under 100 chars). Tools like 'close_gitlab_issue' have 47-char descriptions: 'Close an issue in a GitLab project'. This omits critical context: does it change state? does it add a comment? when should you use this vs. another tool? LLMs cannot select the right tool without more specific guidance.
Some parameters lack type constraints. 'visibility' in create_gitlab_project is a free-form string with description '(private, internal, public)' but no enum constraint. LLMs can hallucinate invalid values like 'hidden' or 'restricted'. Access-level values (10, 20, 30, etc.) in add_project_member are numeric without enum constraints or range declarations.
Default behavior for optional parameters is not transparent. Tools describe 'project_id (optional, uses default if not provided)' and 'user_id (optional, uses default if not provided)' but the server-side default source (environment variable DEFAULT_GITLAB_PROJECT_ID) is only visible in the truncated config-loading code. Full default resolution logic is not documented in tool descriptions.
Credentials stored in plaintext config file. The code loads gitlab_url and access_token from 'gitlab-mcp-config.json' in the script directory. This is a security anti-pattern, credentials should use environment variables, vaults, or system key managers, not readable JSON files in the codebase.
No permission gates or audit trails. Destructive and administrative operations (delete_all_project_issues, create_gitlab_project, add_project_member, create_milestone) have no permission checks or audit logging. An agent with the token can perform any action without rate limits, permission validation, or logged accountability.