Loading ToolBench
The server implements 9 tools with TypeScript/Zod schemas and partial descriptions. Tool naming follows verb_noun conventions correctly (create_*, get_*, search_, push_, fork_). Schemas are present and properly typed using Zod validators. However, tool descriptions are extremely brief (10-30 chars), parameter descriptions are minimal, and output schemas are not documented. Error handling is generic (throws raw API errors without recovery guidance). No idempotency hints, permission gates, or destructive operation warnings. The schema field type declarations are good, but descriptions fall well below the 50-200 char LLM-optimized baseline.
Create a new branch in a GitLab project
Create a new issue in a GitLab project
Create a new merge request in a GitLab project
Create or update a single file in a GitLab project
Create a new GitLab project
Fork a GitLab project
Get the contents of a file in a GitLab project
Tool descriptions are critically short (10-30 characters). LLM-optimized descriptions should be 50-200 chars, providing context on WHAT the tool does, WHEN to use it, and any prerequisites. Examples: 'Create or update a single file in a GitLab project' (52 chars) is borderline acceptable, but 'Push multiple files to a branch in a GitLab project' (51 chars) lacks detail on when to use push_files vs create_or_update_file.
Output schemas are not documented. The code uses Zod schema validators internally (e.g., GitLabIssueSchema.parse()) but returns unstructured responses to the LLM. The MCP spec requires documented return types so agents know what fields to expect. Example: create_issue should document that it returns {id, iid, title, description, state, author, assignees, labels, created_at}.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Push multiple files to a branch in a GitLab project
Search for GitLab projects
Error handling is generic. All tools throw 'GitLab API error: {statusText}' with no guidance on recovery. Per pattern:recovery-guide, errors should classify as retryable/user-fixable/fatal and suggest next steps. Example: 404 should return 'Project not found. Try search_repositories() to find the correct project ID.' 401 should indicate 'Authentication failed. Verify GITLAB_PERSONAL_ACCESS_TOKEN is set.'
Destructive and write operations (create_or_update_file, push_files, create_issue, create_merge_request, fork_repository, create_branch) lack idempotency hints and confirmation patterns. Per pattern:confirmation-request, irreversible operations should support dry-run or confirmation to prevent accidental side effects. No destructiveHint or idempotentHint annotations present.
Parameter descriptions are minimal or absent. For example, in create_or_update_file, the 'branch' parameter has description 'Branch to create/update the file in', adequate but sparse. Many parameters (e.g., assignee_ids, milestone_id in create_issue) lack dependency hints or constraints. Per pattern:constrained-input, parameters with allowed values should use enums; visibility in create_repository should document valid enum values.
search_repositories returns pagination parameters (page, per_page) but no guidance on total count or next_cursor. Per pattern:paginated-result, large result sets should be capped at 20-50 items with a total count so the LLM knows if more results exist and how to fetch them.
No scope declarations or permission gates. Tools access GitLab API via a single GITLAB_PERSONAL_ACCESS_TOKEN with no per-tool permission checks. Per pattern:scope-declaration, each tool should declare required scopes (e.g., 'api:write', 'read:repository') for audit and least-privilege configuration.
Tool composition has potential breaks: create_branch returns GitLabReference but downstream tools (create_or_update_file, push_files) expect a branch name string. No explicit assurance that returned IDs chain correctly into next calls. Per pattern:tool-chain, tool A output should contain IDs that tool B accepts.