Loading ToolBench
Server has 6 tools with complete input schemas and descriptions. Naming follows verb_noun convention (list_, get_, create_, merge_). Descriptions are present but generic, most lack context on WHEN to use the tool or what makes it distinct from similar tools. Parameters are typed and mostly described, but lack constraints (enums, ranges, patterns). Output schemas are documented in code but not exposed to the LLM. Error handling returns JSON with error fields but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Composition is sound: tools chain well (list_projects → get_project → list_mrs → get_mr → create_mr → merge_mr). Missing pagination metadata (total count, next_cursor) in list responses.
Create a new merge request. Args: project_id: Project ID or path source_branch: Source branch name target_branch: Target branch name title: MR title description: MR description (optional) draft: Create as draft MR (optional) assignee_ids: List of user IDs to assign (optional) Returns: JSON string containing created MR details
Get detailed information about a specific merge request. Args: project_id: Project ID or path mr_iid: Merge request IID (internal ID) Returns: JSON string containing MR details
Get detailed information about a specific project. Args: project_id: Project ID or path (e.g., '123' or 'group/project') Returns: JSON string containing project details
List merge requests for a project. Args: project_id: Project ID or path state: MR state filter (opened, closed, merged, all) limit: Maximum number of MRs to return Returns: JSON string containing list of merge requests
List GitLab projects. Args: search: Search query to filter projects owned: Only return projects owned by current user limit: Maximum number of projects to return (default: 20) Returns: JSON string containing list of projects
Descriptions lack WHEN-to-use context. E.g., list_projects says 'List GitLab projects' but doesn't explain when to call it vs get_project, or that it returns summaries suitable for discovery. LLMs need explicit selection guidance.
No enum constraints on state parameter in list_mrs. Description says '(opened, closed, merged, all)' as free text, not a formal enum. LLMs may hallucinate invalid values like 'pending' or 'archived'.
List tools (list_projects, list_mrs) lack pagination metadata. No total count, next_cursor, or has_more field returned. Large result sets will blow context window; no way for LLM to know if results are truncated.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Merge a merge request. Args: project_id: Project ID or path mr_iid: Merge request IID squash: Squash commits when merging Returns: JSON string containing merge result
Destructive tools (create_mr, merge_mr) lack tool annotations. No destructiveHint or idempotentHint in schema. LLM cannot distinguish safe retries from irreversible operations.
Error responses return generic JSON with 'error' field but no recovery guidance. E.g., 'Failed to list projects: <exception>' tells LLM nothing about whether to retry, ask user, or try a different tool.