Loading ToolBench
Self-hosted AI coding agent platform for GitLab, built with LangGraph. Reviews merge requests, resolves issues autonomously, and chats with full project context.
Strong tool definitions with clear naming, comprehensive descriptions, and well-structured schemas. All 15 tools have verb-noun names and detailed descriptions (avg ~150 chars). Input schemas are present with typed parameters and enums where appropriate. However, output schemas are not documented, error handling lacks recovery guidance, and some parameter descriptions could be more prescriptive about constraints and formats. Tool composition is excellent, each tool has a single responsibility and chains naturally (e.g., list_projects → search_project_history → get_project_issue). Security considerations are present (WRITE vs READ_ONLY risk labels) but not formalized as permission scopes.
Flag a durable, repo-specific fact worth remembering for future reviews of this repository — a coding convention, a recurring false positive, a team preference surfaced in discussion, or an architectural decision.
Record a durable personal fact, preference, or decision about yourself — not tied to any project. These persist across all conversations and workflows.
Fetch and parse a web page, returning its readable text content. Useful for reading documentation, blog posts, or other web resources referenced in conversation.
Fetch the full GitLab issue description and discussion thread for a specific issue, given its project and IID from a search_project_history result. Use this to verify a fact before relying on it — the full source may confirm it, add missing context, or show it doesn't actually answer the question (e.g. the origin/reasoning may genuinely not be recorded anywhere in the issue).
The authoritative list of files this MR changes (name + status), diffed against the merge-base with the target branch. Start here to scope the change.
Output schemas not documented. LLMs cannot plan downstream tool calls or extract required fields without knowing response structure. E.g., search_project_history returns 'facts' but structure is not specified; get_project_issue returns issue data but fields are not enumerated.
Error handling lacks recovery guidance. Tools do not specify what errors are retryable, user-fixable, or fatal. E.g., fetch_web_page may timeout or return 404, but no guidance on next steps. No actionable error messages documented.
Parameter constraints under-specified. 'limit' in search_project_history has min/max (1-20) but other numeric params (offset, timeoutSeconds) lack bounds. Format constraints (e.g., 'project' must match GitLab path format) not documented. LLMs may pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
The MR's full patch, diffed against the merge-base with the target branch (net change, not per-commit), capped at large sizes. A capped result names the files it had to leave out — fetch those individually with git_file_diff instead of retrying git_diff. Read surrounding code with the filesystem tools for context the patch omits — callers, callees, shared helpers.
One file's patch, diffed against the merge-base with the target branch. Use this for a file git_diff's truncated notice named, or to re-check a single file's change without re-fetching the whole MR.
List the GitLab projects whose issue/discussion history is searchable via search_project_history and get_project_issue. Call this first if the user's question names or clearly implies one specific project, so you can scope to it — omitting `project` on search_project_history searches all of them, which is slower and can mix results from unrelated projects.
Load step-by-step workflow instructions from a skill file. Call this first when you need guidance on how to perform a specific workflow. Available skills: gitlab-mcp/code-review
Look up your own stored personal memories — durable facts, preferences, or decisions about the user, not tied to any project — optionally filtered by category. Call this before create_personal_memory to check whether something similar is already recorded.
Inspect harness's stored per-project review memories. With a projectId, returns that project's memories (optionally filtered by category); without one, returns memory counts per project.
Read a text file from the repository, with optional line-range windowing. Paths are relative to the repo root. Returns line numbers alongside content for easy reference in follow-up queries.
Run a shell command inside the sandbox (cwd defaults to the repo checkout). Use it for reading, listing, and searching files (cat, ls, grep, find) as well as running builds, tests, and git commands.
Search extracted facts from ingested GitLab issue and discussion history (who reported/requested/fixed what, and when) — sourced from supermemory, not live GitLab. Each result is a short standalone fact, not the full discussion — it can be partial or lose context out of the surrounding thread. Before asserting anything consequential (a requirement's origin, who's responsible, current status), follow up with get_project_issue on the relevant issue's IID (and project) to verify against the full source rather than trusting one fact in isolation.
Write one or more files inside the sandbox, creating or overwriting them. Paths are relative to the repo checkout unless absolute.
Permission scopes not declared. Tools labeled WRITE vs READ_ONLY but no formal scope declarations (e.g., 'read:project', 'write:memory'). Prevents least-privilege agent configuration and audit clarity.
Destructive operations lack confirmation step. run_command and write_files can modify repo state but no dry-run or confirmation mechanism documented. Agents may accidentally delete or overwrite files.