Loading ToolBench
A server that provides access to Gerrit code review functionality
This Gerrit MCP server has 8 tools with consistent verb-based naming (get_*, create_*, set_) and basic parameter schemas. However, significant quality gaps reduce the score: output schemas are not documented, parameter descriptions are minimal (10-20 chars, below the baseline of 72), error handling returns raw dicts without actionable recovery guidance, and write-path tools (create_draft_comment, set_review) lack confirmation patterns or dry-run support. The tool definitions are properly registered with types, but the descriptions lack LLM-optimization depth. No evidence of pagination handling for list-like operations. The average tool quality is fair but falls short of production standards.
Create a new draft comment on a specific line in a file for the current revision. If line is -1, creates a global comment on the file without line number.
Get detailed information about a change.
Fetch commit information for the current revision of a change.
Get the commit message for the current revision of a change.
Get the diff for a specific file in the current revision of a change.
Get a detailed list of files for the current revision of a change.
Get related changes for the current revision of a change.
Output schemas not documented. None of the 8 tools include documented return types or response field specifications. LLMs cannot plan downstream tool calls or extract specific fields without knowing the response structure.
Parameter descriptions are below baseline (10-20 characters vs. 72 char baseline). Examples: 'The ID of the change to get commit info for' (45 chars) lacks context on format, mutually-exclusive params, or when to call. LLMs cannot disambiguate between similar tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 8 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Submit all draft comments for a change and apply the specified Code-Review label.
Write-path tools (create_draft_comment, set_review) lack confirmation or dry-run support. Agents can apply -2 labels (block merge) or create comments without validation. No pattern prevents accidental or malicious review rejection.
Error responses are raw dicts (e.g., {"error": "Gerrit session not available"}) without actionable recovery guidance. Code example in commit_tools.py shows no retry hints, escalation paths, or user-fixable suggestions. LLM has no context on next steps.
No input validation or error classification. Tools accept change_id and file_path as free strings with no format constraints, enums, or length limits. A typo in change_id fails silently with a 404-like response rather than guiding the LLM toward valid alternatives.
No pagination parameters or limits declared. Tools like get_file_list and get_related_changes have no offset/limit or next_cursor. Large result sets will explode token usage and degrade LLM reasoning.
Gerrit session obtained from context.request_context.lifespan_context.get('gerrit_session') without null safety or binding documentation. If session is None, tools return error dicts instead of failing gracefully with structured errors. No evidence of session initialization or lifecycle management.
Tool description for create_draft_comment mentions 'If line is -1, creates a global comment' but the input schema does not document -1 as a special case or constrain line to a valid range. LLMs may pass invalid line numbers or misunderstand the semantics.
Code-Review label in set_review accepts integers (-1, -2) but schema description does not enumerate valid values. Does it accept 0, 1, 2? Schema must document the full range or use an enum.