Loading ToolBench
A Next.js application for repository analysis and code review with GitHub integration, featuring semantic code search, PR analysis, and AI-powered review tools
repochat defines 7 tools with adequate naming and descriptions, but exhibits significant gaps in schema completeness, parameter descriptions, and error handling. All tools follow verb_noun naming convention (analyzePR, getFileContent, postReviewComment, submitReview, mergePR, getRepoTree, searchCode). Descriptions are present for all tools (130-220 chars, within baseline range of 34-392). However, many input parameters lack descriptions, output schemas are incomplete or missing, and error handling guidance is absent. The server is STDIO-only (hard cap 50 for protocol readiness). This is a typical community MCP server with moderate definition quality but significant production readiness issues.
Analyze a GitHub pull request to review code changes. Use this when the user wants to review a PR or asks about PR changes. Returns PR metadata, changed files with diffs, and relevant codebase context from the indexed repository when available.
Get the content of a file from a GitHub repository. Use this when the user wants to see a specific file or needs context about code.
Get the file tree structure of a repository. Use this when the user wants to explore a repository or see its structure.
Merge a pull request. Use this when the user explicitly asks to merge a PR. ALWAYS confirm with the user before merging.
Post a review comment on a specific line of a pull request. Use this when the user wants to add a comment about specific code.
Input parameter descriptions missing for most tools. Parameters like 'owner', 'repo', 'prNumber', 'path', 'ref', 'line', 'body', 'event', 'mergeMethod', 'branch', 'query' are present in schemas but lack descriptions beyond minimal text. LLMs cannot infer parameter semantics from names alone and will struggle with optional vs required distinctions and valid value ranges.
Output schemas incomplete or missing. analyzePR declares outputSchema with nested objects and optional codebaseContext array, but getFileContent, getRepoTree, and searchCode have no visible output schema documentation. Without output schemas, LLMs cannot predict what fields to extract or plan downstream tool chains.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Search for code in a repository using semantic search on indexed codebase when available, with GitHub API fallback. Use this when the user wants to find specific code patterns, functions, or understand how something works. When the repo is indexed, results include code snippets, descriptions, and line numbers.
Submit a review on a pull request (approve, request changes, or comment). Use this when the user wants to approve a PR or request changes.
No error handling or recovery guidance. Descriptions for mergePR state 'ALWAYS confirm with the user before merging' but there is no visible confirmation/dry-run mechanism or error response documentation. Tools that modify state (postReviewComment, submitReview, mergePR) lack guidance on retryability, partial failures, or recovery steps.
Optional parameters not clearly marked in descriptions. 'ref' in getFileContent and 'branch' in getRepoTree/searchCode are optional, but the descriptions do not explain defaults or when they must/should be supplied. Parameter descriptions should include constraint info (e.g., 'defaults to main branch if omitted').
Enum parameter 'event' in submitReview lacks explanation of each option. The enum ['APPROVE', 'REQUEST_CHANGES', 'COMMENT'] is valid but descriptions do not clarify when to use each, or whether body is required for some options. LLMs may hallucinate invalid states.
mergePR 'mergeMethod' parameter enum has no description of options. Valid values are ['merge', 'squash', 'rebase'], but the tool description does not explain when to use each method or what the default is. This forces LLMs to guess or default naively.
Lack of parameter validation constraints. Numeric parameters (prNumber, line) and string parameters (query, path, body) have no min/max, length limits, or regex patterns documented. Unbounded numeric inputs (line: any number) could cause API errors; long free-text strings could exhaust token budgets.