Loading ToolBench
MCP server for code analysis - provides code quality, security, documentation, and API analysis tools
The code-analyzer-mcp server provides 4 tools with explicit schemas and descriptions, but exhibits significant quality gaps that prevent a higher score. Tool names start with action verbs (analyze, get, scan) which is correct. However, descriptions are present but generic and lack LLM-optimized guidance on WHEN to use each tool versus alternatives. Parameter descriptions exist but are sparse, e.g., the 'exclude' parameter in analyze_repository lacks detail on expected pattern format (glob? regex? paths?). Output schemas are NOT documented anywhere in the source, forcing LLMs to guess what fields they'll receive and breaking the tool chain (pattern:tool-chain violation). Error handling is minimal, no recovery guidance, no classification of retryable vs fatal errors, and no validation of path inputs. The server calls external 'agents' (RepoAnalyzer, ExclusionManager, LLMInterface) but does not expose or constrain their behavior, creating unpredictability. Overall, this is a C-grade server, functional but with gaps that would cause misuse and context waste in production.
Analyze specific files with a chosen agent
Analyze a repository with specified agents for code quality, security, documentation, and API design
Get basic information about a repository structure
Scan a repository to understand its structure, complexity, and suggest optimal analysis approach
Output schemas not documented. LLMs cannot infer what fields analyze_repository, analyze_files, get_repository_info, or scan_repository will return. This breaks tool composition (pattern:tool-chain), if the next step requires repository_id or analysis_summary, the agent has no way to extract it or know to ask.
Parameter descriptions are too generic. 'exclude' in analyze_repository says 'Patterns to exclude from analysis' but does not specify format (glob, regex, file paths?), precedence, or examples. 'path' is described only as 'Path to the repository', no guidance on absolute vs relative, or symlink handling.
No error handling guidance. If a path does not exist or is not readable, the LLM receives no actionable error. No recovery hints (e.g., 'Call get_repository_info first to verify the path'), no distinction between retryable (permissions) vs fatal (path not found) errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 50 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 20 | - | v1 |
Tool descriptions lack WHEN-to-use guidance. analyze_repository and scan_repository appear to overlap, both scan a repo, but one analyzes with agents and one 'suggests optimal analysis approach'. The description does not clarify which to call first or when to use one over the other.
No pagination or result limits documented. If analyze_repository returns 500+ issues, token usage explodes and LLM reasoning degrades. No mention of limits, pagination, or how to cap results.
Parameter 'agents' uses enum values (api-quality, documentation, code-quality, security, next-task) but the description says 'defaults to all', however, no explicit mechanism for 'all' is documented. LLM must guess whether to omit the param or pass an undocumented value.
No tool annotation hints (readOnlyHint, idempotentHint). All four tools are read-only and should declare this via idempotentHint: true and readOnlyHint: true so agents know they are safe to retry or call in any order.