Loading ToolBench
MCP Server for CodeGraph - enables AI assistants to query code knowledge graph
CodeGraph MCP Server has 9 tools with basic naming and descriptions, but significant gaps in schema completeness, parameter documentation, and error handling. Most tools have 1-2 parameters with minimal descriptions. No input schemas are visible in the source code provided, only parameter names and basic types are inferred from the tool list. Output schemas are completely undocumented. Tools follow verb_noun naming convention (search_, analyze_, trace_, query_, get_, list_, extract_, find_), which is positive, but parameter descriptions lack detail on valid ranges, formats, constraints, and dependencies. Error handling is not described. No evidence of pagination, idempotency, or structured error recovery. STDIO transport caps protocol readiness at 50, which also affects overall scoring.
Analyze code complexity metrics for files or functions
Analyze a specific file and extract its AST structure, functions, classes, and relationships
Extract documentation, comments, and docstrings from code entities
Find all references to a specific code entity across the codebase
Retrieve the full content of a file from the codebase
List files in the codebase with optional filtering by type or pattern
Execute queries against the code knowledge graph using graph query language
Search the codebase by semantic meaning or code patterns
No input schemas visible in source code. Tools list shows parameter names and basic types (string, number, boolean, enum) but no JSON Schema definitions, constraints, min/max bounds, or regex patterns. Cannot verify parameter validation rules.
Output schemas completely undocumented. No documentation of what fields are returned, their types, or structure. LLMs cannot plan downstream tool calls or extract required data when return structure is unknown.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Trace dependencies and relationships between files, functions, and classes
Parameter descriptions lack actionable detail. 'Search query for semantic or pattern-based code search' does not specify: what query language is accepted, min/max length, example patterns, or what happens if no matches found. Descriptions should state constraints, valid values, and error recovery paths.
No pagination support documented. Tools like list_files and search_codebase may return large result sets, but there is no mention of limit/offset, page_size, next_cursor, or total_count. Without pagination, large result sets blow context windows.
No error handling guidance. Descriptions do not explain what happens on failure (e.g. file not found, invalid query syntax, graph unavailable). LLMs need recovery hints: 'If file not found, try list_files() first with a pattern.'
Parameter 'limit' in search_codebase lacks constraints. No min/max specified. LLMs may pass absurd values (1 or 1000000) without bounds checking. Should specify 'limit (integer, 1 - 100, default 20)'.
Parameter 'type' in analyze_complexity has enum values documented ('function', 'file') but description is minimal ('Type of entity to analyze'). Should clarify behavior differences: 'For functions, analyzes cyclomatic complexity, line count, and nesting depth. For files, returns aggregate metrics.'
query_graph tool is high-risk. Accepts arbitrary 'Cypher or similar query language' with no syntax validation guidance or limit on execution time. No documentation of performance implications, timeouts, or safe vs unsafe queries. Requires query language specification and sandbox details.
Parameter 'filePath' appears in multiple tools (analyze_file, get_file_content, extract_documentation) but descriptions do not clarify: relative vs absolute paths, whether symlinks are followed, or if path traversal is prevented. Must document: 'Relative to codebase root. Path traversal (../) is prevented.'