Loading ToolBench
No-train, on-prem code knowledge graph served to AI agents over MCP, with a hash-chained audit log.
Codegraph-mcp has 11 well-named, read-only tools with clear verb-noun naming (search_symbols, find_references, impact_analysis). All tools have descriptions (avg ~100 chars, within baseline 34-392 range). Input schemas are present with types and enums where appropriate (e.g., kind enum in search_symbols). However, parameter descriptions are sparse or missing entirely, most params lack explanatory text beyond the name. Output schemas are not documented; responses are inferred from code (e.g., search_symbols returns {results: [...]}, get_symbol returns {symbol: ...}). Error handling is minimal, no recovery guidance, no actionable error messages. No tool annotations (readOnlyHint, etc.). The server is read-only and stateless, which reduces risk, but lacks LLM-optimized descriptions and comprehensive parameter documentation expected of production tools.
List resolved cross-language HTTP edges (e.g. a TS fetch -> a Go/Python handler).
List symbols the given symbol directly calls.
List symbols that directly call the given symbol (includes cross-language).
Most depended-on symbols (highest caller count) — where changes ripple furthest.
Dead-code candidates: functions/methods with no callers and not HTTP entrypoints.
List every call site / identifier use of a name across the repo.
Fetch one symbol's full record (signature, location, container) by id.
Parameter descriptions missing or minimal. Most params (symbol_id, name, limit, max_depth) lack explanatory text. LLMs cannot infer meaning from names alone, 'symbol_id' could be a database ID, AST node ID, or hash. Descriptions should explain what the ID refers to and how to obtain it.
Output schemas not documented. Tools return JSON objects (e.g., {results: [...], symbol: {...}, callers: [...]}) but the response structure is inferred from code, not declared in tool specs. LLMs need explicit field documentation to plan downstream calls and extract data correctly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Summary counts for the indexed graph (files, symbols, edges, languages).
Transitive callers ('blast radius') of a symbol — what may break if it changes.
Module/package-level dependency graph: modules and the (weighted) edges between them.
Search indexed symbols (functions, methods, classes, types) by name substring.
No error handling or recovery guidance. Tools have no documented error cases, no actionable error messages, and no guidance for LLMs on what to do if a call fails. E.g., if get_symbol(999) returns null, should the LLM retry, search for the symbol, or report to the user?
No tool annotations. Tools lack readOnlyHint, destructiveHint, or idempotentHint. All 11 tools are read-only and safe to retry, but this is not declared in the schema. Agents cannot infer safety properties without explicit annotations.
Tools with no input parameters (cross_language_edges, project_graph, graph_stats) have empty input schemas. While valid, descriptions should explain when to call these discovery tools and what they reveal. E.g., 'Call this first to understand module dependencies before analyzing impact.'