Loading ToolBench
An MCP server that provides tools for querying and analyzing a code graph built from C/C++ source code using Neo4j, with semantic search and source code retrieval capabilities.
The server has 14 tools with mostly consistent naming (verb-first pattern) and reasonable descriptions, but exhibits significant gaps in schema completeness, parameter validation, and error guidance. 12 of 14 tools show input schemas, but several lack parameter descriptions or have minimal descriptions. Output schemas are undocumented. Error handling is present but generic. No tool annotations (readOnlyHint/destructiveHint). The semantic search tool chain is well-structured, but composition could be tighter.
Executes a custom Cypher query against the Neo4j graph database.
Generates vector embeddings for a query string to be used for semantic similarity search.
Retrieves the call graph for a given function, showing all functions it calls and functions that call it.
Retrieves the definition of a symbol, including its location in the source code.
Retrieves the Neo4j vector embedding indexes available for similarity search.
Retrieves the structure of a file, including all functions, classes, and other definitions.
Retrieves the Neo4j graph schema to understand node properties and relationships.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., field names, types, structure of returned objects). Source code shows return types are implicit (Dict, List, str) but response structure is nowhere documented in descriptions.
Several tools have minimal descriptions under 60 characters, providing little context for LLM selection. Examples: 'Retrieves the definition of a symbol, including its location in the source code.' (get_definition, ~70 chars but vague on 'which symbol'), 'Executes a custom Cypher query against the Neo4j graph database.' (execute_cypher_query, ~65 chars, no guidance on safety or when to use).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 68 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 45 | - | v1 |
Retrieves all relationships for a given node, including both incoming and outgoing edges.
Retrieves the project's name, root path and its high-level summary.
Retrieves all references to a given symbol in the codebase.
Retrieves the source code for any node (file, function, class, etc.) by its unique ID.
Retrieves the source code of a file by its path.
Searches for nodes in the graph by their name using exact matching or fuzzy matching.
Performs a semantic similarity search across all nodes in the graph to find the most relevant ones for a given query.
Parameter descriptions are often missing or too generic. Example: 'limit' in get_references has description 'Maximum number of references to return' with default=100, but no guidance on impact if omitted or if there's no pagination/offset support. 'fuzzy' in search_nodes_by_name lacks explanation of fuzzy algorithm or quality tradeoffs.
execute_cypher_query (tool #14) is a WRITE risk tool with minimal safeguards. Description does not warn that arbitrary Cypher queries can modify or delete graph data. No dry-run, confirmation, or permission-gate pattern implemented. LLM could accidentally execute destructive queries (DELETE, SET, CREATE) without warning.
Error handling is generic and non-actionable. Example from code: 'return {"error": f"An error occurred during semantic search: {e}"}', no recovery guidance, no classification (retryable vs fatal), no alternatives. Patterns like 'User not found. Try search_users() with a partial name.' are absent.
No tool annotations present. Tools lack readOnlyHint, destructiveHint, or idempotentHint metadata. This is required by current MCP spec (2026-07-28) for agents to understand which calls are safe to retry and which modify state. execute_cypher_query especially needs destructiveHint=true.
Tools return verbose or unstructured responses. Example: search_nodes_for_semantic_similarity returns raw Neo4j result dicts; get_graph_schema dumps entire schema as a string. No pagination for large result sets. No documented limits on result counts, an agent querying a large codebase could receive thousands of nodes, blowing context window.
Composition gaps: tools accept system IDs (node_id, symbol_id, file_id, function_id) but provide no lookup tools for common user inputs (function name, class name, file path). An agent knowing 'I want the call graph for myFunction' must first call search_nodes_by_name, extract the ID, then call get_call_graph, two steps to do one thing.
execute_cypher_query accepts 'parameters' as an untyped object with no schema. This is a prompt-injection risk, LLMs could construct malicious Cypher payloads. No input validation, escaping, or constraints documented. Requires server-side query sanitization.