Loading ToolBench
Code intelligence MCP server — indexes codebases into a queryable graph of symbols, call chains, and relationships
Synapps MCP exhibits significant gaps in definition quality. Tool descriptions vary widely in quality and completeness. Most parameters lack formal type definitions and descriptions in verifiable source code, many tool definitions appear inferred from Svelte UI code rather than explicitly registered in the MCP server implementation. Input schemas are either missing or not directly visible in the Python server code (src/synapps/mcp/tools.py). Output schemas are entirely undocumented. Error handling and recovery guidance are absent. While the tool names follow verb_noun conventions reasonably well, parameter naming lacks consistency (e.g., 'full_name' vs 'query' for similar lookups). The server shows promise in conceptual design but lacks the rigor needed for production LLM agent integration.
Assess the impact of changes to a symbol by analyzing its usage patterns and dependents.
Execute a read-only Cypher query against the code graph database.
Explore the graph around a symbol by traversing relationships to a given depth.
Find all methods called by a given method.
Find methods and classes that are not called by any other code.
Find HTTP endpoints and their handler methods.
Find all classes that implement the given interface. Accepts both full names (e.g. "MyNs.IFoo") and short names (e.g. "IFoo"). Short names use a suffix match when an exact match is not found. When a short type name matches both an interface and concrete class, the interface is preferred. Method-level ambiguity (e.g. CreateAsync on multiple classes) still requires a qualified name.
Tool definitions inferred from UI code, not verified in MCP server implementation. Tool registration in src/synapps/mcp/tools.py not directly shown with explicit schema definitions. This violates the principle that tool definitions must be verifiable in source code.
Input parameter descriptions are minimal or absent. Most tools lack descriptions for individual parameters, forcing LLMs to infer meaning from names alone. E.g., search_symbols 'query' param has no description of acceptable format, length, or matching behavior. read_symbol 'max_lines' defaults to 100 but no justification provided.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Find methods that do not have test coverage.
Find all places where a symbol is used.
Get the high-level architecture of the project including packages, namespaces, and their relationships.
Get contextual information about a symbol including its container, members, and relationships.
Index a project's codebase into the graph. Uses MERGE (upsert) so nodes are updated in place rather than deleted and recreated. Summaries and other non-structural properties are preserved. Uses LSP for structural analysis (symbols, inheritance, implementations) and tree-sitter for call site detection. Supports C# (language='csharp'), Python (language='python'), and TypeScript/JavaScript (language='typescript') projects. Language is auto-detected from file extensions when not specified.
List all indexed projects. Returns path, languages (list), and last-indexed timestamp for each. path: if provided, returns detailed index status for that specific project (file count, symbol count, per-label breakdown) instead of the project list.
Read the source code and metadata of a symbol.
Search for symbols in the graph by name, kind, language, and namespace.
Sync the graph with the current filesystem state. Detects files that changed, were added, or were deleted since last indexing and re-indexes only what changed. Requires the project to have been fully indexed at least once (run index_project first).
No documented output schemas for any tool. LLMs cannot determine what fields are returned or their types. For example, search_symbols must return symbol objects with fields like 'full_name', 'kind', 'file_path', etc., but this structure is never documented. This violates the requirement that tools document their return types.
No error handling or recovery guidance. Tools provide no indication of failure modes, retryability, or actionable next steps. E.g., if index_project fails (permission denied, language not supported, invalid path), the agent has no guidance on what to do. This violates error-classification and recovery-guide patterns.
Incomplete parameter schemas in visible code. 'language' parameter in index_project and find_http_endpoints lacks enum constraints; agents could pass unsupported values like 'rust' or 'go'. Parameter types not explicitly declared in most tools.
Parameter naming inconsistencies invite confusion. 'full_name' is used in find_implementations, search_symbols, read_symbol, find_usages, find_callees, explore, get_context_for, assess_impact, but search_symbols also accepts 'query'. An agent must reason about when to use 'query' vs 'full_name', increasing error risk. Consider standardizing on one approach with clear guidance on which tools accept partial names vs full qualified names.
Ambiguous tool purposes and overlapping functionality. 'find_implementations' vs 'find_usages' distinction unclear, when should an LLM choose one over the other? 'explore' is a generic name that could apply to many graph operations. 'get_context_for' vs 'read_symbol', what's the difference? Overlapping tool names force LLMs to waste reasoning cycles on disambiguation.
Missing pagination and result-limiting guidance in descriptions. find_dead_code and find_untested have 'limit' and 'offset' parameters marked 'hidden', but no description explains pagination strategy or why results are hidden from the LLM. Other tools (search_symbols, find_implementations) specify defaults (50, 20) but no rationale. This violates the pattern of making result limits explicit and justified.
Tool composition gaps: execute_query accepts arbitrary Cypher. No validation, no documented safe query patterns, no examples of safe vs unsafe queries. This invites misuse and potential injection attacks. An LLM given unconstrained SQL-like power will hallucinate dangerous queries.
Missing idempotency guarantees. index_project and sync_project both modify state. No documentation of whether calling index_project twice on the same path with the same settings is safe and produces identical results. This matters for agent retry logic, if tools are not idempotent, retries risk duplicate work or side effects.