Loading ToolBench
MCP server for deep codebase understanding - gives LLMs complete project context
16 tools with complete schemas and descriptions, but significant quality gaps. All tools have input schemas with types and descriptions (baseline met). However, descriptions are often generic and lack LLM-optimized guidance on WHEN to use each tool vs. similar ones. Output schemas are documented in descriptions but not formally specified. Error handling is minimal, most tools return generic error dicts without recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Parameter descriptions are adequate but lack constraints (enums, ranges, format specs). No batch variants despite many tools operating on single items. Tool composition is weak: semantic_search, smart_context, and get_repo_map overlap significantly without clear differentiation.
Analyze what would be affected by changing a file. Returns: - Direct dependents: Files that import this file - Indirect dependents: Files that import direct dependents - Symbols exported: What this file provides - Risk level: Based on number of dependents
Analyze codebase for patterns and issues. Detects: - Security issues (hardcoded secrets, SQL injection) - Performance problems (N+1 queries, blocking I/O) - Code quality issues (God objects, long functions) - Design patterns (Singleton, Factory, Observer) - Frameworks used
Find ALL places where a symbol is used across the codebase. Unlike search_symbols (finds definitions), this finds every reference: - Function calls - Import statements - Variable assignments - Attribute access Returns: List of usages with file, line, content, and usage type.
Generate architecture diagram showing project layers. Auto-detects: - API routes / endpoints - Business logic / services - Data layer / models - UI components - Utilities
Build call graph for a function. Shows: - What functions call this function (callers) - What functions this function calls (callees) - Mermaid diagram for visualization Returns: Call graph with callers, callees, and Mermaid diagram.
Overlapping tool semantics without clear differentiation. semantic_search, smart_context, and search_symbols all find code but with unclear boundaries. LLMs will struggle to choose the right tool.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only but this is not declared. Agents cannot infer safety properties.
Error handling returns generic dicts without recovery guidance. E.g. {'error': str(e)} tells LLM nothing about what to do next. No error classification (retryable vs. user-fixable).
Parameter descriptions lack constraints. E.g. 'max_tokens' has no min/max bounds; 'checks' array in analyze_patterns has no enum constraint; 'direction' in get_call_graph should be an enum.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Get multiple files in token-efficient format. Modes: - full: Complete file contents - signatures: Only function/class signatures - smart: Signatures for large files, full for small
Get dependency information for a file. Returns: - imports: Files this file imports - imported_by: Files that import this file - symbols: Symbols defined in this file
Get a file with smart related context. Returns the file content along with: - Symbols defined in the file - Files it imports (with their symbols) - Files that import this file Use this after get_repo_map to dive into specific files.
Get project statistics. Returns: Total files, lines, languages breakdown.
Get recently modified files using git. Returns list of recently changed files with commit info.
Get a condensed repository map showing all symbols. This is the most important tool - it gives you complete codebase structure in a single context window. Start with this to understand any codebase. Returns: Markdown-formatted map with all classes, functions, and their signatures.
Read file content with optional line range. Use this to read specific sections after using get_repo_map to identify what you need.
Find all occurrences of a symbol (function, class, method). Returns list of matches with file path, line numbers, and signature.
Search code by meaning, not just keywords. Finds code related to your query even if exact words don't match. Searches both symbol names and file contents. Returns: List of matches with file, line, and relevance.
Auto-find all relevant files for a question. Analyzes your question, extracts keywords, and finds the most relevant code sections. Saves tokens by filtering out irrelevant files. Example: "how does authentication work?" → finds auth-related files Returns: Relevant files with content, sorted by relevance score.
Trace execution flow from an entry point. Follows function calls to show step-by-step execution path.
Output schemas not formally documented. Descriptions mention return structure (e.g. 'Returns: Markdown-formatted map') but no JSON Schema for response shape. LLMs cannot reliably parse outputs.