Loading ToolBench
LSP-Based Codebase Summarization Tool - provides MCP tools for code analysis, symbol searching, and hierarchical codebase summarization with LLM-powered analysis
Five tools with complete input schemas and descriptions, but significant quality gaps. All tools have descriptions (10-200 chars, baseline 194), and all parameters are typed. However, descriptions are generic and lack LLM-optimized guidance on WHEN to use each tool or HOW to chain them. No output schemas documented. Error handling is minimal, tools return plain strings or empty lists on failure, providing no recovery guidance. Parameter descriptions are sparse (e.g., 'Path to the project' lacks format/constraint details). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear read/write semantics. Naming is verb-first and clear (summarize_, search_, get_, invalidate_), but descriptions do not explain dependencies (e.g., search_symbols requires summarize_codebase to run first).
Retrieves all extracted key identifiers from a processed codebase hierarchy
Retrieves the summary for a specific code symbol by name and file path
Invalidates cached symbol hierarchy data for a project, optionally matching a specific pattern
Searches for code symbols in a project by name or summary content, returning matching CodeSymbol objects with optional filtering by symbol kind and file patterns
Summarizes a codebase by processing it with the Compressor service and returning hierarchical symbol information including root symbols count, extracted keys, and last update timestamp
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, forcing them to parse unstructured strings or guess at result structure.
Descriptions lack dependency hints and selection guidance. E.g., search_symbols requires summarize_codebase to run first, but this is not stated. LLMs cannot plan multi-step workflows.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). invalidate_cache is clearly destructive, but LLMs have no signal to treat it with caution or avoid retrying.
Error handling returns plain strings or empty lists with no recovery guidance. E.g., 'No hierarchy found. Run summarize_codebase first.' is a string, not a structured error with actionable next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Parameter descriptions are minimal. E.g., 'Path to the project directory' lacks format constraints, validation rules, or examples. LLMs cannot validate inputs before calling.