Loading ToolBench
Model Context Protocol (MCP) server for Language Server Protocol (LSP) integration
LSP-MCP defines 5 tools with good verb-based naming (get_*, find_*, search_*) and descriptions present in docstrings. However, analysis reveals significant gaps in parameter schema completeness, output documentation, and error handling guidance. Tools are READ_ONLY and well-intentioned, but lack the structured precision required for production LLM agents. Parameter descriptions are adequate but schema definitions lack explicit type constraints and validation ranges. No documented output schemas, making it unclear what fields agents should expect. Error handling is minimal, most failures return generic strings like 'Error: LSP client not initialized' or 'No definition found' without recovery guidance.
Find all references or implementations of a symbol.
Find the definition, declaration, or type definition of a symbol.
Get hover information for a symbol at a specific location.
Get the structural outline of a file showing all symbols.
Search for symbols across the entire workspace.
Missing output schema documentation. Tools return strings formatted via response.format() but the agent has no specification of what fields to extract, structure, or what to do with the result. Without documented output schemas, agents cannot reliably parse results or chain tools.
Generic error responses without recovery guidance. Errors like 'Error: LSP client not initialized', 'No definition found', and 'No outline available' provide no actionable next steps. Agents cannot distinguish retryable failures from permanent ones or know what alternative tool to call.
Incomplete parameter constraints. Parameters like 'mode' (get_definition: 'definition'|'declaration'|'type_definition'; find_references: 'references'|'implementations') are described as strings but lack explicit enum validation in the schema. Numeric parameters (max_items, context_lines, line, character) lack min/max bounds documented in descriptions, inviting invalid input from LLMs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 53 | - | v1 |
Ambiguous parameter relationships not documented. get_definition, find_references, and get_hover_info accept EITHER (symbol_name) OR (line + character). The mutual exclusivity and resolution strategy are not explained in parameter descriptions. Agents may pass both or neither, causing silent misuse.
No result limits or pagination documented. search_workspace defaults to max_items=50 but this cap is not stated in the description. If results exceed the limit, there is no mechanism (next_cursor, offset, total_count) for pagination. Agents have no way to retrieve additional results or know if they are seeing a complete result set.
Lack of tool-chaining support. Tools return markdown-formatted strings with no structured IDs or references. An agent cannot extract a definition location and pass it to find_references or get_hover_info, it must re-specify file_path and symbol details. This breaks the tool composition pattern.
No permission gates or scope declarations. Tools offer read-only access to code, but there is no permission check, audit trail, or scope declaration (e.g., 'read:workspace', 'read:definitions'). This makes it impossible to enforce least-privilege agent configurations or trace agent-initiated calls for compliance.
Unclear error classification. Tools return simple strings on failure but do not categorize errors as retryable (LSP client temporary issue), user-fixable (invalid file path), or fatal (project not detected). Agents cannot decide whether to retry, ask for input, or abandon the task.