Loading ToolBench
The server provides 13 well-structured tools with consistent naming patterns (all starting with 'treesitter_' prefix and using verb_noun convention). Tool descriptions are present and moderately detailed (avg 150-200 chars), exceeding the 20-char minimum. Input schemas are visible and include type definitions. However, several critical gaps reduce the score: (1) output schemas are not formally documented, the code shows what fields are returned, but LLMs cannot see structured schema definitions; (2) error handling is present but generic, lacking recovery guidance; (3) parameter descriptions vary in quality, with some being vague about constraints; (4) the 'output_file' pattern, while useful, is repeated across all tools and complicates the interface; (5) no pagination support despite tools that could return large result sets; (6) no discussion of limits or truncation behavior. The STDIO-only transport (below in Protocol Readiness) is the binding constraint that prevents this from scoring higher.
Analyze a source code file and extract symbols (functions, classes, etc.).
Return a cursor-style view with the focused node, its ancestors, siblings, and children at a given point.
Search for a specific function definition by name.
Find all usages of a symbol by name in a source file.
Search for variable declarations and usages by name.
Extract and return the complete Abstract Syntax Tree (AST) for a source file.
Generate a call graph showing function calls and their relationships.
Output schemas not formally documented. While function docstrings describe return fields (file_path, language, symbols, errors), LLMs cannot parse structured schema definitions. According to pattern:tool requirement, 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls and extract the right data.'
No pagination support for tools that return lists (symbols, nodes, call graphs, usages). According to pattern:paginated-result, 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor. Without pagination, large results blow the context window.' treesitter_get_ast, treesitter_analyze_file, and others could produce unbounded output.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
List all dependencies (imports/includes) from a source file.
Return the AST node covering a specific point (row, column) in the source code.
Return the smallest AST node covering a given range in the source code.
Extract the source code text for a given line/column range.
List all programming languages supported by this MCP server.
Run a custom Tree-sitter S-expression query against a source file.
Error handling is generic and lacks recovery guidance. All tools return {'error': 'message'} but do not classify errors as retryable, user-fixable, or fatal. Per pattern:recovery-guide, 'Error responses must tell the LLM what to do next.' For example, when file is not found, the tool should suggest 'Verify the file path and try again' or 'Use list_files() to discover available files.'
Parameter 'output_file' is repeated in all 13 tools, creating coupling and interface bloat. Per pattern:tool, 'Each tool should do exactly one thing.' The output_file mechanism mixes I/O concerns into every tool. Better: make output handling a separate capability (e.g., via a write_result tool or response streaming) so the core tools remain focused on analysis.
Parameter descriptions lack detail on constraints and expected formats. For example, 'row' and 'column' are described as 'Row number (0-based)' and 'Column number (0-based)' but do not specify valid ranges, what happens if row exceeds file length, or whether columns must be within line bounds. Per pattern:tool-description, 'Describe the expected format, range, and allowed values directly in the parameter description.'
The 'include_source' parameter in treesitter_find_function and treesitter_find_variable is a boolean but its behavior is not fully specified. Does it include full source code or a snippet? Is there a size limit to prevent context explosion? Per pattern:tool-description, absence of such details forces LLMs to guess.
treesitter_run_query accepts a custom Tree-sitter S-expression query as a free-form string with no validation hints or error guidance. Per pattern:constrained-input, LLMs should be guided on valid query syntax. The description should include a link to Tree-sitter query documentation or example queries to reduce hallucination.
Tool descriptions do not specify what 'file_path' argument means in terms of relative vs. absolute paths, or whether ~/ expansion is supported. The normalize_path() function suggests expansion is supported, but the descriptions do not communicate this to LLMs. Per pattern:tool-description, 'Write descriptions as if prompt-engineering. State WHAT the tool does, WHEN to use it, and any prerequisites.'