Loading ToolBench
An MCP server for analyzing Python codebase structure, extracting AST outlines, searching symbols, auditing docstring coverage, and mapping import dependencies.
Five tools with clear action verbs and reasonable descriptions (avg 180 chars), but critical gaps in schema documentation and parameter descriptions. All tools are READ_ONLY and return JSON strings, but output schemas are not formally documented. Parameter descriptions exist but lack format/constraint details. No error handling guidance, no pagination support despite tools returning potentially large result sets. Tool composition is sound (each does one thing), but missing idempotency guarantees and recovery hints.
Parse a Python file using Python's AST parser and return a detailed outline of its docstrings, imports, top-level classes, functions, variables, and line counts.
Locate a specific class within a Python file and retrieve all its methods, argument signatures, type annotations, line numbers, and docstrings.
Audit Python files in a directory to identify modules, classes, and functions that lack docstrings, calculating overall docstring coverage percentage.
Recursively search a directory or file for classes, functions, methods, or variables matching the target symbol name.
Scan all Python files in a directory to build an import dependency graph and output both a structured map and a Mermaid diagram definition.
Output schemas not documented. All tools return JSON strings, but the structure of that JSON (fields, types, nesting) is not declared. LLMs cannot plan downstream operations or extract specific fields without guessing the response shape.
No pagination or result limits declared. Tools like get_imports_graph and find_missing_docstrings scan entire directories and return all results. Large codebases could return thousands of items, exhausting context windows. No limit, offset, or cursor parameters.
Parameter descriptions lack format/constraint details. 'file_path' and 'target_dir' have no guidance on absolute vs relative paths, symlink handling, or error cases. 'symbol_name' accepts 'partial name' but no regex pattern or matching rules documented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No error handling guidance. Tools return error dicts (e.g. {'error': 'File not found'}) but do not guide LLM recovery. No distinction between retryable (transient I/O) vs user-fixable (invalid path) vs fatal errors.
No idempotency guarantees. Tools are read-only (safe to retry), but this is not declared via toolAnnotations (readOnlyHint). LLMs cannot confidently retry on transient failures without explicit marking.