Loading ToolBench
Code structure for coding agents: classes, functions, callers and headings with line numbers, as an MCP server and the sct shell command. 20+ languages, tree-sitter, no index.
Scantool provides 13 well-named code analysis tools with clear, descriptive documentation. All tool names follow verb_noun convention and are specific (preview_directory, scan_file, search_structures). Most tools have detailed descriptions explaining purpose and use cases. However, there are significant gaps in schema completeness: while parameters are documented, many lack explicit type definitions in the visible code, and output schemas are not documented. Error handling guidance is minimal. The primary tool (preview_directory) has excellent documentation with multiple parameters, defaults, and usage guidance, but this quality is inconsistent across the toolset.
Find callers of a given function
Find divergence in peer call patterns
Get git history for a file or reference
List directories in the filesystem
Find overlapping code structures
Intelligent directory preview - analyzes all file types including code, markdown, text, HTML, CSS, SQL, and config files. PRIMARY TOOL - Use this instead of ls/find/grep for project exploration! This tool automatically analyzes code structure, entry points, and architecture. Much faster and more informative than manual ls/grep exploration.
Resolve code references
Output schemas not documented. Tools return structured data but LLMs cannot see what fields to expect in responses, forcing parsing of unstructured text and preventing proper chaining of downstream operations.
Most tools lack detailed descriptions (40-60 chars vs recommended 100-200 chars). Tools like 'scan_diff', 'find_divergence', 'surface', 'overlap', 'resolve', 'history' have descriptions under 50 characters, which is insufficient for LLM tool selection. Examples: 'Scan differences between versions of code' (45 chars), 'Surface code structures' (23 chars), 'Resolve code references' (23 chars).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 24 | - | v1 |
Scan differences between versions of code
Scan directory for code structures
Scan a single file for code structures
Scan file content without downloading to disk
Search for code structures by name or content pattern
Surface code structures
Parameter descriptions lack specificity on constraints and formats. Many parameters like 'pattern' (scan_directory, search_structures), 'focus' (scan_file), 'reference' (resolve), and 'function_name' (callers) have minimal descriptions without explaining valid formats, ranges, or constraints. LLMs cannot validate input without explicit format guidance.
No error handling guidance. Tools provide no documentation on what errors can occur, how to recover, or which operations are retryable. Without error classification, agents cannot determine whether to retry, ask the user, or abort.
Vague tool names reduce clarity. 'surface' (what does it surface?), 'overlap' (overlap of what?), 'resolve' (resolve to what?), and 'find_divergence' (divergence in what context?) are ambiguous without reading the description. These should be 'surface_code_structures', 'find_overlapping_structures', 'resolve_code_references', 'find_call_pattern_divergence'.
Undocumented parameter dependencies. 'preview_directory' accepts 'depth' with values 'quick', 'normal', 'deep' and 'part' to filter output, but no description of how these interact or which combinations are valid. 'focus' in scan_file depends on the file being scanned but has no guidance on format.
No pagination or result limiting documented for tools returning lists. 'scan_directory' with 'max_files=10000' could return enormous datasets, and tools like 'search_structures' and 'find_divergence' have no documented limit, risking context window exhaustion.
Tool naming ambiguity: 'scan_*' tools (scan_directory, scan_file, scan_file_content, scan_diff) all start with 'scan' but do different things (recursive directory analysis vs single file analysis vs text scanning vs diff analysis). Without reading descriptions, LLMs conflate these tools.