Loading ToolBench
Evidence-grounded repository audit CLI with MCP server interface: deterministic scanner, live dashboard, and GitHub Action for code review automation
Aletheore presents a 32-tool suite for code repository analysis with consistent naming conventions and reasonable descriptions. However, the server exhibits significant gaps in schema completeness, output documentation, and error handling guidance. All tools follow verb_noun naming (aletheore_scan, aletheore_search, etc.) with clear intent. Descriptions are present and contextual, averaging ~120 characters, which is adequate but often lacks guidance on prerequisites, output structure, and error recovery. Input schemas are consistently defined with types and basic descriptions, but output schemas are entirely undocumented, critical for tools like aletheore_search_codebase and aletheore_answer that return complex results. Parameter descriptions are functional but rarely include format constraints, ranges, or examples. Error handling is minimal; most tools lack guidance on failure modes or recovery steps. The semantic search tools (aletheore_search, aletheore_search_codebase, aletheore_answer) show design maturity but no documented output format, making downstream chaining difficult. Security considerations are largely absent, tools accept repo_path as string with no path traversal validation documented.
Semantic search with LLM-generated natural language answer. Requires aletheore_index to have run first.
Impact analysis: files that depend on a given file, showing change impact scope.
Git branch metadata (head commit, tracking info). target: branch name (e.g. 'main').
Architecture cluster information and member modules. target: cluster ID or name.
Database schema: tables, columns, relationships, constraints extracted from migrations or ORM definitions.
Unused code: symbols that are defined but never called or imported.
Diff against a previous snapshot: what changed in the evidence between runs.
Output schemas completely undocumented. Tools like aletheore_search_codebase, aletheore_answer, aletheore_secrets, aletheore_vulnerabilities return complex nested structures (search results, threat data, credential findings) with zero schema documentation. LLMs cannot predict field names, types, or structure for downstream planning.
No error handling guidance. Tools lack descriptions of failure modes (e.g., what happens if repo_path is invalid, if evidence is stale, if index is missing). No recovery hints. LLMs cannot self-correct or choose alternate paths on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
HTTP endpoints, routes, and APIs exposed by the repository.
Environment variables: all configured and referenced environment variables.
Code evidence for a dependency: where it is imported, used, and what it provides.
Code evidence for an endpoint: its implementation, handlers, and related code.
Code evidence for a symbol: its definition, usage, and related code. target: symbol path (e.g. 'src/app.py:MyClass.method_name').
Health check: run evidence integrity and completeness checks on the scan output.
Hotspots: files with highest churn and maintenance burden.
Which modules import this one. target: file path exactly as it appears in evidence.
This module's own list of imports. target: file path exactly as it appears in evidence (e.g. 'src/app.py').
Builds the semantic search index on top of scan evidence. Required for aletheore_search_codebase and aletheore_answer. Reports live progress while running.
Infrastructure and deployment: detected infrastructure-as-code, container images, cloud resources.
Architecture layer violations: imports that break layering rules.
License compliance analysis: licenses of all dependencies and potential conflicts.
List all modules/files, branches, clusters, or symbols in the repository from evidence.
Symbols and files related to a target symbol: callees, callers, and nearby code in the dependency graph.
High-level repository overview: what the codebase does, its main components, and structure. Start here on unfamiliar repos.
Code ownership derived from git blame history. Optional target: a file path exactly as it appears in evidence.
Deterministic parse and dependency-graph pass for the repository. All other tools read from this evidence. Reports live progress while running.
Literal/regex search for strings in source code. Does not use semantic matching - search term must be exact or regex pattern.
Semantic search of the codebase for open-ended questions. Requires aletheore_index to have run first. Degrade more gracefully with paraphrasing than aletheore_search.
Detected secrets and credential exposure in the repository.
Static analysis findings: code quality issues, dead code, style violations.
Source code of a specific function or class. target: symbol path (e.g. 'src/app.py:MyClass.method_name').
This module's extracted functions and classes. target: file path exactly as it appears in evidence.
Dependency vulnerabilities and CVEs found in the repository.
Prerequisite dependencies not documented. aletheore_search_codebase and aletheore_answer require aletheore_index to run first, but this dependency is mentioned only in short descriptions, not in structured prerequisites/return_type guidance. An LLM may call these tools before index exists.
Parameter descriptions lack format/constraint details. repo_path is described only as 'Path to the repository' with no guidance on absolute vs relative paths, trailing slashes, or validation rules. Similar gaps in 'query' (max length? regex support?), 'target' (exact format requirements?), 'limit' (default? max?).
Input path parameters (repo_path, target) accept strings with no visible validation or sanitization documented. No protection against path traversal (../../../etc/passwd), symlink attacks, or non-existent paths. Security baseline not met.
Limited composition guidance. Tools like aletheore_find_evidence_for_dependency, aletheore_find_evidence_for_endpoint, aletheore_find_evidence_for_symbol suggest a 'find evidence' pattern but descriptions do not make clear when to use each vs. raw data tools (aletheore_symbols, aletheore_endpoints, aletheore_search_codebase). LLM may redundantly call overlapping tools.