Loading ToolBench
Codanna is a Rust-based code intelligence MCP server with 9 read-only tools for semantic code search, symbol analysis, and documentation retrieval. All tools have descriptions and basic input schemas, but several critical gaps limit production readiness: (1) Output schemas are not documented, responses are unstructured and LLMs cannot plan downstream actions; (2) Parameter descriptions are minimal, often just restating the parameter name; (3) No error handling guidance or recovery paths documented; (4) Parameters lack constraints (enums, ranges, format specs); (5) Tool composition is reasonable but responses lack chaining IDs needed for multi-step workflows. The codebase shows solid engineering (Rust, tantivy-based indexing, multiple parser support), but the tool interface falls short of production standards. Average tool score: 62/100.
Analyze the impact of changes to a symbol. Shows dependent code and call chains.
Find all callers of a specific function or method. Returns call graph information.
Find a specific symbol by name and optionally filter by kind. Returns exact matches with location information.
Get all function/method calls made by a specific symbol. Returns call sites and targets.
Get information about the current index. Returns statistics about indexed symbols and files.
Search project documentation and markdown files. Returns relevant documentation sections.
Search for symbols by pattern or keyword. Returns filtered symbol matches.
Output schemas not documented for any tool. LLMs cannot infer response structure, plan downstream tool calls, or extract required fields for chaining. Users must reverse-engineer responses through trial and error.
Parameter descriptions are minimal and terse (3 - 20 chars). Many lack format guidance, constraints, or examples. 'Pattern or keyword to search for' does not clarify regex vs glob vs substring. 'The depth of impact analysis' does not specify numeric range or semantics (AST depth? call chain depth?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | <=2025-11-25 | v2 |
Search project documentation using semantic understanding. Finds relevant markdown files and documentation.
Search for symbols and references using semantic understanding of code. Returns relevant code snippets with context.
No pagination support documented or implemented. Tools with 'limit' parameters (semantic_search_*, search_symbols, find_callers, search_documents) lack next_cursor, page, or offset. Without pagination, large result sets blow the context window and LLMs cannot retrieve all matches.
Semantic composition: semantic_search_with_context (#1), semantic_search_docs (#2), search_symbols (#4), and search_documents (#8) overlap significantly. An LLM with access to all four will struggle to choose the right one. Consider consolidating or adding explicit 'prefer X when' guidance in descriptions.
No error handling guidance. Tools do not document: what happens on no matches? Invalid symbol names? Indexing errors? Timeout handling? LLMs cannot self-correct without explicit error classifications and recovery paths.
Parameters lack enumeration constraints. 'kind' in find_symbol and search_symbols should declare allowed symbol types (function|class|struct|enum|module|import|variable|type). Free-form strings invite hallucinated values.
Numeric parameters (limit, depth) lack range constraints. No indication of min/max or performance implications. A limit=10000 or depth=1000 could trigger timeouts without documented bounds.
Ambiguous tool naming and semantics. 'get_calls' is ambiguous (calls FROM or TO?). 'analyze_impact' does not clarify direction (downstream breakage or upstream dependencies?). Descriptions should explicitly state directionality.