Loading ToolBench
Code Analysis and Dependency Resolution tool - analyzes codebases to build call graphs, detect API endpoints, and perform impact analysis
CADR is a code analysis tool exposing 7 tools via MCP. Core issues: (1) Most tool descriptions are adequate but generic (100-150 chars), lacking LLM-optimized guidance on WHEN to use each tool and WHAT consequences follow. (2) Parameter schemas are visible but many lack enum constraints where applicable (e.g., 'run' and 'rec' language parameter has no validation; 'api' subcommand has enum but other params lack descriptions). (3) Tool naming is verb-forward ('analyze', 'impact', 'export') which is good, but 'api' is overloaded, it conflates discovery (list), mutation (add/delete), and configuration (global flag), mixing responsibilities. (4) No error handling guidance visible, no indication of what happens if a symbol is not found, if a path doesn't exist, or if a command fails. (5) Output schemas are not documented, agents don't know what fields to expect from 'analyze' result JSON, from 'impact' function list, or from 'export'. (6) Risk classification is present (READ_ONLY, WRITE, IRREVERSIBLE) but not reflected in tool descriptions or annotations. (7) Tools 'run' and 'rec' execute arbitrary commands (IRREVERSIBLE risk), no confirmation step or dry-run option documented.
Analyzes the codebase and returns summary statistics including function count, call edges, total nodes, and total edges
Manages API endpoint detection and external endpoint registry for Flask and FastAPI applications
Exports the complete call graph as JSON data structure
Performs impact analysis to find all functions that call a given symbol, with their line numbers
Records runtime trace execution to .cadr/traces/last_run.jsonl file
Executes a command with runtime tracing enabled to capture function calls
Starts an HTTP server on localhost:6433 serving the call graph data via REST API
No output schemas documented for any tool. Agents cannot infer what fields to expect from analyze, impact, export, api (list), or run. This forces agents to guess or make trial-and-error calls.
Parameter 'language' in 'run' and 'rec' tools is a free-form string with no enum constraint, despite visible support for specific languages (python, javascript, go, rust, java, ruby, c). LLMs will hallucinate invalid values like 'node.js' or 'javascript3'.
Tools 'run' and 'rec' execute arbitrary commands (IRREVERSIBLE risk) with no dry-run, confirmation step, or sandbox indication. Agents could accidentally execute destructive commands (rm -rf). No error handling guidance on what to do if command fails.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
'api' tool name is a noun, not a verb. Violates verb_noun naming pattern. Overloads multiple responsibilities (add, list, delete, global mode) in one tool, should be split into separate tools or renamed to 'manage_api_endpoints' to clarify intent.
No error handling guidance in any tool description. No indication of what happens if path doesn't exist, symbol not found, invalid language, or network issues. LLM cannot self-correct or retry intelligently.
Tool descriptions are generic and do not explain WHEN to call each tool vs similar ones. E.g., 'analyze' vs 'export', when should agent choose analyze (summary) vs export (full data)? What are latency/cost trade-offs? Missing context forces LLM guessing.
Parameters 'method' and 'url' in 'api' tool add subcommand lack descriptions. 'method' description says 'HTTP method for add command (GET, POST, PUT, DELETE, PATCH, OPTIONS, HEAD)' but should be an enum constraint, not a help string. 'url' has no description at all.
Tool 'serve' starts a long-running HTTP server but description does not clarify if it blocks agent execution, how agent stops the server, or whether it returns a URL/port. Agents cannot determine lifecycle.