Loading ToolBench
Visual code workflow explorer for coding agents. Builds deterministic, source-grounded Mermaid flowcharts via MCP, local-first, with no API key.
CodeDebrief defines 6 tools via fastmcp framework with explicit schemas and descriptions. However, quality is inconsistent across tools. Strengths: all tools have descriptions (10-20 lines each); input schemas are visible with type hints and parameter descriptions. Major weaknesses: (1) Naming lacks action verbs, 'agent_context', 'expand_slice', 'query_model' are weak; should be 'get_code_context', 'expand_workflow', 'search_model'. (2) Parameter descriptions are present but generic and lack specificity about format, constraints, and valid ranges. (3) Output schemas are NOT documented anywhere in the source, tools describe what they return in prose (e.g. 'Returns one bounded workflow_slice plus compatible...') but no structured output schema is visible. (4) Token budget parameters appear in 4 tools with defaults (900, 1200, 1500) but no explanation of what a token is in this context or how the budget is enforced. (5) Error handling is mentioned in docstrings ('Surfaced to the agent as a clean {"error": ...}') but not formally specified with error codes, recovery guidance, or categorization. (6) One tool (update_codedebrief) performs mutations, but descriptions don't explicitly state 'WRITE' or 'DESTRUCTIVE', this appears only in the Risk field in the metadata, not the tool description itself.
Primary agent entrypoint for code-logic questions and change impact. Accepts the context a coding agent naturally has: user question, changed files, selected code/current file, or focused flow/symbol/dependency targets. Returns one bounded workflow_slice plus compatible query, impact, navigation, source-range, guardrail, and optional visual snapshot context.
Widen or deepen a workflow slice from stable flow handles.
Analyze the impact of code changes on the workflow.
Query the code model using symbol, file path, or scope filters.
Synchronize codedebrief.json and codedebrief.md artifacts with source code changes.
Validate and report on codedebrief.json and codedebrief.md artifact integrity.
Output schemas are not documented. Tools describe return values in prose (e.g. 'Returns one bounded workflow_slice plus compatible...') but no structured output schema is visible in the source. LLMs cannot plan downstream tool calls or extract specific fields without a documented schema.
Tool names lack action verbs and are vague. 'agent_context', 'expand_slice', 'query_model', 'impact_analysis' are nouns or noun phrases. Verb-based names (get_, create_, search_, analyze_, expand_) help LLMs infer what happens when called without reading descriptions.
Parameter descriptions lack specificity about format, constraints, and valid ranges. E.g., 'question' could be any string; 'direction' is explained in prose as 'neighbors, callers, callees' but is not a formal enum constraint; 'token_budget' appears in 4 tools with defaults but no explanation of what a 'token' is in this context or how the budget is enforced; 'scope' and 'dependency_path' lack examples or format specs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
The update_codedebrief tool performs a WRITE operation (syncs artifacts, may overwrite files) but the description does not explicitly state 'This modifies on-disk artifacts' or warn of side effects. The Risk field says 'WRITE' but the tool description omits this critical context.
No error handling guidance. MCP_INSTRUCTIONS mention 'Surfaced to the agent as a clean {"error": ...}' but tools have no documented error codes, recovery steps, or categorization (retryable vs user-fixable vs fatal). If a tool fails, the LLM has no guidance.
Tool disambiguation is unclear. agent_context, expand_slice, query_model, and impact_analysis all appear to search or query the code model but descriptions don't explain the distinction or when to pick one over another. agent_context says 'Accepts the context a coding agent naturally has' but doesn't explain when query_model should be used instead.
Domain concepts (workflow_slice, flow_id, symbol, dependency_path) are used in tool descriptions and parameters but not defined. LLMs unfamiliar with CodeDebrief internals will not understand what these mean or how to construct valid values.