Loading ToolBench
Exposes the archdiagram engine to MCP-capable agents. Validates architecture specs, renders diagrams (PDF, draw.io, VSDX), and searches/lists vendor service catalogs.
Four tools with complete schemas and descriptions, but quality is uneven. validate_spec and render_diagram have good parameter documentation; search_catalog and list_catalog are minimal. No tool annotations (readOnlyHint/destructiveHint). Output schemas are documented in code but not formally declared in tool definitions. Error handling is present but lacks recovery guidance. Naming is clear and verb-driven. No security issues detected (no secrets in params). Average description length ~120 chars (below 194 baseline), and parameter descriptions are sparse for catalog tools.
List all catalog services, optionally filtered by vendor.
Render a spec to a diagram file. Args: spec_json: the architecture spec as a JSON string. fmt: one of "pdf", "drawio", "vsdx". out_path: absolute or relative path to write. drawio_mode: "portable" (embedded icons, best for Lucid) or "native". icon_cache: optional icon cache directory override.
Search the vendor service catalog by service id or label.
Validate an architecture spec (JSON string). Returns ok + errors/summary.
Minimal parameter descriptions for catalog tools. search_catalog has only 'search query string' for the query param; list_catalog has only 'optional vendor filter' for vendor. LLMs cannot infer valid query syntax or vendor names without examples or constraints.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). render_diagram is destructive (writes files) but not marked. validate_spec and catalog tools are read-only but not annotated. Agents cannot infer safety properties without explicit hints.
Error responses lack recovery guidance. render_diagram returns {'ok': False, 'errors': [...]} but does not suggest next steps (e.g., 'Invalid format. Try pdf|drawio|vsdx' or 'Icon cache not found. Check path or omit to use default'). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Output schemas not formally documented in tool definitions. Code shows return dicts (e.g., {'ok': True, 'summary': {...}}) but tool descriptions do not specify field names, types, or structure. LLMs must infer output shape from code.
render_diagram's fmt parameter accepts 'pdf', 'drawio', 'vsdx' but is not declared as an enum in the schema. Free-form string invites hallucinated formats. Schema shows type:string with description but no enum constraint.