Loading ToolBench
CADCLAW: automated assembly and validation for STEP CAD. Places authored parts by declared constraints, then runs geometric, structural, tolerance, and BOM-vs-CAD gates. Exposes validation, analysis, and audit checks as MCP tools.
CADCLAW MCP server has 24 tools with variable quality. Strengths: all tools have names starting with action verbs (load_, check_, compute_, export_, run_, compare_, assemble_), all have descriptions (ranging 20-180 chars), and all input schemas are visible with typed parameters. Weaknesses: output schemas are largely undocumented in the tool definitions (descriptions refer to structured returns but don't specify field names/types); error handling guidance is minimal; descriptions are functional but not optimized for LLM selection (many read like API docs rather than agent guidance); parameter descriptions are inconsistent in detail (some rich, others terse); no tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk profiles (READ_ONLY vs WRITE). Per-tool analysis below shows average schema score of 52 (parameters typed but output undocumented), average description score of 65 (present and action-oriented but lacks selection guidance), average naming score of 85 (clear verbs, but some compound names like 'check_bom_against_cad' and 'assemble_check_round' hint at multiple responsibilities).
Resolve sources and compile the assembly with CadQuery - builds authored parts into STEP assembly.
Build + inventory-check + render one review round - single-shot assembly validation with visuals.
Inspect one authored STEP component - examine properties and metadata.
Export step-by-step build STEPs, views, and BOM CSV - generate assembly documentation and build sequence.
Render the spec's declared review views - generate PNG renders of assembly from specified angles.
Validate an assembly spec without compiling - checks structure and references.
Output schemas undocumented. Tool descriptions say 'returns structured Report' or reference inline PNG content, but no field names, types, or structure are specified in source. LLMs cannot plan downstream tool usage without knowing what fields to extract.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Validate spatial relationships between part types.
BOM JSON vs STEP audit - catches mismatches between bill of materials and CAD geometry.
Public-claim audit - validates text in BOM and documentation against declared claims.
Check part dimensions against expected ranges.
Find solid-solid overlaps between parts.
Check loaded assembly part counts against expected.
Privacy / publish-boundary audit - ensures sensitive data doesn't leak in public exports.
Inventory validation with region constraints (spatial boundaries).
STEP-vs-STEP dimensional signature comparison - detects geometry changes.
Belt tension safety analysis.
Beam deflection analysis for structural validation.
Motor torque budget analysis.
Generate ordered part-removal plan for disassembly.
Environment diagnostics for CADCLAW and dependencies.
Radial or axial exploded STEP export.
Load a STEP file and return a summary of parts found.
Run the YAML-backed harness without depending on loaded MCP state.
Worst-case / RSS / Monte Carlo tolerance analysis.
Tool annotations missing. All 24 tools have explicit Risk classifications (READ_ONLY or WRITE) in source, but these are not exposed to the MCP client via readOnlyHint, destructiveHint, or idempotentHint. Agents cannot distinguish safe read-only tools from destructive writes (export_exploded_view, assemble_build, assemble_render_views, assemble_render_sequence all write files).
Descriptions lack LLM-optimized selection guidance. Examples: 'Check loaded assembly part counts against expected' (check_inventory) does not explain when to use it vs load_assembly; 'Environment diagnostics for CADCLAW and dependencies' (doctor) doesn't state when/why an agent should call it. Descriptions read like API docs, not agent prompts.
Parameter descriptions inconsistent in specificity. Some tools (e.g., tolerance_stack with 'Array of tolerance dimensions' for stack_chain) provide minimal context; others (compute_deflection) are clearer. No consistent guidance on ranges, formats, or constraints. Example: load_assembly 'labels' param described as 'Optional mapping' with no example structure, LLM must guess if it's {'sig1': 'label1'} or {'part_name': 'label'}.
Error handling guidance absent. Tool descriptions do not state what errors can occur, which are retryable, or what recovery steps to take. Example: check_inventory references '_loaded_parts is None' error but does not explain in the description that load_assembly must be called first. Agents must infer dependencies from error messages.
Compound tool names (check_bom_against_cad, assemble_check_round) suggest multiple responsibilities. 'check_bom_against_cad' combines BOM validation AND CAD comparison; 'assemble_check_round' combines build, inventory-check, AND render. These should be split (e.g., validate_bom, audit_bom_vs_cad) so agents can compose them independently.
Global state (_loaded_parts, _loaded_path, _label_map) breaks stateless MCP requirement. Calling check_inventory, check_interference, or check_adjacency without calling load_assembly first silently fails or returns misleading results. This violates stateless request handling (tools should work independently or declare clear prerequisites).
No pagination or result limits documented. Tools like run_harness, assemble_render_sequence, and check_claims that process file systems or YAML configs may return unbounded output. No limit parameter, no guidance on result size, no next_cursor/offset for large datasets.