Loading ToolBench
Community Edition CAD understanding, assembly diagnostics, and design review for mechanical engineering. Analyzes STEP files and SolidWorks assemblies for BOM generation, interference detection, DFM checks, and risk scoring.
20 tools with READ_ONLY operations. Naming is verb-driven (generate, detect, extract, build, classify, assess, read, interference, clearance, adjacency, risk_score) and generally clear. Descriptions are present and substantive (avg ~120 chars), explaining WHAT and WHEN. However, parameter descriptions are sparse or missing entirely for many tools. Input schemas are visible but incomplete: many lack type constraints, enums, or detailed parameter docs. Output schemas are undocumented, LLMs cannot infer what fields to expect. Error handling is absent from visible code. No tool annotations (readOnlyHint, etc.). Overall: solid naming and basic descriptions, but parameter/output documentation and error guidance are weak.
Determine geometric adjacency by identifying which solids touch or nearly touch within a contact tolerance
Run all export-quality signals on a BOM to identify data loss, flattening, missing names, and coverage issues
Convert a component list into a bill of materials with quantity tallying and standard-part classification
Detect when a large fraction of instances fell into no category, indicating low classification coverage
Classify a part name as a standard component type (metric_screw, bearing, fastener, etc.) using heuristic pattern matching
Detect approximate minimum gaps between non-touching solid pairs, reporting pairs below a threshold
Output schemas completely undocumented. LLMs cannot infer what fields tools return, forcing them to guess or make wasteful follow-up calls. E.g., build_bom returns a list with 'item', 'part', 'qty', 'standard_part' fields, but this is invisible to the agent.
Parameter descriptions are sparse or missing. E.g., 'assess' has a complex 'bom' object parameter with nested properties (unique_parts, solid_bodies, etc.), but no description of what each field means or how to construct it. LLMs cannot reliably call this tool.
No error handling guidance. Code shows signal detection (geometry_only_export, flat_bom_export, missing_names) but tool definitions do not document what errors can occur, when to retry, or how to recover. Agents will not know how to handle failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Identify the CAD format of a file and determine if it is supported, with guidance on conversion or export
Extract de-duplicated PRODUCT names from a STEP file with Unicode decoding and sanitization
Detect when BOM hierarchy was collapsed on export (many parts at depth 1 with no sub-assembly structure)
Generate a bill of materials from an assembly, with standard-part classification and quantity tallying
Detect when an assembly tree was stripped on export, leaving only geometry with no structure (1 PRODUCT but many solid bodies)
Detect approximate solid-solid interference via pairwise boolean intersection, with volume reporting and scale guards
Determine if a file is a valid STEP file by content inspection, supporting .step, .stp, directories, and .zip archives
Detect when a large fraction of instances have no resolvable name, making name-driven analysis unreliable
Extract component (PRODUCT) names from a STEP file by text parsing without requiring a geometry kernel
Read and parse a STEP file, returning entity count and availability status of STEP kernel
Return solids found in a STEP file with volume and centroid calculations using OpenCASCADE geometry engine
Normalize any input (STEP file, directory, or zip) to a usable STEP file path, with automatic extraction and selection of the best STEP from archives
Generate a simple 0-100 weighted risk score from free-tier signals (interferences, DFM blockers, safety factor, resonance)
Detect and report when part names were Unicode-encoded in STEP (ISO-10303-21 \X2\) and have been decoded
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 20 tools are READ_ONLY, but this is not declared in the tool definitions. Agents cannot infer safety properties and may make incorrect retry decisions.
Parameter constraints missing. E.g., 'interference' accepts min_volume_mm3, max_solids, max_pairs but no min/max bounds are declared. LLMs may pass absurd values (negative volumes, 1M solids) that break the tool or cause timeouts.