Loading ToolBench
MCP server for creating buildingSMART IDS files with 100% compliance
This server has 17 tools with mixed quality. Naming is verb-first and clear (create_ids, load_ids, add_entity_facet, etc.). Descriptions exist for all tools and most parameters, ranging 30-180 characters. However, critical gaps emerge: (1) Input schemas are present but parameter type definitions are inconsistent, many lack explicit 'type' fields in the schema JSON; (2) Output schemas are completely absent, no tool documents what it returns, forcing LLMs to guess at response structure; (3) Parameter descriptions, while present, often lack format/constraint details (e.g., 'ifc_versions' array description doesn't specify allowed values like 'IFC2X3|IFC4|IFC4X3'); (4) Error handling is not visible in tool definitions, no guidance on recovery or retryability; (5) Several tools mix concerns or have vague cardinality semantics (e.g., 'cardinality' parameter in facets could mean 'required|optional|prohibited' or other values, not enumerated). The tool set is reasonable for IDS domain, but the definitions lack production rigor. Average per-tool score: 52.
Add an attribute facet to a specification.
Add numeric bounds restriction.
Add a classification facet to a specification.
Add an entity facet to a specification. IMPORTANT: IDS 1.0 allows only ONE entity facet per applicability section. If you need multiple entity types, create separate specifications.
Add enumeration restriction (list of allowed values).
Add string length restriction.
Add a material facet to a specification.
NO OUTPUT SCHEMAS DOCUMENTED. All 17 tools lack return-value documentation. LLMs cannot predict response structure, field names, or types. This violates the fundamental requirement that 'Tools returning lists should accept page/offset and limit parameters and return a total count.' For example, validate_ids() likely returns pass/fail + errors, but this is invisible to the agent.
PARAMETER ENUMS NOT ENFORCED. Many parameters accept enums but lack 'enum' constraint in schema. Examples: (1) 'source_type' in load_ids should be enum ['file', 'string'] but is just string; (2) 'location' in facet tools should be enum ['applicability', 'requirements'] but is free-form string; (3) 'cardinality' in all facet tools should be enum ['required', 'optional', 'prohibited'] but no constraint visible. Free-form strings invite hallucinated values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 59 | - | v1 |
Add a PartOf facet to a specification.
Add pattern restriction (regex matching).
Add a property facet to a specification. IMPORTANT: The property_set parameter is REQUIRED for valid IDS export.
Add a specification to the current session's IDS document.
Create a new IDS document for this session. Session is automatically tracked by FastMCP - no session_id parameter needed!
Export IDS document to XML file using IfcTester. Uses current session automatically - no session_id parameter needed!
Get current session's IDS document structure. Uses current session automatically - no session_id parameter needed!
Load an existing IDS file into the current session. Replaces any existing IDS in this session.
Validate current session's IDS document. Validates: 1. Required fields present (title, specifications, etc.) 2. Each specification has applicability 3. IFC versions are valid 4. XSD schema compliance (via IfcTester)
Validate an IFC model against the current session's IDS specifications. This bonus feature leverages IfcTester's IFC validation capabilities.
PARAMETER DESCRIPTIONS LACK CONSTRAINTS. Format, range, and allowed values missing. Example: 'ifc_versions' in add_specification says 'List of IFC versions (e.g., ["IFC4", "IFC4X3"])' but does NOT enumerate valid values (IFC2X3, IFC4, IFC4X3, etc.). 'min_occurs' says 'Minimum occurrences (0 = optional)' but does not state if negative values are rejected. 'max_occurs' accepts 'int or "unbounded"' but the string value is not clearly the literal string 'unbounded'. These ambiguities force LLMs to guess.
PARAMETER TYPE DEFINITIONS INCOMPLETE. JSON Schema in input objects lacks explicit 'type' for some fields. Example: add_bounds_restriction has min_inclusive, max_inclusive, min_exclusive, max_exclusive all marked 'number' but no minInclusive/maxInclusive constraints in the schema object itself. This forces validation to happen at runtime or in the description, increasing LLM error rates.
ERROR HANDLING NOT VISIBLE. No tool definitions include error guidance or recovery paths. What happens if add_entity_facet() fails because spec_id does not exist? Does it return a 404-like error with 'Try add_specification() first'? Or a cryptic exception? No recovery pattern is documented.
SESSION MANAGEMENT MIXING. create_ids description says 'Session is automatically tracked by FastMCP, no session_id parameter needed!' This is vague and suggests stateful session handling via FastMCP context. However, MCP spec (2026-07-28) requires stateless request handling, each request must be self-contained. If FastMCP's Context object is holding session state across requests, this violates the spec's move away from Mcp-Session-Id and stateful initialize.
FACET COMPOSITION UNCLEAR. Tools add_entity_facet, add_property_facet, etc., all take 'location' (applicability or requirements) but the relationship between location and cardinality is not explained. Can you set cardinality='prohibited' in applicability? What does that mean semantically in IDS? The descriptions do not clarify valid combinations.
MISSING IDEMPOTENCY HINTS. Irreversible operations like export_ids, load_ids, add_specification lack indication of whether they are safe to retry. If an LLM calls create_ids twice with identical input, does it overwrite the session or error? Tool descriptions do not clarify, violating the pattern that 'agents retry on ambiguous failures, non-idempotent tools risk duplicate side effects.'
EXPORT OUTPUT AMBIGUITY. export_ids description says 'returns XML string if not provided' (when output_path is not given). However, how is this XML formatted? Is it pretty-printed? Does it include XML declaration? What encoding? LLMs need to know what to expect so they can parse or pass it downstream.