Loading ToolBench
MCP server for Asset Administration Shell (AAS) exploration and parsing. Provides tools to fetch metainformation from AAS servers and parse AASX packages into structured data.
Two tools with acceptable naming but weak descriptions and incomplete schema documentation. Schemas are present in code but sparse on constraints, defaults, and output structure. No error handling guidance, no idempotency statements, no pagination support despite likely large result sets. Tool definitions appear to be inferred from code inspection rather than explicit registration with complete metadata, applying the 50 cap for inferred definitions.
Given an endpoint of an AAS server, fetch the metainformation of AAS instances and their submodels hosted on the AAS server.
To parse AAS properties for more detailed information. Given the AAS id and server endpoint, this tool fetches the full AAS as AASX and extracts property paths into a structured table.
Tool descriptions lack WHEN/WHY context. Both tools describe WHAT they do but not when to use aas_explore vs aas_parser, what prerequisites exist (valid AAS server endpoint format, expected response size), or how they compose into workflows.
No output schema documentation. aas_explore returns 'metainformation of AAS instances and their submodels', unstructured. aas_parser returns 'structured table' but column names, types, and field list are absent from tool definition. LLMs cannot predict downstream tool calls or data extraction.
Input parameters use example-based guidance ('e.g., http://localhost:8081', 'e.g., https://dpp40.harting.com/shells/...') rather than formal constraints (regex, format, pattern). LLMs latch onto examples and pass them literally. Replace with enums, format keywords (uri, url), or pattern regexes.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
No error handling or recovery guidance. Tools make HTTP calls to remote endpoints (aas_explore, aas_parser download AASX via BasyxApiClient). No description of what fails when (timeout, invalid endpoint, malformed response, permission denied), whether errors are retryable, or what the LLM should do next.
aas_parser combines fetching AASX and parsing into one operation. If either step fails (download timeout vs parse error), error handling is muddled. Consider splitting into separate tools: fetch_aasx_package and parse_aasx_file. This follows the single-responsibility principle and allows agents to retry independently.
No pagination or result-limit documentation. aas_explore 'fetches the metainformation of AAS instances and their submodels', if a server hosts 1000 submodels, does the tool return all 1000? If yes, the LLM context window explodes. If it truncates, the description must state the limit and offer pagination parameters (limit, offset, cursor).
Parameter descriptions for 'endpoint' and 'id' do not explain format constraints, valid ranges, or how to obtain them. 'The base URL of the AAS server, e.g., http://localhost:8081' teaches by example rather than rule. A user with a server at port 8090 might not know if it's valid; an LLM will guess. Rewrite: 'The base URL of the AAS server (http:// or https://, port 1-65535, no trailing slash)'.
Tool registration appears inferred from tool_list iteration in server.py. Explicit tool definitions with full metadata (name, description, input schema, output schema) are not visible in the provided code. This limits confidence in schema completeness.