Loading ToolBench
A Model Context Protocol server that exposes SysML v2 API capabilities as MCP tools, allowing AI applications to interact with SysML v2 model repositories through standardized protocol interfaces.
The server provides 28 well-structured SysML v2 API tools with consistent naming conventions (verb_noun pattern: sysml_get_*, sysml_create_*, sysml_delete_*, etc.). All tools have descriptions and input schemas present in the source. However, descriptions are terse (averaging 40-80 chars, well below the 194-char baseline for A+ tools), lack context on WHEN to use each tool, and omit recovery guidance. Parameter descriptions are minimal. No output schemas are documented, the source shows only input schemas. Error handling is not visible in the tool definitions. Tool composition is sound (each does one thing), but critical details for LLM selection are missing. The 'echo' tool and API client tools follow the verb_noun pattern consistently, but descriptions do not explain prerequisite dependencies (e.g., 'get_element_by_id requires a valid projectId and commitId from prior calls') or when to prefer one tool over another (e.g., get_elements vs get_root_elements). No evidence of pagination result limits documented in descriptions despite pageSize parameters. The HTTP transport is modern, but tool metadata lacks the richness needed for production LLM agents.
A simple tool that returns the passed message as its response.
Create a new branch in a SysML v2 project.
Create a new commit in a SysML v2 project.
Create a new SysML v2 project.
Create a new query in a SysML v2 project.
Create a new tag in a SysML v2 project.
Delete a branch from a SysML v2 project.
Delete a SysML v2 project by ID.
Tool descriptions are uniformly terse (40-50 chars) and lack context. They state WHAT the tool does but omit WHEN to use it, what prerequisites exist, or how it relates to similar tools (e.g., get_elements vs get_root_elements). LLM selection suffers without this guidance.
No output schemas are documented. The source code shows only input schemas. LLMs cannot plan downstream tool calls or know what fields to expect in responses. This violates pattern:tool (Document the output schema).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
Delete a tag from a SysML v2 project.
Get differences between two commits in a SysML v2 project.
Execute an ad-hoc query in a SysML v2 project.
Execute a saved query by ID in a SysML v2 project.
Get a specific branch by ID from a SysML v2 project.
Get all branches in a SysML v2 project.
Get a specific commit by ID from a SysML v2 project.
Get changes in a specific commit from a SysML v2 project.
Get all commits in a SysML v2 project.
Get a specific element by ID from a SysML v2 project.
Get all elements in a SysML v2 project at a specific commit.
Get one specific SysML v2 project by ID.
Get all queries in a SysML v2 project.
Get a specific query by ID from a SysML v2 project.
Get relationships for a related element in a SysML v2 project.
Get root elements in a SysML v2 project at a specific commit.
Get a specific tag by ID from a SysML v2 project.
Get all tags in a SysML v2 project.
List all available SysML v2 projects.
Update an existing SysML v2 project.
Pagination not documented in tool descriptions. Tools like sysml_list_projects, sysml_get_elements, sysml_get_commits accept pageSize parameters but lack guidance on default behavior, max results, or how to iterate. LLM agents cannot reason about result truncation or plan pagination strategies.
No error handling guidance visible in tool definitions. Descriptions do not specify what errors can occur, whether they are retryable, or what the LLM should do next. Per pattern:recovery-guide, errors must tell LLMs what to do: 'Resource not found. Try search_* with a partial name.'
Destructive operations (sysml_delete_project, sysml_delete_branch, sysml_delete_tag) lack confirmation steps. Pattern:confirmation-request suggests LLMs should confirm irreversible operations before execution to prevent accidental data loss.
Parameter descriptions are minimal or absent for some parameters. E.g., 'changeTypes' in sysml_get_commit_changes lacks enum values and format guidance. Per pattern:constrained-input, free-form string arrays invite hallucinated values; should declare valid enum options.
No tool annotations visible (readOnlyHint, destructiveHint, idempotentHint). The rubric rewards current MCP patterns including tool annotations. While the 'Risk' field indicates tool safety (READ_ONLY, WRITE, DESTRUCTIVE), this metadata should be formalized in the protocol as tool annotations for proper agent authorization and safety checks.
Tool composition: no guidance on chaining. E.g., get_elements requires projectId and commitId. Tools should document prerequisite discovery steps ('Call sysml_get_commits first to obtain a valid commitId') to prevent wasted calls and guide agent planning.
'direction' parameter in sysml_get_relationships_by_related_element lacks enum constraint. Description says 'Direction of relationships (both, in, out)' but does not formally constrain to enum. Per pattern:constrained-input, enums prevent hallucinated invalid values.