Loading ToolBench
MCP server that executes SimPy simulation code and generates Mermaid state diagrams for hardware specification verification
Two tools with minimal documentation and no input validation. execute_simulation_code has a basic description but lacks parameter details, constraints, or error guidance. generate_mermaid_diagram similarly lacks depth. Neither tool documents output schemas. No error handling guidance for LLMs. Parameter descriptions are absent (code_str and spec lack detail). No mention of prerequisites, when to use each tool, or what happens on failure. Schemas are inferred from type hints but lack formal JSON Schema structure with constraints.
Safely executes SimPy simulation code and returns the logs.
Convert spec into Mermaid stateDiagram-v2.
execute_simulation_code accepts arbitrary Python code via exec() with minimal sandboxing. No input validation, no constraints on code complexity, no timeout mechanism. Comment acknowledges 'real production environment should be sandboxed' but lite version is unsafe. LLM could pass malicious code.
Parameter descriptions missing entirely. 'code_str' parameter has no description explaining format, constraints, or expected content. 'spec' parameter lacks description of required structure (what keys must exist in transitions/states dicts?). LLMs cannot infer parameter meaning from names alone.
No output schema documentation. execute_simulation_code returns a string but LLM cannot know if it contains structured logs, error traces, or both. generate_mermaid_diagram returns Mermaid syntax but no schema describes the format or structure. LLMs need documented return types to plan downstream actions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Error handling provides no recovery guidance. execute_simulation_code catches exceptions and prints tracebacks but does not categorize errors (retryable vs fatal) or suggest next steps. LLM receives raw stderr output with no actionable guidance.
generate_mermaid_diagram does not validate spec structure. If 'transitions' or 'states' keys are missing or malformed, tool silently produces incomplete output. No error message guides LLM to fix the input.