Loading ToolBench
Model Context Protocol (MCP) server for Python-based Cocotb hardware testbenches
Server has 4 well-defined tools with complete input schemas and descriptions. Tool names follow verb_noun convention (cocotb_run, cocotb_list_tests, cocotb_generate_runner, cocotb_toolchain_info). Descriptions are substantive (100-150 chars), explaining what each tool does and when to use it. All parameters have type definitions and descriptions. However, output schemas are not documented, LLMs cannot predict response structure. Error handling is minimal; no recovery guidance or actionable error messages visible in code. Parameters lack constraints (e.g., timeout_ms unbounded, simulator enum is good but simulator param in generate_runner lacks enum). No tool annotations (readOnlyHint/destructiveHint). cocotb_run is destructive (modifies filesystem, runs simulation) but lacks explicit warning.
Generates a deterministic, standard Cocotb simulation Makefile for a given Verilog DUT and Python test module.
Discovers all @cocotb.test() coroutines and their docstrings in a Python testbench file without executing simulation.
Compiles a Verilog DUT, executes an asynchronous Python cocotb testbench, and parses results into structured JSON test reports with assertion tracebacks.
Returns active container or host execution runtime info and versions of Cocotb, Python, and simulator.
Output schemas not documented. LLMs cannot predict response structure or plan downstream tool calls. cocotb_run returns JSON test reports but structure is undocumented; cocotb_list_tests returns test discovery but fields are unknown.
cocotb_run is destructive (compiles, executes, modifies filesystem) but lacks tool annotations (destructiveHint) and no confirmation/dry-run pattern. Agents cannot distinguish safe from risky calls.
Error handling not visible in tool definitions. No recovery guidance, error classification, or actionable error messages. If cocotb_run fails, LLM has no guidance on retry vs user-fixable vs fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 71 | 2026-07-28+ | v2 |
Parameter constraints incomplete. timeout_ms is unbounded (should be 1 - 300000 ms); simulator in cocotb_generate_runner lacks enum (cocotb_run has enum but generate_runner does not). Inconsistent parameter validation invites hallucinated values.
cocotb_toolchain_info has empty inputSchema properties (no parameters documented). While the tool accepts no inputs, the schema should explicitly state this with a description explaining what runtime info is returned.