Loading ToolBench
Agentic PCB design accelerator converting natural language prompts into SKiDL code and schematics
Circuitron has 8 tools with complete naming and input schemas, but suffers from severely underdeveloped descriptions and missing output schema documentation. Tool descriptions are extremely brief (10-30 chars), well below the 10-1024 char baseline and far too terse to guide LLM selection. Parameter descriptions exist but are minimal. No output schemas are documented, making it impossible for agents to plan downstream tool calls or extract return data reliably. The 'calculate_id' correlation parameter in execute_calculation is undocumented and opaque. Tools like execute_calculation and run_runtime_check_tool lack guidance on error recovery, retry eligibility, or consequences. The server imports openai-agents (legacy framework), suggesting potential spec drift. Overall: tool naming is solid, schemas are present, but descriptions are critically sparse and lack the depth needed for confident LLM selection.
Execute pure-Python maths code *generated by the LLM* in an isolated Docker container.
Execute the final SKiDL script to generate KiCad schematic and netlist files.
Return pin details by creating Part object and accessing pins directly.
Get guidance on using the SKiDL knowledge graph for code generation and validation.
Run electrical rule checks on the generated KiCad schematic.
Execute the SKiDL script in an isolated Docker container and capture runtime errors.
Search KiCad footprint libraries using ``skidl.search_footprints``.
Tool descriptions are critically underdeveloped (10-30 chars). The rubric baseline is 194 chars; Circuitron averages ~20 chars. Descriptions like 'Run electrical rule checks on the generated KiCad schematic' lack context on WHEN to call the tool, WHAT it returns, or error recovery paths. This makes it impossible for LLMs to confidently select the right tool or understand consequences.
No output schemas are documented for any tool. Agents cannot plan downstream calls or extract necessary fields. If execute_calculation returns stdout, JSON, or structured error, that must be declared. If search_kicad_libraries returns a list of results with IDs, names, and libraries, the LLM needs to know the field names for chaining.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 19 | - | v1 |
Search KiCad libraries using ``skidl.search``.
execute_calculation and run_runtime_check_tool are marked IRREVERSIBLE but lack error handling guidance. When an LLM passes malicious code or triggers a timeout, what happens? Can it retry? Must it call a recovery tool? Should it inform the user? No guidance means agents either fail silently or attempt unsafe retries.
execute_calculation accepts 'code' (arbitrary Python) and 'calculation_id' (undefined purpose). The 'calculation_id' parameter is not documented, does it enable replay? Trace correlation? Resume? Without explanation, LLMs cannot use it reliably. Additionally, accepting arbitrary Python code from an LLM without sandboxing guidance is noted as high-risk.
Tool names 'run_erc_tool', 'run_runtime_check_tool', 'execute_final_script_tool' contain redundant 'tool' or 'run' + 'tool' suffixes (e.g., 'run_erc_tool', is 'erc_tool' the action or the resource?). Cleaner: 'run_erc', 'check_skidl_runtime', 'generate_schematic'. Current names create parsing ambiguity for LLMs.
search_kicad_libraries and search_kicad_footprints accept 'max_results' integer with no bounds specified. Rubric requires min/max constraints (e.g., 1 - 100). Unbounded integers risk runaway calls returning thousands of results, exhausting context and token budgets.
extract_pin_details requires 'library' and 'part_name' but provides no guidance on discovering valid library names or handling case sensitivity. If 'library' is case-sensitive or must match an internal enum, that must be explicit. Current description assumes prior discovery.