Loading ToolBench
solder-mcp has well-structured tool definitions with clear naming (verb_noun pattern) and comprehensive descriptions (avg 180 chars). All 10 tools have input schemas with type definitions and parameter descriptions. However, output schemas are not documented, LLMs cannot predict return structure. Error handling is minimal (no recovery guidance, no error categorization). Tool descriptions lack WHEN-to-use context and dependency hints. Parameters like 'detail' and 'filter' are present but lack constraint documentation (e.g., glob pattern syntax not formally specified). The remote_agent_run tool exposes a timeout_ms parameter without bounds, risking runaway calls. Overall: solid foundation, but missing output documentation and error guidance that would elevate to 75+.
Compare two netlists and report differences in components and nets. Useful for design revisions and change tracking.
List all components (symbols) in a schematic or netlist with their references, values, and footprints. Useful for BOM generation and design review. Pass detail=true to include all properties and coordinates.
List all footprints in a PCB file with their references, positions, and pad counts. Useful for layout review and footprint validation.
List all nets (electrical connections) in a netlist with their connected pins. Useful for verifying connectivity and finding unconnected nets.
Get a summary of a KiCad file. Use this FIRST when encountering any KiCad file to understand its scope before drilling into details. Returns concise counts by default; pass detail=true for the full parsed structure with all properties and coordinates. Examples: 'overview of main.kicad_sch', 'show me the PCB stats'.
Search for components, nets, or footprints across a KiCad project by name or reference. Supports glob patterns and substring matching.
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Tools return JSON but no schema is visible in definitions.
No error handling guidance. Tools return errors but do not tell LLM what to do next (retry? ask user? fatal?). No recovery hints.
remote_agent_run timeout_ms parameter unbounded. No min/max specified. LLM could pass 0 or 999999999, causing failures or hangs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 70 | 2026-07-28+ | v2 |
List all tracks (traces) in a PCB file with their start/end positions, width, and layer. Useful for layout analysis and design review.
Run a remote agent task via the Solderable remote TUI server. Executes a prompt with access to KiCad tools and returns the agent's response, tool calls, and phases.
Compile a SKiDL circuit definition to a KiCad netlist (.net file). Requires SKiDL to be installed.
Lint a SKiDL circuit definition file for common errors (missing references, invalid part names, syntax errors). Returns a list of issues with severity levels.
Glob pattern syntax mentioned in descriptions but not formally specified. 'glob pattern, e.g. R*' lacks detail on escaping, case sensitivity, or character classes.
Descriptions lack WHEN-to-use context. E.g., kicad_overview says 'Use this FIRST' but other tools don't explain when they're preferred over alternatives or what prerequisites exist.