Loading ToolBench
MCP server that wraps ipspace/netlab as an engine to serve validated, lab-tested network device configs to LLMs.
netlab-mcp has 8 tools with clear naming (verb_noun pattern: generate_topology, render_config, query_compatibility, etc.). Descriptions are present and moderately detailed (avg ~120 chars), explaining WHAT each tool does. However, input schemas are partially visible but lack complete type definitions for all parameters. Output schemas are not documented. Error handling is minimal, tools return dicts with 'ok' flags but lack actionable recovery guidance. Tool composition is reasonable (each does one thing), but parameter descriptions could be more explicit about constraints, formats, and dependencies. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels (READ_ONLY vs WRITE vs IRREVERSIBLE).
Turn an intent + target platforms into a netlab topology YAML (parse-validated).
Return a previously lab-passed topology + rendered config for module+platform, if any.
Diagnose this host's lab readiness in one call — run this first when anything fails.
Index netlab's integration test topologies (real, maintained multi-platform scenarios).
What netlab *declares* a platform supports, overlaid with what was *observed* in the lab.
Render real per-device config from a netlab topology — offline, no containers.
Record a negative result (a combo that did not work) into the compatibility matrix.
Output schemas not documented. Tools return dicts with 'ok', 'error', 'data' fields, but LLMs cannot predict structure for downstream chaining. E.g., generate_topology returns 'topology_yaml' but render_config expects 'topology_yaml', this coupling is implicit, not documented.
Tool annotations missing. Tools have explicit risk levels (READ_ONLY, WRITE, IRREVERSIBLE) documented in the spec but not exposed via readOnlyHint/destructiveHint/idempotentHint. LLMs cannot infer safety properties and may retry irreversible operations.
Parameter descriptions lack explicit constraints. E.g., 'platforms' is described as 'NOS list, dut first (MVP free set: srlinux, frr, cumulus, vyos, linux)' but no enum constraint is declared. LLMs may hallucinate invalid platform names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2025-06-18+ | v2 |
Deploy a topology to containerlab, run netlab validate, and record the verdict.
Error handling lacks recovery guidance. Tools return {'ok': False, 'error': '...'} but do not suggest next steps. E.g., if generate_topology rejects platforms, it returns allowed_platforms list but does not say 'Call generate_topology again with one of these platforms.'
host_check has minimal description (50 chars: 'Diagnose this host's lab readiness in one call, run this first when anything fails.'). No parameter documentation (tool accepts empty input {}). LLMs cannot determine what it checks or when to call it.