Loading ToolBench
MCP server for network device automation — give AI direct access to your network
Network-MCP provides 42 tools with generally clear naming conventions (verb_noun pattern dominant: get_, calculate_, parse_, etc.) and descriptions present for all tools. However, the server has significant gaps in parameter validation, output schema documentation, and error handling guidance. Parameter descriptions are minimal, most lack detail about valid ranges, formats, or dependencies. Output schemas are not documented in the visible source code. Error handling is limited; tools do not provide recovery guidance or next-step suggestions for LLMs. The tool definitions appear to be inferred from function signatures in config/ and core/ modules rather than explicitly registered with full MCP metadata, which caps individual tool scores at 50 for schema visibility. Security concerns are notable: no tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite many write operations (run_command, edit_config, netmiko_send_config, cache_* operations). Composition is reasonable, tools are single-responsibility (get device, send command, parse output), but the lack of structured error responses and missing output schema documentation limits downstream chaining.
Clear all cache entries
Remove a key from the cache
Get a value from the cache, or None if missing/expired
Store a value in the cache with optional custom TTL
Return cache statistics
Calculate optimal tunnel MTU and TCP MSS
Calculate subnet details from a CIDR address or IP+netmask pair
Basic health check for a containerlab container
CRITICAL: Output schemas not documented. Tools like get_device, get_config, discover_lldp_topology return data structures, but the visible source code does not specify what fields or types are in the response. LLMs cannot reliably extract nested data or plan follow-up tool calls without documented output schemas.
CRITICAL: Missing tool annotations (readOnlyHint, destructiveHint, idempotentHint). Many tools modify state (run_command, edit_config, netmiko_send_config, cache_set, cache_delete, cache_clear, reset_all_breakers) but have no annotations marking them as destructive. LLMs cannot reliably determine which tools are safe to retry or have irreversible side effects.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Check LLDP status on a device
Remove ANSI escape codes and trailing whitespace from CLI output
Get LLDP neighbors for a device
Discover full network topology via LLDP across all devices
Apply configuration via NETCONF <edit-config>
Get or create a circuit breaker for a device
Return the list of NETCONF capabilities advertised by the device
Return pre-calculated MTU/MSS for common deployment scenarios
Retrieve configuration via NETCONF <get-config>
Get a device by name
Get all devices of a specific type
Retrieve operational data via NETCONF <get>
Build a Scrapli connection dict for a device
Check if device is a Cisco IOS-XE device
Check if device is a containerlab device
Check if device is a Linux host
Simple event logger
Send a show command via Netmiko (runs in a thread)
Send configuration commands via Netmiko (runs in a thread)
Normalize interface name abbreviations to full form
Normalize MAC address to colon-separated format (aa:bb:cc:dd:ee:ff)
Normalize interface speed strings to Mbps
Parse raw CLI output into a list of dicts using ntc-templates
Parse CLI output using ntc-templates
Parse show ip interface brief output
Parse show ip route output into structured routes
Reset all circuit breakers
Execute command inside a containerlab container via docker exec
Send a single show command and return its output as a string
Send configuration commands to an IOS-XE device
SNMP GET for a single OID
Poll common metrics via SNMP (CPU, memory, interface counters)
SNMP WALK an OID subtree
Split a network into subnets with a given prefix length
HIGH: Parameter descriptions lack validation rules and constraints. Parameters like device_name, command, oid, tunnel_type, and platform are strings but lack guidance on valid values, lengths, formats, or examples. E.g., tunnel_type accepts 'gre, gre_ipsec, ipsec_tunnel, ipsec_transport, vxlan, wireguard' but this is prose in the description, not a formal enum constraint. LLMs cannot programmatically validate input and will hallucinate invalid values.
HIGH: No error handling guidance. Tools do not document error conditions, recovery steps, or next actions for LLMs. E.g., if run_command times out or fails, there is no error message telling the LLM to retry, check device connectivity, or try a different command. Agents will stall on failures without actionable feedback.
HIGH: Tool definitions inferred rather than explicitly registered. The source shows tool functions defined in core/ and config/ modules (e.g., config/devices.py, core/containerlab.py), but the MCP registration code in network_mcp_server.py uses a generic loop: 'for _entry in ALL_TOOLS: mcp.tool()(_entry["fn"])'. This pattern suggests tool metadata (schema, descriptions) may be extracted from docstrings or function signatures rather than explicitly defined MCP ToolDefinition objects. This caps schema visibility and score for all tools.
MEDIUM: Empty input schemas for some tools. Tools like discover_lldp_topology and reset_all_breakers have '{}' as input, no parameters, but lack documentation of side effects or what they return. discover_lldp_topology 'Discovers full network topology via LLDP across all devices', this is expensive and slow; the tool description should warn the LLM it may take time. reset_all_breakers has no description of what 'breakers' it resets or the impact.
MEDIUM: Minimal parameter descriptions. Most parameters have 1-2 line descriptions that do not explain allowed values, constraints, or dependencies. E.g., 'filter_xml' in get_config says 'Optional XML filter' but does not specify XPath syntax, schema, or examples. 'device_type' in scrapli_send_command says 'Device type (default: cisco_xe)' but does not list valid types. LLMs must guess or use defaults.
MEDIUM: No pagination or result limits documented. Tools like get_devices_by_type, snmp_walk, and discover_lldp_topology may return large result sets, but there is no mention of pagination, limits, or truncation. If a network has hundreds of devices or thousands of OIDs, LLMs will receive unbounded responses that blow context limits.
MEDIUM: write tools lack confirmation or dry-run capability. Destructive operations (run_command, edit_config, netmiko_send_config, scrapli_send_config, cache_clear, reset_all_breakers) execute immediately with no confirmation step. An LLM can accidentally delete configuration or reset circuit breakers without user approval. No dry-run or confirmation pattern is visible.
LOW: Generic parameter names without type hints. Parameters named 'name', 'type', 'command', 'data', 'raw', 'raw_output' are vague. E.g., 'name' could mean device_name, interface_name, or username. Descriptions disambiguate, but parameter suffixes (e.g., name → device_name, command → show_command) would improve clarity and reduce LLM confusion.