Loading ToolBench
Network diagnostic tools for AI agents - connectivity testing and pcap analysis
Network MCP has 34 well-organized tools covering CIDR math, connectivity testing, and pcap analysis. Tool names follow verb_noun conventions (cidr_info, ip_in_subnet, ping, traceroute). Descriptions are present and moderately detailed, explaining WHEN to use tools (e.g., 'Tier 1 hero tool' for find_vlan_for_ip). However, the server has significant gaps: (1) most tools lack explicit parameter descriptions in the provided source, parameters are named but their purpose and constraints are not documented in the schema; (2) output schemas are not visible/documented in the code; (3) no evidence of error handling guidance; (4) no parameter validation documentation (ranges, enums, formats); (5) examples in descriptions (common pattern in descriptions like '192.168.0.0/24') risk LLM literal reuse. The capabilities tool shows the server is conscious of dependencies, but overall parameter and output documentation is below production standards. Tools are narrowly scoped (one per action), which is good, but lack the richness needed for confident LLM tool selection.
Analyze DNS traffic in a pcap file, extracting queries and responses.
Analyze throughput and bandwidth usage from a pcap file.
Lookup origin ASN for an IP address (BGP origin intel). Use this to quickly identify the ASN and prefix associated with an external IP.
Perform DNS lookups for multiple hostnames concurrently.
Ping multiple targets concurrently.
Check multiple ports on a target concurrently.
Report server/runtime capabilities and dependency status. Use this tool first when running with local/smaller models so the agent can decide which tools will work (e.g., whether `mtr` is installed) and what security/pcap guardrails are active.
Output schemas not documented in visible code. Tools return .model_dump() from Pydantic models, but the structure of those models is not shown in source excerpts. LLMs cannot plan downstream tool calls or extract specific fields without knowing the response shape.
Parameter descriptions are minimal or absent in the visible JSON schema definitions. Parameters like 'cidr', 'ip', 'target', 'vlan_map' are named but lack descriptions explaining what format is expected, what constraints apply (e.g., valid CIDR notation, IPv4 vs IPv6), or what happens if the input is invalid.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Detect overlaps/containment between CIDRs (high-leverage sanity check).
CIDR primitives (IPv4/IPv6): mask, wildcard, usable range, counts. NOC use cases: - Validate the mask math in a change ticket - Quickly answer "what's the usable range for this VLAN subnet?"
Summarize/aggregate routes from a list of CIDRs (IPv4/IPv6 collapsed separately).
Apply a custom Scapy filter expression to a pcap file.
Perform DNS lookup for a hostname or reverse lookup for an IP. Use this tool to resolve hostnames to IP addresses, find mail servers (MX), or perform reverse DNS lookups. Args: query: Hostname to lookup (e.g., "google.com") or IP for reverse lookup record_type: DNS record type - A, AAAA, CNAME, MX, TXT, NS, SOA, PTR (default: A) nameserver: Optional specific nameserver to query (e.g., "8.8.8.8") Returns: DNS records found with TTL and response time
Filter packets from a pcap file by protocol and direction.
Analyze a pcap file for TCP-level issues like retransmissions, timeouts, and resets.
Find which VLAN subnet(s) match an IP in a provided VLAN map. Tier 1 "hero tool": - "What VLAN does this IP belong to?" Example vlan_map: {"10": "192.168.10.0/24", "20": {"cidr": "192.168.20.0/24", "name": "Voice"}}
Get the ARP (Address Resolution Protocol) table showing IP to MAC mappings.
List all active network connections on the system.
Extract and analyze conversations (flows) from a pcap file.
Get the DNS servers configured on the local system.
Get all network interfaces on the local system with IP addresses and status.
Get the protocol hierarchy (OSI layer stack) from packets in a pcap file.
Get the public IP address of the system.
Get the system routing table.
Check if an IP is in a subnet and whether it is a usable host address. NOC use cases: - "Is this IP in this VLAN subnet?" - Catch network/broadcast mistakes (/24 .0 / .255) quickly
Check if an IP belongs to a VLAN (1 subnet per VLAN) using a provided VLAN map. If it does NOT match, this tool will attempt a best-guess VLAN match to help Tier 1/2 triage. Example vlan_map: {"20": {"cidr": "10.10.20.0/24", "name": "Voice"}, "50": "10.10.50.0/24"}
Run MTR (My Trace Route) to analyze the network path with statistics.
Analyze a pcap file and provide a summary including packet counts, protocols, and conversations.
Ping a host to check connectivity and measure latency. Use this tool to verify if a host is reachable and measure round-trip latency. Returns packet loss percentage and latency statistics (min/avg/max). Args: target: Hostname or IP address to ping (e.g., "google.com" or "8.8.8.8") count: Number of ICMP packets to send (default: 4) timeout: Timeout in seconds for each packet (default: 5) Returns: Ping results including success status, packet loss, and latency statistics
Allocate VLAN subnets from a parent IPv4 block (deterministic, no network calls). Each requirement is 1 subnet per VLAN. Use either hosts (alias: needed_hosts) OR prefix (alias: desired_prefix). Example: parent_cidr="10.0.0.0/23" requirements=[ {"vlan_id": 10, "name": "Users", "hosts": 120}, {"vlan_id": 20, "name": "Voice", "hosts": 60}, {"vlan_id": 30, "name": "Printers", "prefix": 26}, ]
Check if a specific port is open/listening on a target host.
WHOIS-style lookup using RDAP (Registration Data Access Protocol). Use this to identify who owns an IP range or domain and to get registration metadata.
Split a CIDR into equal-size child subnets (by new_prefix or power-of-two count).
Trace the network path to a destination, showing each hop. Use this tool to understand the network path between you and a target, identify where latency is introduced, or find where packets are being dropped. Args: target: Hostname or IP address to trace (e.g., "google.com") max_hops: Maximum number of hops to trace (default: 30) timeout: Timeout in seconds for each probe (default: 5) Returns: Path analysis with hop-by-hop details including IP, hostname, and latency
Validate a simple VLAN map (1 subnet per VLAN) and surface overlaps. Input formats supported: - Shorthand: {"10": "192.168.10.0/24"} - Structured: {"10": {"cidr": "192.168.10.0/24", "name": "Management"}}
No error handling or recovery guidance. Tools do not document what errors can occur (e.g., 'Invalid CIDR format', 'Host unreachable', 'Timeout'), how to classify them (retryable vs user-fixable), or what the LLM should do next. A timeout in traceroute is indistinguishable from a host being offline.
Examples in descriptions (e.g., 'e.g., 192.168.0.0/24' in cidr_info, 'e.g., google.com' in ping) risk LLMs treating them as literal required formats or values. Should use formal constraints (enum, pattern, format) instead of example strings.
Batch tools (batch_ping, batch_port_check, batch_dns_lookup) do not document per-item success/failure handling or pagination. If 10 of 100 targets timeout, does the tool return 90 successes + 10 failures, or does it fail entirely? Are results paginated if large?
Numeric parameters lack ranges. 'count' in ping defaults to 4 but max is unstated (could be 1 or 10000). 'timeout' in traceroute and mtr is documented but ranges are absent. LLMs may pass absurd values (timeout=999999) that break APIs or cause hangs.
Local system tools (get_interfaces, get_routes, get_dns_config, get_arp_table, get_connections) have trivial descriptions (1-2 sentences) that don't explain when to call them, what to expect, or how to interpret results. A NOC engineer wouldn't know 'get_arp_table' reveals local ARP cache vs network-wide ARP tables.