Loading ToolBench
Network analysis via tshark -- MCP server for capture, analysis, and profiling
The server provides 8 network analysis tools with reasonable naming convention (verb_noun pattern). Tool descriptions are present and informative, averaging ~150 characters, which is within the baseline range (p10=34, p90=392). However, critical gaps exist: (1) Tool #8 (wireshark_dns_analysis) has a truncated description ending mid-sentence ('Returns query types, response codes, response'). (2) Parameter descriptions are inconsistent, some tools provide detailed parameter docs inline in the tool description, but the actual input schemas lack individual parameter descriptions in the JSON schema itself. (3) Output schemas are not documented anywhere in the visible code, we can see parameter input schemas but no documentation of what fields are returned. (4) Error handling strategies are not visible in the code snippets provided. (5) Security considerations for filter injection are addressed in the executor module via sanitize_param(), which is good, but no mention of permission gating for the WRITE-risk wireshark_capture tool. Overall, this is a solid mid-tier implementation with good naming and some parameter validation, but falls short of A-grade due to missing schema documentation, incomplete descriptions, and unclear error handling.
Capture live network traffic to a pcapng file. Args: interface: Network interface name or number (from wireshark_list_interfaces) duration: Capture duration in seconds (default: from config, typically 120) capture_filter: BPF capture filter (e.g., 'host 192.168.1.1', 'port 80') max_packets: Maximum packets to capture (default: from config) output_name: Output filename (default: auto-generated with timestamp) Returns path to the captured pcapng file.
Check if tshark can capture packets on this host. Tests tshark availability, version, and whether we have capture privileges. On Windows, typically requires Npcap installed. On Linux, may require root or cap_net_raw capability.
Get conversation table showing who is talking to whom. Returns top conversations sorted by total bytes transferred. Args: pcap_file: Path to pcapng/pcap file layer: Protocol layer -- 'ip', 'tcp', or 'udp' (default: ip) top_n: Number of top conversations to return (default: from config) display_filter: Optional display filter
Analyze DNS traffic in a capture file. Returns query types, response codes, response
Get endpoint table showing unique addresses with traffic volumes. Returns top endpoints sorted by total bytes. Args: pcap_file: Path to pcapng/pcap file layer: Protocol layer -- 'ip', 'tcp', or 'udp' (default: ip) top_n: Number of top endpoints to return (default: from config) display_filter: Optional display filter
Tool #8 (wireshark_dns_analysis) has truncated description ending mid-sentence. Description reads 'Returns query types, response codes, response', incomplete, under 20 chars of semantic content. This violates the requirement that descriptions be 10 - 1024 chars and answer WHAT/WHEN/WHY. LLM cannot determine selection criteria.
Output schemas are not documented in visible code. Tools return results, but no structured schema documentation exists for what fields each tool returns (e.g., does wireshark_conversations return 'src_ip', 'dst_ip', 'bytes_sent', 'packets'?). LLMs cannot plan downstream tool chains without knowing output structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
List saved pcapng/pcap capture files in the capture directory. Returns list of files with sizes and modification times.
List available network capture interfaces. Returns numbered list of interfaces with descriptions. Useful for finding the right interface name before starting a capture.
Get protocol distribution tree from a capture file. Shows what protocols are in the capture and their relative proportions. Returns structured hierarchy with frame counts, byte counts, and percentages. Args: pcap_file: Path to pcapng/pcap file display_filter: Optional display filter (e.g., 'ip.addr == 192.168.1.1')
Parameter descriptions embedded in tool description rather than in JSON schema. While wireshark_capture tool description includes 'Args: interface, duration, capture_filter, max_packets, output_name', these same parameters in the input schema JSON lack individual 'description' fields. Schema should contain both type AND description for each parameter. Currently, the schema has type info but parameter-level descriptions are missing from the JSON schema object.
wireshark_capture tool (WRITE risk) lacks permission gating and confirmation step. No evidence of a dry-run mode, confirmation request, or permission check before executing a live capture. Given that this tool writes to disk and could consume bandwidth, agents should be prompted to confirm or have explicit permission before execution.
Error handling strategy not visible. Code shows input sanitization via regex check for dangerous characters (semicolons, pipes, backticks), but no guidance on how tools respond to invalid inputs, network timeouts, or missing files. No evidence of recovery guidance like 'Try search_captures() first if file not found' or error categorization (retryable vs user-fixable).
Parameter 'layer' in wireshark_conversations and wireshark_endpoints accepts 'ip', 'tcp', 'udp', 'eth', but no enum constraint visible in schema. Schema shows type='string' with description, but no 'enum' array limiting values. LLMs can hallucinate invalid layers like 'ssl', 'http', or 'ipv6'.
No pagination or result limit documented. Tools like wireshark_conversations and wireshark_endpoints return 'top_n' results, but no mention of default behavior if top_n is omitted, what happens if top_n=1000000, or how to iterate through remaining results. Tools should enforce max result caps.