Loading ToolBench
Network scanning and CVE vulnerability detection MCP server with Nmap integration, multiple CVE database support, and per-user scan logging via dashboard
This server has significant structural issues across naming, descriptions, and error handling. All 5 tools are present with basic schemas, but descriptions are minimal and lack LLM-optimization guidance. Parameter descriptions exist but lack format/constraint details. No output schemas are documented. Error handling is absent, the code logs errors but returns no actionable recovery guidance to the LLM. Schemas are present but incomplete (missing minLength/maxLength, enum constraints, and format specifications). The server reads properly from the source code, so tool definitions are not inferred.
Perform a comprehensive Nmap scan with OS detection and vulnerability checking
Discover active hosts in a network subnet
Check if a specific port is open and identify the service running on it
Perform a quick Nmap scan on common ports and check for vulnerabilities
Search for known CVE vulnerabilities for a specific service and version
Tool descriptions are too brief (<80 chars) and lack guidance on when to use each tool vs. alternatives. 'Perform a quick Nmap scan on common ports...' does not explain the difference between quick_scan and full_scan or when an LLM should choose one over the other.
Parameter descriptions lack format and constraint details. 'Target IP address or hostname to scan' does not specify: must it be a valid IPv4/IPv6 (with regex pattern), or can it be any string? What about port ranges in port_check, is '22' valid, '22-80', '22,80,443'? Without these constraints, LLMs pass invalid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | - | v1 |
No output schemas documented for any tool. Code shows async functions return structured data (e.g., parse_nmap_services returns a list of dicts with 'port', 'name', 'product', 'version', etc.), but the tool definitions do not declare what fields the LLM should expect. LLMs cannot plan downstream calls or extract values without documented response structures.
Error handling is silent. The code catches exceptions (e.g., 'except Exception as e: logger.error(f"Vulners API error: {e}")') and logs them to stderr, but tool handlers return no error message to the LLM. When search_cves_vulners fails, the LLM receives an empty list and does not know whether to retry, use an alternative tool, or inform the user. No recovery guidance.
Input validation is missing. No code visible validates the 'target' parameter as a valid IP or hostname before passing to nmap, or the 'subnet' parameter as valid CIDR notation. LLMs will pass '999.999.999.999' or 'not-a-subnet' and the tool will fail with an unhelpful error. Validation should happen early with clear error messages.
Naming ambiguity: quick_scan vs full_scan. An LLM cannot tell from the names alone which is faster or when to use one over the other. The descriptions say 'quick' and 'comprehensive', but an LLM reasoning about a user's request 'scan this network quickly' might still pick the wrong one. Consider: scan_common_ports vs scan_all_ports, or add explicit guidance: 'Use quick_scan for rapid assessments (~30s); use full_scan for thorough audits (~5-10min)'.
The 'port' parameter in port_check is described as 'Port number to check' but the type is 'string'. It should be 'integer' (0-65535) or if 'string' is intentional (to support ranges like '22-80'), the description must explain the expected format and whether ranges are supported.
Tool composition issue: vulnerability_scan searches for CVEs by service name and optional version, but it is not clear how this chains with the port scanning tools. After quick_scan or full_scan identifies an open service (e.g., 'Apache 2.4.41'), the LLM must manually extract the product and version and pass them to vulnerability_scan. The response from port_check should include 'product' and 'version' fields explicitly so vulnerability_scan can be called with minimal manual mapping.
The 'version' parameter in vulnerability_scan is optional with a default of empty string, but an empty version string is not useful for CVE search. The description should explain: 'If omitted, search will look for the service name only (broader results, slower). Provide a version for faster, more targeted results.' This helps LLMs decide whether to include version info.